On Join Sampling and the Hardness of Combinatorial Output-Sensitive Join Algorithms

We present a dynamic index structure for join sampling. Built for an (equi-) join Q - let IN be the total number of tuples in the input relations of Q - the structure uses (O) over tilde( IN) space, supports a tuple update of any relation in (O) over tilde (1) time, and returns a uniform sample from the join result in (O) over tilde (IN rho* /max{1, OUT}) time with high probability (w.h.p.), where OUT and rho* are the join's output size and fractional edge covering number, respectively; notation (O) over tilde(.) hides a factor polylogarithmic to IN. We further show how our result justifies the (O) over tilde (IN rho*) running time of existing worst-case optimal join algorithms (for full result reporting) even when OUT << IN rho*. Specifically, unless the combinatorial K-clique hypothesis is false, no combinatorial algorithms (i.e., algorithms not relying on fast matrix multiplication) can compute the join result in O( IN rho*-epsilon) time w.h.p. even if OUT = IN epsilon, regardless of how small the constant epsilon > 0 is.