Reservoir Sampling over Joins
arxiv(2024)
摘要
Sampling over joins is a fundamental task in large-scale data analytics.
Instead of computing the full join results, which could be massive, a uniform
sample of the join results would suffice for many purposes, such as answering
analytical queries or training machine learning models. In this paper, we study
the problem of how to maintain a random sample over joins while the tuples are
streaming in. Without the join, this problem can be solved by some simple and
classical reservoir sampling algorithms. However, the join operator makes the
problem significantly harder, as the join size can be polynomially larger than
the input. We present a new algorithm for this problem that achieves a
near-linear complexity. The key technical components are a generalized
reservoir sampling algorithm that supports a predicate, and a dynamic index for
sampling over joins. We also conduct extensive experiments on both graph and
relational data over various join queries, and the experimental results
demonstrate significant performance improvement over the state of the art.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要