Binomial Multifractal Curve Fitting for View Size Estimation in OLAP

On Line Analytical Processing (OLAP) aims at gaining useful information quickly from large amounts of data residing in a data warehouse. To improve the quickness of response to queries, pre-aggregation is a useful strategy. However, it is usually impossible to pre-aggregate along all combinations of the dimensions. The multi-dimensional aspects of the data lead to combinatorial explosion in the number and potential storage size of the aggregates. We must selectively pre-aggregate. Cost/benefit analysis involves estimating the storage requirements of the aggregates in question. We introduce a useful diagram illustrating rows in the fact table versus rows in an aggregate. We demonstrate predictable trends in these diagrams. We present an original curve-fitting approach to the problem of estimating the number of rows in an aggregate. We test the curve-fitting algorithm empirically against three published algorithms, and conclude the curve-fitting approach is the most accurate at small sample sizes.