Mining For Functional Dependencies Using Shared Radix Trees In Many-Core Multi-Threaded Systems

We consider the problem of mining for functional dependencies in relational databases. Intermediate data structures, although simple, explode in size and a solution is proposed using radix trees to reduce memory utilization. Parallelism is further applied in a Multi-Core computer to further speedup the process. Because bit-permutations are the basis of the construction of a binary intermediate matrix, radix trees reduce the memory usage 10 times. Multi-Threading the construction and processing of the intermediate data leads to a concurrent computing average-overtime of 63% on an equivalent speedup of 6.3 on a system with 12 cores, 256 GB of memory and 1 TB SSD.