SciSD: Novel Subgroup Discovery Over Scientific Datasets Using Bitmap Indices

Subgroup discovery is a broadly applicable exploratory tec hnique, which identifies interesting subgroups with respect to a pro pe ty of interest. While there is clearly a need to apply this method t o iscover interesting patterns from scientific datasets compri sing largescale arrays, the existing algorithms primarily apply to re lational datasets. In this paper, we present a novel algorithm, SciSD , for exhaustive but efficient subgroup discovery over array-bas ed scientific datasets, in which all attributes are numeric. Our al gorithm handles a key challenge associated with array data, which is that a subgroup identified over array data can be described based o n value-based and/or dimension-based attributes. To reduce the computational costs, our SciSD algorithm extensively uses bit map indices (and fast bitwise operations on them). We demonstrate both high efficiency and effectiveness of our algorithm by using m ultiple real-life datasets.