PIQUE: Progressive Integrated QUery Evaluation with Pay-As-You-Go Enrichment

Unstructured data such as text, images, video, and sensor observations need to be enriched (i.e., annotated with tags) prior to be effectively queried or analyzed. Data enrichment (that often uses expensive machine learning and/or signal processing techniques) cannot be performed in its entirety as a pre-processing step at the time of data ingestion. Enriching data as a separate offline step after ingestion makes it unavailable for analysis during the period between ingest and enrichment. To bridge such a gap, this paper explores a novel approach, entitled PIQUE, that jointly enriches and queries the data in order to support interactive exploratory analysis. PIQUE strives to enrich the right data to the right degree in the context of the query in order to maximize the quality of the answers in a pay-as-you-go fashion. The experimental results on real datasets (images, tweets) show that PIQUE performs significantly better (~three times better) compared to the baseline approaches.