Data quality: Experiences and lessons from operationalizing big data

Data quality issues pose a significant barrier to operationalizing big data. They pertain to the meaning of the data, the consistency of that meaning, the human interpretation of results, and the contexts in which the results are used. Data quality issues arise after organizations have moved past clear-cut technical solutions to early bottlenecks in using data. Left unaddressed, such issues can and have led to high profile missteps, and raise doubts about the data-driven world view altogether. In this paper, we share real-world case studies of tackling data quality challenges across industry verticals. We present initial ideas on how to systematically address data quality issues via technology. The success of operationalizing big data will depend on the quality of data involved, and whether such data causes uncertainty and disruptions, or delivers genuine knowledge and value.