Data-driven and On-Demand Conceptual Modeling.

The data infrastructure of an organization involves data stored in different models and systems, spreadsheets, files, data mining models, streams, even output of stand-alone programs. In many cases, it is necessary to build on demand, agilely a virtual schema on top of this data infrastructure. We propose a novel graph-based conceptual model, called data virtual machine (DVM), which is designed bottom-up. DVM nodes represent attribute domains and edges represent programs that (through their output) relate values between domains. The data engineer defines a collection of such programs – which can be done quickly, on demand – and the schema, a graph, is derived. This is the inverse process of traditional approaches, where the schema is a priori set and the data engineer designs how to “fit” existing data to this schema. In a DVM, nodes are entities and attributes at the same time, which is useful in many settings. In this paper we also formalize the translation of the relational model to DVM and we present an extended use case that showcases the novelty of the DVM model and the new capabilities it comes with.