Effective Metadata Management in Federated Sensor Networks

As sensor networks become increasingly popular, heterogeneous sensor networks are being interconnected into federated sensor networks and provide huge volumes of sensor data to large user communities for a variety of applications. Effective metadata management plays a crucial role in processing and properly interpreting raw sensor measurement data, and needs to be performed in a collaborative fashion. Previous data management work has concentrated on metadata and data as two separate entities and has not provided specific support for joint real-time processing of metadata and sensor data. In this paper we propose a framework that allows effective sensor data and metadata management based on real-time metadata creation and join processing over federated sensor networks. The framework is established on three key mechanisms: (i) distributed metadata joins to allow streaming sensor data to be efficiently processed with their associated metadata, regardless of their location in the network, (ii) automated metadata generation to permit users to define monitoring conditions or operations for extracting and storing metadata from streaming sensor data, (iii) advanced metadata search utilizing various techniques specifically designed for sensor metadata querying and visualization. This framework is currently deployed and used as the backbone of a concrete application in environmental science and engineering, the Swiss Experiment, which runs a wide variety of measurements and experiments for environmental hazard forecasting and warning.