Beyond data management: How to foster data exploitation and better science?

The advent of large scale, high-bandwidth on-line access and generously powered ocean floor observatories were predicted to transform the way ocean science is conducted. They have lived up to this promise. New cabled observatories have sometimes in excess of 100 different instruments semipermanently installed in areas of scientific interest. The instruments represent hundreds of different variables measured at a high frequency (often once per second or more) on a 24/7/365 basis, thereby generating large data volumes that accumulate in archives. Not only are we talking about large data volumes, we are also facing many different data types as instruments are often preprocessing data and assembling them in complex data structures or matrices. For practical purposes, three major data types can be described based on the duty cycles of the instruments producing them: · scalar data coming from single point measurement sensors on a periodic basis with a sampling frequency <; 10Hz; · complex data structures, often produced by active acoustic instruments such as ADCPs and current profilers that produce multi-dimensional matrices typically at a rate ≤ 1 Hz · streams that are produced by passive sensors, continuously sampling their environments at high frequencies (≥ ~20Hz) such as cameras and hydrophones, seismometers, etc. The complexity also resides in that many of the instruments are in some way mapping their wider environment and not just their immediate vicinity. Therefore the matrices they sent back represent their own specific encoding of spatial data. Examples here include ADCPs, sonars, but also vehicles moving in one, two or three dimensions (vertical profiler, crawlers, gliders, AUVs, etc.) The instruments and sensors are often of a legacy type, from different vendors, with proprietary formats and usually are not optimized to work in connected, powered environment. The stated goal of the observatories is to enable not onl- science in a particular discipline, but to foster multi-disciplinary experiments, to support real-time science as well as to provide a longitudinal view on the evolutions of the oceans. The above considerations represent significant challenges not only for the scientists, but also for data providers as they need to be in a position to support: · many different science disciplines · scientists not used to dealing with vast amounts of data · scientists not necessarily used to data from instrumentation they are not always familiar with · users still expecting to be able to just “download a couple of years of data” At Ocean Networks Canada, those challenges have been addressed in a number of ways, oftentimes creative and sometimes disruptive. With the support of CANARIE, Canada's advanced research and innovation network, we have adopted the Web 2.0 concepts of participation and contribution. Those two concepts are very powerful and are at the origin of wikipedia, FaceBook, Twitter and, to some extent, massively parallel computer games: they allow individuals to contribute/share content and to participate in groups of like-minded citizens in activities around a specific topic or theme for a period of time. In this contribution, we are describing the innovations that are now available to users of the NEPTUNE Canada cabled network access interfaces and how they contribute to address many of the challenges that scientists face in dealing with those first Big Science infrastructures for the study of the oceans. They include a data visualization utility that allows plots created to be posted and shared with other users, a data search tool that offers a multitude of data formatting and averaging/decimation options, a real-time camera control interface that allows multiple users to simultaneously use the instruments and chat among themselves, regardless of where they are each located on the planet, a tool that encourages users