Imagers as sensors: using visible light images to measure natural phenomena

There exist many natural phenomena where direct measurement is either impossible or extremely invasive. The signals that biologists wish to measure about these phenomena have a variety of characteristic forms. We consider three specific forms: continuous signals (like CO2 flux), discrete signals (like pollinator presence on a flower), or discrete spatio-temporal signals (like pollinators visiting a field of flowers). We propose using imagers as sensors by constructing a set of template procedures that uses images to obtain estimates of such phenomena. These procedures, composed of state-of-the-art computer vision, image processing, and statistical learning and sampling algorithms, are evaluated in the context of specific applications and shown to be general through their limited assumptions. We describe various methodologies that can be used to isolate changes in the subject from changes in the local environment, making existing algorithms robust to field conditions present in real applications. Finally, we rigorously define the proposed procedures and evaluate their accuracy on real data gathered in the field, augmented by simulation when required. Our goal is to influence future sensing system design through the identification of mechanisms that regularize the input to these procedures, making subsequent processing simpler. For each form of signal we consider, we apply our approach to a specific application. Our procedure for predicting continuous signals is applied to the prediction of CO2 flux from a moss plant, measurements that would otherwise require encasing the plant in an air-tight box. We consider pollinator occupancy of a flower, data that would otherwise be collected manually by humans in the field, as a representative instance of discrete signals. Scaling pollinator occupancy measurement to an entire field of flowers is the application we consider when evaluating our procedure for collecting discrete spatial temporal signals.