Enabling the Discovery of Manual Processes Using a Multi-modal Activity Recognition Approach.

The analysis of business processes using process mining requires structured log data. Regarding manual activities, this data can be generated from sensor data acquired from the Internet of Things. The main objective of this paper is the development and evaluation of an approach which recognizes and logs manually performed activities, enabling the application of established process discovery methods. A system was implemented which uses a body area network, image data of the process environment and feedback from the executing workers in case of uncertainties during detection. Both feedback and image data are acquired and processed during process execution. In a case study in a laboratory environment, the system was evaluated using an example process. The implemented approach shows that the inclusion of image data of the environment and user feedback in ambiguous situations during recognition generate log data which well represent actual process behavior.