[Journal First] Inference of Development Activities from Interaction with Uninstrumented Applications

This paper is published in Journal of Empirical Software Engineering (DOI: 10.1007/s10664-017-9547-8). Studying developersu0027 behavior is crucial for designing effective techniques and tools to support developersu0027 daily work. However, there are two challenges in collecting and analyzing developersu0027 behavior data. First, instrumenting many software tools commonly used in real work settings (e.g., IDEs, web browsers) is difficult and requires significant resources. Second, the collected behavior data consist of low-level and fine-grained event sequences, which must be abstracted into high-level development activities for further analysis. To address these two challenges, we first use our ActivitySpace framework to improve the generalizability of the data collection. Then, we propose a Condition Random Field (CRF) based approach to segment and label the developersu0027 low-level actions into a set of basic, yet meaningful development activities. To evaluate our proposed approach, we deploy the ActivitySpace framework in an industry partneru0027s company and collect the real working data from ten professional developersu0027 one-week work. We conduct an experiment with the collected data and a small number of initial human-labeled training data using the CRF model and the other three baselines (i.e., a heuristic-rules based method, a SVM classifier, and a random weighted classifier). The proposed CRF model achieves better performance (i.e., 0.728 accuracy and 0.672 macro-averaged F1-score) than the other three baselines.