Examining task engagement in sensor-based statistical models of human interruptibility

CHI, pp. 331-340, 2005.

Cited by: 134|Bibtex|Views29|Links
EI
Keywords:
social engagementexamining task engagementhuman interruptibilitytask engagementrealistic programming taskMore(10+)
Weibo:
We have presented our work to more carefully explore task engagement in sensor-based statistical models of human interruptibility by studying the interruption of programmers working on a realistic programming task

Abstract:

The computer and communication systems that office workers currently use tend to interrupt at inappropriate times or unduly demand attention because they have no way to determine when an interruption is appropriate. Sensor?based statistical models of human interruptibility offer a potential solution to this problem. Prior work to examine ...More

Code:

Data:

0
Introduction
  • Modern office workers increasingly find computing and communication systems at the core of their everyday work experience.
  • At any given point in time, a person might be notified of the arrival of a new email, receive an instant message from a colleague, be reminded by a handheld computer of an upcoming appointment, receive a phone call on their office or mobile phone, and be involved in a face-to-face interaction with a colleague
  • Any one of these demands for attention can be addressed relatively but simultaneous or repeated demands can quickly become disruptive.
  • Email clients and other systems could consider the importance of a notification relative to a person’s current interruptibility, perhaps deferring or adjusting the salience of the notification when a person is busy
Highlights
  • Modern office workers increasingly find computing and communication systems at the core of their everyday work experience
  • Using an approach that we developed for our original studies of predicting human interruptibility, we examine how programmers respond to interruptions while they are programming and how statistical models can be used to predict their interruptibility
  • We develop a statistical model of the interruptibility of programmers that is based on low-level input events in a development environment
  • In order to control the effects of social engagement and focus on task engagement, we studied programmers in a laboratory environment completing a realistic programming task while being subjected to interruptions
  • We have presented our work to more carefully explore task engagement in sensor-based statistical models of human interruptibility by studying the interruption of programmers working on a realistic programming task
  • Its performance is significantly better than the base performance typical of current systems that generally assume people are always interruptible
  • This work contributes an evaluation of a model based on low-level system events in a development environment, finding that it can distinguish situations where a programmer is interruptible from other situations with an accuracy of 71.8%, significantly better than the base accuracy of 58.5% accuracy typical of current systems that generally assume that a programmer is always interruptible and significantly better than a model that assumes programmers are non-interruptible if they recently edited their code
Methods
  • In order to control the effects of social engagement and focus on task engagement, the authors studied programmers in a laboratory environment completing a realistic programming task while being subjected to interruptions.
  • The Paint program, shown in Figure 1, is a 503-line program consisting of nine classes implemented in Java with the Swing toolkit.
  • It provides basic paint support, allowing users to draw, erase, clear, and undo colored strokes on a white canvas.
  • Participants were given the Paint program and allowed 70 minutes to address five requests.
Results
  • Its performance is significantly better than the base performance typical of current systems that generally assume people are always interruptible.
  • With an overall accuracy of 71.8%, this model is significantly more accurate than the base performance of 58.5% typical of current systems, which generally assume that people are always interruptible (χ2(1, 950) = 18.4, p < .001).
  • Because considering a programmer non-interruptible if they have recently edited text is a more satisfying baseline than assuming a programmer is always interruptible, the authors note that the full model performs significantly better than a.
Conclusion
  • The authors have presented the work to more carefully explore task engagement in sensor-based statistical models of human interruptibility by studying the interruption of programmers working on a realistic programming task.
  • The authors' approach to developing sensor-based statistical models of human interruptibility has yielded strong results in both the prior work and this work, but the authors remain interested in the possibility of refining or improving it.
  • The authors intend to integrate the results of this work and the results of the prior work, with the goal of developing more accurate models of human interruptibility
Summary
  • Introduction:

    Modern office workers increasingly find computing and communication systems at the core of their everyday work experience.
  • At any given point in time, a person might be notified of the arrival of a new email, receive an instant message from a colleague, be reminded by a handheld computer of an upcoming appointment, receive a phone call on their office or mobile phone, and be involved in a face-to-face interaction with a colleague
  • Any one of these demands for attention can be addressed relatively but simultaneous or repeated demands can quickly become disruptive.
  • Email clients and other systems could consider the importance of a notification relative to a person’s current interruptibility, perhaps deferring or adjusting the salience of the notification when a person is busy
  • Methods:

    In order to control the effects of social engagement and focus on task engagement, the authors studied programmers in a laboratory environment completing a realistic programming task while being subjected to interruptions.
  • The Paint program, shown in Figure 1, is a 503-line program consisting of nine classes implemented in Java with the Swing toolkit.
  • It provides basic paint support, allowing users to draw, erase, clear, and undo colored strokes on a white canvas.
  • Participants were given the Paint program and allowed 70 minutes to address five requests.
  • Results:

    Its performance is significantly better than the base performance typical of current systems that generally assume people are always interruptible.
  • With an overall accuracy of 71.8%, this model is significantly more accurate than the base performance of 58.5% typical of current systems, which generally assume that people are always interruptible (χ2(1, 950) = 18.4, p < .001).
  • Because considering a programmer non-interruptible if they have recently edited text is a more satisfying baseline than assuming a programmer is always interruptible, the authors note that the full model performs significantly better than a.
  • Conclusion:

    The authors have presented the work to more carefully explore task engagement in sensor-based statistical models of human interruptibility by studying the interruption of programmers working on a realistic programming task.
  • The authors' approach to developing sensor-based statistical models of human interruptibility has yielded strong results in both the prior work and this work, but the authors remain interested in the possibility of refining or improving it.
  • The authors intend to integrate the results of this work and the results of the prior work, with the goal of developing more accurate models of human interruptibility
Related work
  • Field studies of interruptions and how people perceive their interruptibility have informed our work, but do not directly inform the deployment of sensor-based statistical models of human interruptibility. As mentioned in our introduction, Hudson et al studied the perceptions of interruptions held by managers in a research organization, finding that some managers consider interruptions to be such a problem that they physically move away from their computers or even away from their offices in order to obtain uninterrupted working time [14]. Perlow found a self-perpetuating cycle of interruptions in the workplace, wherein workers in danger of missing a deadline interrupt other workers with requests, which then causes the interrupted workers to fall behind in their own work, leading them to then interrupt others [23]. Field studies that more directly inform sensor-based statistical models of human interruptibility have typically reported findings primarily related to social engagement. In our own prior work, we have used the approach presented in this paper to study the interruptibility of office workers in their normal working environments [8, 9, 15]. In that work, we measured interruptibility using an experience sampling technique, prompting workers to report their interruptibility at random intervals approximately once per hour. We collected data for several weeks from each participant, and showed that real sensors could support models of their interruptibility with accuracies as good as or better than human observers. These results were largely dominated by social engagement, helping to motivate the work presented in this paper. Horvitz and Apacible also directly studied interruptibility, asking workers to retrospectively review several hours of collected recordings to provide labels of their interruptibility, then examining models of these labels based on system events, perceptual analyses of audio and video streams, and electronic calendar entries [11]. They do not explicitly differentiate between sensors related to task engagement and social engagement, but the perceptual systems and electronic calendar analyses on which their discussion is focused seem to be primarily related to social engagement. Other work has studied interruptibility in laboratory tasks, but without the goal of enabling sensor-based statistical models of human interruptibility. For example, Czerwinski et al examined interruptions by instant message notifications during some relatively simple list-browsing and office software tasks, finding that even ignored notifications can be disruptive [4, 5]. Gillie and Broadbent studied resumption of a task after different types of interruptions, also finding that the externalization of working memory into the state of the task meant that very few errors resulted from interruptions [10]. McFarlane points out that models of interruptibility can be used as part of a mediated approach to coordinating interruptions, but his studies of interruptions and task performance compare strategies for coordinating interruptions, rather than informing the development of sensor-based statistical models of human interruptibility [21]. Robertson et al studied interruption coordination specifically in the context of spreadsheet programming, finding that negotiated coordination lead to better task performance, but their work does not inform the development of statistical models of the interruptibility of programmers [24]. A variety of systems have explored concepts related to interruptibility. The Priorities system, by Horvitz et al, considers patterns of prior device access to reason about when a person is likely to be available on a given device, such as a personal computer or a mobile phone, and can consider the apparent importance of a message in deciding whether to forward it to a mobile device [12]. The Coordinate system, also by Horvitz et al, adds perceptual sensors based on audio and video streams, together with analyses of electronic calendar entries, to reason about the presence and availability of people [13]. Begole et al analyzed logs of presence in their Awarenex system and developed a method to automatically extract temporal patterns, such as recurring meetings [2, 3]. Kern and Schiele examined the ability of wearable sensors to detect different contexts and activities that they argue relate to interruptibility [16]. A major difference between our work and these systems is our use of an evaluation based on an explicit measure of interruptibility. Evaluations of other systems typically examine the ability of a system to recognize particular contexts, but do not explicitly evaluate how those contexts actually relate to interruptibility.
Funding
  • This work was funded in part by DARPA, by the NASA High Dependability Computing Program under cooperative agreement NCC-2-1298, by an NDSEG fellowship, by an AT&T Labs fellowship, and by the National Science Foundation under grants CCR-03244770, IIS-0329090, IIS-0121560, and IIS-0325351
Reference
  • 2. Begole, J.B., Tang, J.C. and Hill, R. (2003) Rhythm Modeling, Visualizations, and Applications. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2003), 11-20.
    Google ScholarLocate open access versionFindings
  • 3. Begole, J.B., Tang, J.C., Smith, R.B. and Yankelovich, N. (2002) Work Rhythms: Analyzing Visualizations of Awareness Histories of Distributed Groups. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2002), 334-343.
    Google ScholarLocate open access versionFindings
  • 4. Cutrell, E., Czerwinski, M. and Horvitz, E. (2001) Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance. Proceedings of the IFIP Conference on Human-Computer Interaction (INTERACT 2001), 263-269.
    Google ScholarLocate open access versionFindings
  • 5. Czerwinski, M., Cutrell, E. and Horvitz, E. (2000) Instant Messaging and Interruptions: Influence of Task Type on Performance. Proceedings of the Australian Conference on Computer-Human Interaction (OZCHI 2000), 356-361.
    Google ScholarLocate open access versionFindings
  • 6. Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, 39 (1). 1-38.
    Google ScholarLocate open access versionFindings
  • 7. Duda, R.O. and Hart, P.E. (1973) Pattern Classification and Scene Analysis. John Wiley and Sons.
    Google ScholarFindings
  • 8. Fogarty, J., Hudson, S., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J. and Yang, J. (2004) Predicting Human Interruptibility with Sensors. To Appear, ACM Transactions on Computer-Human Interaction (TOCHI). 9. Fogarty, J., Hudson, S. and Lai, J. (2004) Examining the Robustness of Sensor-Based Statistical Models of Human Interruptibility. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2004), 207-214.
    Google ScholarLocate open access versionFindings
  • 10. Gillie, T. and Broadbent, D. (1989) What Makes Interruptions Disruptive? A Study of Length, Similarity, and Complexity. Psychological Research, 50. 243-250.
    Google ScholarLocate open access versionFindings
  • 11. Horvitz, E. and Apacible, J. (2003) Learning and Reasoning about Interruption. Proceedings of the International Conference on Multimodal Interfaces (ICMI 2003), 20-27.
    Google ScholarLocate open access versionFindings
  • 12. Horvitz, E., Jacobs, A. and Hovel, D. (1999) AttentionSensitive Alerting. Proceeding of the Conference on Uncertainty and Artificial Intelligence (UAI 1999), 305-313.
    Google ScholarLocate open access versionFindings
  • 13. Horvitz, E., Koch, P., Kadie, C.M. and Jacobs, A. (2002) Coordinate: Probabilistic Forecasting of Presence and Availability. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI 2002), 224-233.
    Google ScholarLocate open access versionFindings
  • 14. Hudson, J.M., Christensen, J., Kellogg, W.A. and Erickson, T. (2002) "I'd be overwhelmed, but it's just one more thing to do": Availability and Interruption in Research Management. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2002), 97-104.
    Google ScholarLocate open access versionFindings
  • 15. Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J. and Yang, J. (2003) Predicting Human Interruptibility with Sensors: A Wizard of Oz Feasibility Study. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2003), 257-264.
    Google ScholarLocate open access versionFindings
  • 16. Kern, N. and Schiele, B. (2003) Context-Aware Notification for Wearable Computing. Proceedings of the IEEE International Symposium on Wearable Computing (ISWC 2003).
    Google ScholarLocate open access versionFindings
  • 17. Ko, A.J. and Myers, B. (2004) A Framework and Methodology for Studying the Causes of Software Errors in Programming Systems. To Appear, Journal of Visual Languages and Computing.
    Google ScholarLocate open access versionFindings
  • 18. Kohavi, R. and John, G.H. (1997) Wrappers for Feature Subset Selection. Artificial Intelligence, 97 (1-2). 273-324.
    Google ScholarLocate open access versionFindings
  • 19. Langley, P. and Sage, S. (1994) Induction of Selected Bayesian Classifiers. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI 1994), 399-406.
    Google ScholarLocate open access versionFindings
  • 20. Lemaire, P., Abdi, H. and Faylo, M. (1996) The Role of Working Memory Resources in Simple Cognitive Arithmetic. European Journal of Cognitive Psychology, 8 (1). 73-103.
    Google ScholarLocate open access versionFindings
  • 21. McFarlane, D.C. (2002) Comparison of Four Primary Methods for Coordinating the Interruption of People in Human-Computer Interaction. Human-Computer Interaction, 17 (1). 63-139.
    Google ScholarLocate open access versionFindings
  • 22. Milewski, A.E. and Smith, T.M. (2000) Providing Presence Cues to Telephone Users. Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2000), 89-96.
    Google ScholarLocate open access versionFindings
  • 23. Perlow, L.A. (1999) The Time Famine: Toward a Sociology of Work Time. Administrative Science Quarterly, 44 (1). 57-81.
    Google ScholarLocate open access versionFindings
  • 24. Robertson, T.J., Prabhakararao, S., Burnett, M., Cook, C., Ruthruff, J.R., Beckwith, L. and Phalgune, A. Impact of Interruption Style on End-User Debugging. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2004), ACM Press, 2004, 287-294.
    Google ScholarLocate open access versionFindings
  • 25. Seshadri, S. and Shapira, Z. (2001) Managerial Allocation of Time and Effort: The Effects of Interruptions. Management Science, 47 (5). 647-662.
    Google ScholarLocate open access versionFindings
  • 26. Witten, I.H. and Frank, E. (1999) Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann.
    Google ScholarFindings
  • 27. Yu, L. and Liu, H. (2003) Feature Selection for HighDimensional Data: A Fast Correlation-Based Filter Solution. The International Conference on Machine Learning (ICML 2003), 856-863.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Best Paper
Best Paper of CHI, 2005
Tags
Comments