Foraging Among an Overabundance of Similar Variants

Sruti Srinivasa Ragavan
Sruti Srinivasa Ragavan
Charles Hill
Charles Hill

CHI, pp. 3509-3521, 2016.

Cited by: 29|Bibtex|Views45|Links
EI
Keywords:
variantsinformation foraging theory perspectivesimilar variantexploratory programmingcreative taskMore(5+)
Weibo:
Rosson & Carroll studied how Smalltalk programmers find and apply code from one “usage context” to accomplish a task in their “current context”. They define three stages for reuse tasks: 1) Finding a usage context, 2) Evaluating a usage context, 3) Debugging a usage context. This...

Abstract:

Foraging among too many variants of the same artifact can be problematic when many of these variants are similar. This situation, which is largely overlooked in the literature, is commonplace in several types of creative tasks, one of which is exploratory programming. In this paper, we investigate how novice programmers forage through sim...More

Code:

Data:

0
Introduction
  • Computer-supported creative tasks—such as writing, graphic design, creating presentations, and some forms of programming—are often exploratory in nature.
  • This especially occurs when things go wrong, they reach a dead-end, or when they want to use features from other variants [2, 12, 17]
  • One example of such creative tasks is exploratory programming, in which programmers experimentally blend creating new code with reusing bits and pieces of code from various sources, some of which may be earlier variants.
  • If a user edits an application to use a menu instead of buttons, this results in a new variant of the application
Highlights
  • Computer-supported creative tasks—such as writing, graphic design, creating presentations, and some forms of programming—are often exploratory in nature
  • Rosson & Carroll [40] studied how Smalltalk programmers find and apply code from one “usage context” to accomplish a task in their “current context”. They define three stages for reuse tasks: 1) Finding a usage context, 2) Evaluating a usage context, 3) Debugging a usage context. This model was for reuse for a single variant of source code, so we extended the model in two ways to accommodate multiple variants
  • This study is the first to investigate how novice programmers forage through past and present variants. Analyzing their foraging behavior through an information foraging theory lens produced the following insights: x RQ1: Our results revealed new cue types signaling differences and similarities among variants; these cue types were extensively used by participants
  • We found that participants reused cues to reduce the cost of searching for cues in the environment—a new finding that can inform tool development
  • This paper presents the first empirical evidence that Information Foraging Theory has gaps when applied as a theory of variation foraging
  • Information Foraging Theory does not account for the temporal aspect of how participants developed stories, or how participants exploited structural similarities between different variants to locate patches of interest, or the significance to the participants of finding differences among similar variants
Methods
  • The authors conducted a think-aloud study to investigate how the target population would forage in an information space containing a large number of program variants.

    The authors used Hextris [8], a web-based puzzle game inspired by Tetris, as the test environment.
  • The authors conducted a think-aloud study to investigate how the target population would forage in an information space containing a large number of program variants.
  • The authors used Hextris [8], a web-based puzzle game inspired by Tetris, as the test environment.
  • The authors obtained the game’s source from a public GitHub repository [9].
  • Hextris fulfilled the criterion of containing a large number of variants that the participants could potentially use in order to complete their tasks.
  • The authors observed participants while they performed tasks.
  • The authors used screen-capture software to capture and record their actions on screen, and a webcam to capture video and audio of the participants directly
Results
  • Rosson & Carroll [40] studied how Smalltalk programmers find and apply code from one “usage context” to accomplish a task in their “current context”.
  • They define three stages for reuse tasks: 1) Finding a usage context, 2) Evaluating a usage context, 3) Debugging a usage context.
  • In order to find a context, one has to forage the right vari-
Conclusion
  • Implications for Theory and its Models

    Traditionally, IFT models work with collections of dissimilar but connected patches.
  • The authors' results indicated that users navigated across these disconnected collections using two main strategies
  • They focused on the differences between patches across similar variants (6 out of 8 participants).
  • These participants generated temporal stories about how the program evolved over time
  • These stories guided their navigations and were fundamental to some participants’ foraging.This study is the first to investigate how novice programmers forage through past and present variants.
  • The authors believe that extending IFT to incorporate variation-specific constructs will enable them to extend and benefit from a theoretical foundation of how people forage in the presence of variants
Summary
  • Introduction:

    Computer-supported creative tasks—such as writing, graphic design, creating presentations, and some forms of programming—are often exploratory in nature.
  • This especially occurs when things go wrong, they reach a dead-end, or when they want to use features from other variants [2, 12, 17]
  • One example of such creative tasks is exploratory programming, in which programmers experimentally blend creating new code with reusing bits and pieces of code from various sources, some of which may be earlier variants.
  • If a user edits an application to use a menu instead of buttons, this results in a new variant of the application
  • Methods:

    The authors conducted a think-aloud study to investigate how the target population would forage in an information space containing a large number of program variants.

    The authors used Hextris [8], a web-based puzzle game inspired by Tetris, as the test environment.
  • The authors conducted a think-aloud study to investigate how the target population would forage in an information space containing a large number of program variants.
  • The authors used Hextris [8], a web-based puzzle game inspired by Tetris, as the test environment.
  • The authors obtained the game’s source from a public GitHub repository [9].
  • Hextris fulfilled the criterion of containing a large number of variants that the participants could potentially use in order to complete their tasks.
  • The authors observed participants while they performed tasks.
  • The authors used screen-capture software to capture and record their actions on screen, and a webcam to capture video and audio of the participants directly
  • Results:

    Rosson & Carroll [40] studied how Smalltalk programmers find and apply code from one “usage context” to accomplish a task in their “current context”.
  • They define three stages for reuse tasks: 1) Finding a usage context, 2) Evaluating a usage context, 3) Debugging a usage context.
  • In order to find a context, one has to forage the right vari-
  • Conclusion:

    Implications for Theory and its Models

    Traditionally, IFT models work with collections of dissimilar but connected patches.
  • The authors' results indicated that users navigated across these disconnected collections using two main strategies
  • They focused on the differences between patches across similar variants (6 out of 8 participants).
  • These participants generated temporal stories about how the program evolved over time
  • These stories guided their navigations and were fundamental to some participants’ foraging.This study is the first to investigate how novice programmers forage through past and present variants.
  • The authors believe that extending IFT to incorporate variation-specific constructs will enable them to extend and benefit from a theoretical foundation of how people forage in the presence of variants
Tables
  • Table1: General demographics. (*Participant P02 occasionally programmed in JavaScript)
  • Table2: We coded participants’ operations, navigations and the cue-types they attended to. The highlighted items are the new codes we added pertinent to variations foraging that have not been reported in the IFT literature
  • Table3: Participants used some cue types more frequently than others to evaluate variants
Download tables as Excel
Related work
  • Research has explored providing variation support in nonprogramming domains, such as graphic design, documents and personal information management. Several tools support variations in graphics and interface design by allowing users to create multiple options, and providing ways to manipulate and compare them [12, 13, 43, 44]. For example, Kumar et al [18] present Bricolage, which helps web interface designers transfer the style and layout of one web page to another. Mechanisms to automatically support variations have also been proposed for personal information management. Karlson et al [16] introduce the concept of copy aware computing ecosystems that track user edits in a computing environment. They show that such personal information management systems can help users keep track of changes and semantics behind their copy operations.
Funding
  • This work was supported in part by NSF 1302113, 1314384, 1253786, 1314365, and 1439957 and David Piorkowski’s IBM PhD fellowship
Reference
  • Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 15891598. http://doi.acm.org/10.1145/1518701.1518944
    Locate open access versionFindings
  • Margaret Burnett and Brad Myers. Future of End-User Software Engineering: Beyond the Silos. 2014. In International Conference on Software Engineering: Future of Software Engineering track (ICSE Companion Proceedings '14), 201211. http://dx.doi.org/10.1145/2593882.2593896
    Locate open access versionFindings
  • Ed H. Chi, Peter Pirolli, and James Pitkow. The scent of a site: A system for analyzing and predicting information scent, usage, and usability of a web site. In proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 2000. http://dx.doi.org/10.1145/332040.332423
    Locate open access versionFindings
  • Ed H. Chi, Peter Pirolli, Kim Chen, and James Pitkow. Using information scent to model user information needs and actions and the Web. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 490-497. ACM, 2001. http://dx.doi.org/10.1145/365024.365325
    Locate open access versionFindings
  • Ed H. Chi, Adam Rosien, Gesara Supattanasiri, Amanda Williams, Christiaan Royer, Celia Chow, Erica Robles, Brinda Dalal, Julie Chen, and Steve Cousins. The Bloodhound project: Automating discovery of web usability issues using the InfoScent simulator. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 505512. http://dx.doi.org/10.1145/642611.642699
    Locate open access versionFindings
  • Paul Clements and Linda Northrop. 2001. Software product lines: Patterns and practice. Addison-Wesley Professional.
    Google ScholarFindings
  • Cloud9. 2015. Cloud9 – your development environment, in the cloud. Retrieved September 23, 2015 from https://c9.io/
    Findings
  • Logan Engstrom, Garrett Finucane. 2015. Hextris. Retrieved September 23, 2015 from https://hextris.github.io/hextris/
    Findings
  • Logan Engstrom, Garrett Finucane, Noah Moroze, Michael Yang. 2015. Hextris. Retrieved September 23, 2015 from https://github.com/Hextris/hextris
    Findings
  • Scott D. Fleming, Chris Scaffidi, David Piorkowski, Margaret Burnett, Rachel Bellamy, Joseph Lawrance, and Irwin Kwan. 2013. An information foraging theory perspective on tools for debugging, refactoring, and reuse tasks. ACM Transactions on Software Engineering and Methodology 22, 2: 14 http://doi.acm.org/10.1145/2430545.2430551
    Locate open access versionFindings
  • Wai-Tat Fu, and Peter Pirolli. SNIF-ACT: A cognitive model of user navigation on the World Wide Web. Human–Computer Interaction 22.4 (2007): 355412.
    Google ScholarFindings
  • Björn Hartmann, Sean Follmer, Antonio Ricciardi, Timothy Cardenas, and Scott R. Klemmer. 2010. D.note: revising user interfaces through change tracking, annotations, and alternatives. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 493502. http://doi.acm.org/10.1145/1753326.1753400
    Locate open access versionFindings
  • Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R. Klemmer. 2008. Design as exploration: creating interface alternatives through parallel authoring and runtime tuning. In Proceedings of the 21st annual ACM symposium on User interface software and technology (UIST '08), 91100. http://doi.acm.org/10.1145/1449715.1449732
    Locate open access versionFindings
  • Reid Holmes and Robert J. Walker. 2013. Systematizing pragmatic software reuse. ACM Transactions on Software Engineering and Methodology 10, 2: 44. http://dx.doi.org/10.1145/2377656.2377657
    Locate open access versionFindings
  • Paul Jaccard. 1901. Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Société Vaudoise des Sciences Naturelles. 37: 547–579.
    Google ScholarLocate open access versionFindings
  • Amy K. Karlson, Greg Smith, and Bongshin Lee. 2011. Which version is this?: improving the desktop experience within a copy-aware computing ecosystem. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 26692678. http://doi.acm.org/10.1145/1978942.1979334
    Locate open access versionFindings
  • Scott R. Klemmer, Michael Thomsen, Ethan PhelpsGoodman, Robert Lee, and James A. Landay. 2002. Where do web sites come from?: capturing and interacting with design history. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02), 18. http://doi.acm.org/10.1145/503376.503378
    Locate open access versionFindings
  • Ranjitha Kumar, Jerry O. Talton, Salman Ahmad, and Scott R. Klemmer. 2011. Bricolage: example-based retargeting for web design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 21972206. http://doi.acm.org/10.1145/1978942.1979262
    Locate open access versionFindings
  • Sandeep K. Kuttal. 2014a. Leveraging variation management to enhance end users' programming experience. ETD collection for University of Nebraska - Lincoln.
    Google ScholarFindings
  • Sandeep K Kuttal, A. Sarma, and Gregg Rothermel. 2013. Predator behavior in the wild web world of bugs: an information foraging theory perspective. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC '13), 5966. http://dx.doi.org/10.1109/VLHCC.2013.6645244
    Locate open access versionFindings
  • Sandeep K. Kuttal, Anita Sarma, and Gregg Rothermel. 2014. On the benefits of providing versioning support for end users: an empirical study. ACM Transactions on Software Engineering and Methodology 21, 2: 9. http://doi.acm.org/10.1145/2560016
    Locate open access versionFindings
  • Joseph Lawrance, Christopher Bogart, Margaret Burnett, Rachel Bellamy, and Kyle Rector. 2009. How people debug, revisited: an information foraging theory perspective. In Visual Languages and Human-Centric Computing (VL/HCC '11), 117-124.
    Google ScholarLocate open access versionFindings
  • Joseph Lawrance, Rachel Bellamy, and Margaret Burnett. 2007. Scents in programs: does information foraging theory apply to program maintenance?.In Visual Languages and Human-Centric Computing (VL/HCC '07), 15-22. http://dx.doi.org/10.1109/VLHCC.2007.25
    Locate open access versionFindings
  • Joseph Lawrance, Rachel Bellamy, Margaret Burnett, and Kyle Rector. 2008. Using information scent to model the dynamic foraging behavior of programmers in maintenance tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08), 13231332. http://doi.acm.org/10.1145/1357054.1357261
    Locate open access versionFindings
  • Joseph Lawrance, Rachel Bellamy, Margaret Burnett, Kyle Rector. 2008. Can information foraging pick the fix? a field study. In Visual Languages and HumanCentric Computing (VL/HCC '08), 15-19 http://dx.doi.org/10.1109/VLHCC.2008.4639059
    Locate open access versionFindings
  • Joseph Lawrance, Margaret Burnett, Rachel Bellamy, Christopher Bogart, and Calvin Swart. 2010. Reactive information foraging for evolving goals. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 2534. http://doi.acm.org/10.1145/1753326.1753332
    Locate open access versionFindings
  • Brad A. Myers, YoungSeok Yoon, Joel Brandt. 2013. Creativity Support in Authoring and Back-tracking. In Proc. Workshop on Evaluation Methods for Creativity Support Environments at CHI (CHI '13), 40-43.
    Google ScholarFindings
  • Nan Niu, Anas Mahmoud, and Gary Bradshaw. 2011. Information foraging as a foundation for code navigation (NIER track). In Proceedings of the 33rd International Conference on Software Engineering (ICSE '11), 816-819.
    Google ScholarLocate open access versionFindings
  • Seymour Papert and Cynthia Solomon. 1989. Twenty things to do with a computer. In Soloway & Spohrer: Studying the Novice Programmer (SS '89), 3-27.
    Google ScholarFindings
  • David J. Piorkowski, Scott D. Fleming, Irwin Kwan, Margaret M. Burnett, Christopher Scaffidi, Rachel K.E. Bellamy, and Joshua Jordahl. 2013. The whats and hows of programmers' foraging diets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 30633072. http://doi.acm.org/10.1145/2470654.2466418
    Locate open access versionFindings
  • David Piorkowski, Scott Fleming, Christopher Scaffidi, Christopher Bogart, Margaret Burnett, Bonnie John, Rachel Bellamy, and Calvin Swart. 2012. Reactive information foraging: an empirical investigation of theory-based recommender systems for programmers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 14711480. http://doi.acm.org/10.1145/2207676.2208608
    Locate open access versionFindings
  • David Piorkowski, Scott D. Fleming, Christopher Scaffidi, Liza John, Christopher Bogart, Bonnie E. John, Margaret Burnett, Rachel Bellamy. 2011. Modeling programmer navigation: A head-to-head empirical evaluation of predictive models. In Visual Languages and Human-Centric Computing (VL/HCC '11), 18-22. http://dx.doi.org/10.1109/VLHCC.2011.6070387
    Locate open access versionFindings
  • David Piorkowski, Scott D. Fleming, Christopher Scaffidi, Margaret Burnett, Irwin Kwan, Austin Z. Henley, Jamie Macbeth, Charles Hill, Amber Horvath. To Fix or to Learn? How Production Bias Affects Developers' Information Foraging during Debugging. In IEEE International Conference on Software Maintenance and Evolution (ICSME '15). http://dx.doi.org/10.1109/ICSM.2015.7332447
    Locate open access versionFindings
  • Peter Pirolli, Stuart Card. Information foraging in information access environments. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (CHI’95), http://dx.doi.org/10.1145/223904.223911
    Locate open access versionFindings
  • Peter Pirolli, Wai-Tat Fu. SNIF-ACT: A model of information foraging on the World Wide Web. User modeling 2003. Springer Berlin Heidelberg, 2003. 4554. http://dx.doi.org/10.1007/3-540-44963-9_8
    Locate open access versionFindings
  • Peter Pirolli. Rational analyses of information foraging on the web. Cognitive science 29.3 (2005): 343-373.
    Google ScholarFindings
  • Peter Pirolli, Wai-tat Fu, Ed Chi, and Ayman Farahat. Information scent and web navigation: Theory, models and automated usability evaluation. In Proceedings of HCI International. 2005.
    Google ScholarLocate open access versionFindings
  • Peter Pirolli. 1997. Computational models of information scent-following in a very large browsable text collection. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (CHI '97), 3-10. http://doi.acm.org/10.1145/258549.258558
    Locate open access versionFindings
  • Peter Pirolli. 2007. Information Foraging Theory: Adaptive Interaction with Information. Oxford University Press.
    Google ScholarFindings
  • Mary Beth Rosson and John M. Carroll. 1996. The reuse of uses in smalltalk programming. ACM Transactions on Software Engineering and Methodology 3, 3: 219-253. http://doi.acm.org/10.1145/234526.234530
    Locate open access versionFindings
  • Chris Scaffidi, Chris Bogart, Margaret Burnett, Allen Cypher, Brad Myers, and Mary Shaw. 2009. Predicting reuse of end-user web macro scripts. In Visual Languages and Human-Centric Computing (VL/HCC '09), 93100. http://dx.doi.org/10.1109/VLHCC.2009.5295290
    Locate open access versionFindings
  • Sruti Srinivasa Ragavan. Variations Foraging – Tasks. 2015. Retrieved January 08, 2016 from web.engr.oregonstate.edu/~srinivas/variationsforaging-tasks.html
    Google ScholarFindings
  • Michael Terry and Elizabeth D. Mynatt. 2002. Side views: persistent, on-demand previews for open-ended tasks. In Proceedings of the 15th annual ACM symposium on User interface software and technology (UIST '02), 71-80. http://doi.acm.org/10.1145/571985.571996
    Locate open access versionFindings
  • Michael Terry, Elizabeth D. Mynatt, Kumiyo Nakakoji, and Yasuhiro Yamamoto. 2004. Variation in element and action: supporting simultaneous development of alternative solutions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04), 711718. http://doi.acm.org/10.1145/985692.985782
    Locate open access versionFindings
  • Young Seok Yoon and Brad A. Myers. 2014. A longitudinal study of programmers' backtracking. In Visual Languages and Human-Centric Computing (VL/HCC '14), 101108. http://dx.doi.org/10.1109/VLHCC.2014.6883030
    Locate open access versionFindings
Your rating :
0

 

Best Paper
Best Paper of CHI, 2016
Tags
Comments