Diaries as a Methodological Innovation for Studying Grand Challenges

Abstract We illustrate the potential of diaries for advancing scholarship on organization studies and grand challenges. Writing personal diaries is a time-honored and culturally sanctioned way of animating innermost thoughts and feelings, and embodying experiences through self-talk with famous examples, such as the diaries written by Anne Frank, Andy Warhol, or Thomas Mann. However, the use of diaries has long been neglected in organization studies, despite their historical and societal importance. We illustrate how different forms of analyzing diaries enable a “deep analysis of individuals’ internal processes and practices” (Radcliffe, 2018) which cannot be gleaned from other sources of data such as interviews and observations. Diaries exist in different forms, such as “unsolicited diaries” and “solicited diaries” and have different purposes. We evaluate how analyzing diaries can be a valuable source to illuminate the innermost thoughts and feelings of people at the forefront of grand challenges. To exemplify our arguments, we draw on diaries written by medical professionals working for Doctors Without Borders as part of our empirical research project conducted in extreme contexts. We show the value of unsolicited diaries in revealing people’s thought world that is not apprehensible from other modes of communication, and offer a set of practical guidelines on working with data from diaries. Diaries serve to enrich our methodological toolkit by capturing what people think and feel behind the scenes but may not express nor display in public. Keywords Diaries Grand challenges Extreme context Silence Emotions Methodology Citation Rauch, M. and Ansari, S.(. (2022), "Diaries as a Methodological Innovation for Studying Grand Challenges", Gümüsay, A.A., Marti, E., Trittin-Ulbrich, H. and Wickert, C. (Ed.) Organizing for Societal Grand Challenges (Research in the Sociology of Organizations, Vol. 79), Emerald Publishing Limited, Bingley, pp. 205-220. https://doi.org/10.1108/S0733-558X20220000079019 Publisher: Emerald Publishing Limited Copyright © 2022 Madeleine Rauch and Shahzad (Shaz) Ansari License This chapter is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode Miss Prism: Cecily. I really don’t see why you should keep a diary at all Cecily: I keep a diary in order to enter the wonderful secrets of my life. If I didn’t write them down, I should probably forget all about them (Oscar Wilde: Bunbury: 1908: 210) Introduction The burgeoning literature on grand societal challenges spans a variety of spheres (e.g., Ferraro, Etzion, & Gehman, 2015; Gatzweiler, Frey-Heger & Ronzani, 2022), addressing, among others, poverty alleviation (Pradilla, Bento da Silva, & Reinecke, 2022) to affordable health care (Rauch & Ansari, 2021). However, there is still much to learn about how individuals at the frontline of these challenging contexts, experience and cope with extreme situations involving human suffering. While extreme contexts and grand challenges are two distinct strands of literature, many grand challenges play out in extreme contexts (see Hällgen, Rouleau, & de Rond, 2017, for an overview). Relatively little research has attended to the emotional trauma of individuals working to alleviate human suffering in the midst of extreme contexts, such as warzones (De Rond & Lok, 2016; Fraher, Branicki, & Grint, 2017) and refugee camps (De La Chaux, Haugh, & Greenwood, 2018). Social science has documented numerous sources of data to study grand challenges such as surveys, case studies, interviews, ethnographies, and natural experiments, to capture the complex and manifold dimensions of grand challenges. Such data, however, may not be able to apprehend the real-time feelings, thoughts, and experiences of affected individuals confronting extreme situations. For example, in their study of the social stability in a Kenyan refugee camp, De La Chaux et al. (2018, p. 160) acknowledge the difficulty of studying grand challenges: […] sensitive topics such as incidents of rape and the role of mafia gangs remained difficult to discuss as we sensed a clear reluctance to delve into these topics. […] In the follow-up phone interviews, silences, pauses, and hesitation took on a similarly central role and again occurred primarily when we inquired about incidents of gender-based violence and gang structures […]. In the past decades, several methodological innovations have emerged in management scholarship to address various ontological and epistemological considerations. This includes video data, team ethnographies, and multimodality – addressing material, sensory and visual aspects of data (Jancsary, Höllerer, & Meyer, 2016; Kress, 2010) in order to strengthen the existing methodological arsenal, such as field observations and interviews. However, concepts such as emotions and cognitions are still difficult to decipher and access (Zietsma, Toubiana, Voronov, & Roberts, 2018). Oftentimes, “emotions are not worn on sleeves” but instead actors engage in emotional labor, emotional self-censorship, or a strategic display of their emotions (Jarvis, Goodrick, & Hudson, 2019). The well-known example of the air hostess smiling on the job (Hochschild, 1979) points to the emotional labor in the service industry, in which actors perform certain roles (e.g., being friendly and welcoming) that do not betray the feelings that they actually experience. We propose the use of personal diaries (Burgess, 1984) as a fruitful and rich source of data in the endeavor to study grand challenges by giving voice to participants in the daily praxis of their work. Diaries capture people’s innermost thoughts, reflections and memories that they document without the gaze of social judgment. Writing personal diaries described as a “technology of the self” (Foucault, 1982) is a time-honored and culturally sanctioned way of animating personal values and innermost feelings (Klein & Boals, 2001), and embodying experiences through self-talk. Indeed, diary writing is an established tradition and practice across different cultural backgrounds dating back to famous examples such as the so called “Pillow book” by the court lady Sei Shanoagon to Empress Consort Tieshi during the 990s and early 1000s in Japan. Other notable examples include James Cook’s board journals from the eighteenth century, the diaries kept by Samuels Pepys from London in the 1660s, and the diaries written by Anne Frank and Thomas Mann – all of which represent important cultural and historical artifacts of modern society. Drawing on our own experience of working with diary-based data in extreme contexts (e.g., see Rauch & Ansari, 2022 for a diary-based project studying military personnel), we illustrate their potential of diaries to study and advance scholarship on grand challenges and organization studies. While diary writing is known to bring psychological benefits to people coping with traumatic events (Amabile & Kramer, 2011), we showcase the value of diary writing for management studies (Balogun, Huff, & Johnson, 2003; Ohly, Sonnentag, Niessen, & Zapf, 2010). Diary studies seem particularly suitable for investigating personal accounts of people deeply affected by grand challenges, and capturing their innermost thoughts and feelings, as well as their coping strategies in the midst of extreme contexts (Farny, Kibler, & Down, 2019). We show the value of unsolicited diaries in revealing the inner world of people that cannot be captured by other modes of communication. We outline different ways of working with diaries and the advantages and disadvantages of conducting research with diaries, and provide a set of practical guiding principles by drawing on our own experiences of working with diaries-based data. Using Diaries for Studying Grand Challenges Despite its frequent use in private life such as in the famous examples of Anne Frank, Max Frisch, Andy Warhol, and a myriad of diary writers across the globe, the use of data from diaries in (social) science is rather rare. This is despite the few attempts to introduce diaries into social science (Alaszweski, 2006; Balogun & Johnson, 2004; Rauch & Ansari, 2022). In many disciplines, such as chemistry, pharma, and medicine, using and writing diaries are integral parts of research, as well as a means of transferring and communicating knowledge between individuals to keep track of developments (Bartlett & Milligan, 2015; Hislop, Arber, Meadows, & Venn, 2005). In comparison, social science research rarely relies on diaries as a key source of knowledge (see for exceptions Balogun & Johnson, 2004; Hannaway, 1989; Rauch, & Ansari, 2022). The use of self-reporting methods like diaries has been described as an “unconventional method” (Wickert & Schaefer, 2015, p. 124), and can serve “as a supplement to the active role of critical researchers” (Wickert & Schaefer, 2015, p. 124) in shedding new light on managerial work and the performative aspects of language. indeed, “self-reporting methods have been applied successfully [...] and appear to be a promising tool” (Wickert & Schaefer, 2015, p. 124). Diaries vary in terms of the degree of detail but also in the way they are written, as well as regarding the motivation of people to write diaries. Diarists adopt different styles in writing about different aspects of their lives. While some diarists engage in daily writing, for example, in the morning or the evening, others engage in less frequent journaling. The diversity of diaries is also illustrated by the different forms that diaries take; some write on Microsoft Word, others use weblogs or diary-apps, a classical blank page or a book with or without a lock. Here, we follow the definition of diaries by Alaszweski (2006, p. 2) “as a document created by an individual who has maintained a regular, personal and contemporaneous record.” In the extant literature, journaling, keeping a diary, and diarist, are treated as synonyms (Alaszweski, 2006). Diaries are emic reflections and not etic observations. Emic reflections refer to viewpoints obtained from within the social group from the subject’s insider perspective. Etic observations refer to the observer’s outsider perspective (Headland, Pike, & Harris, 1990). We provide granularity and specificities regarding how diary data can be used to study organizational phenomena and provide the distinctions between two forms of diaries, and their advantages and disadvantages (see Table 1); that is, (1) unsolicited diaries, which are diaries that are conducted voluntarily without the observation of others; or (2) solicited diaries, which are written for trials, research projects or to cover a particular event (Milligan & Bartlett, 2019). For example, Rauch and Ansari (2022) use unsolicited diaries of military personnel working in the U.S. Air Force to study their perception of drone technology and its influence on nature and morality of their work, and the subsequent consequences it had for them and their work. Balogun et al. (2003) solicited managers to write their reflections in a diary based on their responses to five questions to track the progress of change implementations in their organizations. Others, such as Buchanan and Boddy (1992) studied the change managers’ experience of managing change by drawing on audio diaries over a two-week period. Schilit (1987) drew on diaries by middle managers to record the frequency, nature, and outcome of interactions among supervisors and themselves, in relation to strategic decisions over two months. These examples highlight the importance of diaries as a tool for research. We zoom in on the use of unsolicited diaries, as they offer unique insights into the “thought world” of the diarist without being subject to social desirability and other biases that potentially affect solicited diaries. One type of such unsolicited diaries is also referred to as “natural diaries.” In other words, “the test is whether the interaction would have taken place in the form that it did had the researcher not been born or if the researcher had got run over on the way to the university that morning” (Potter, 1996, p. 135). Diaries allow for immediate reflections on events, experiences, and thoughts. In particular, diaries provide a more nuanced understanding of situations that evoke strong emotions. As such, unsolicited diaries can mitigate retrospective bias and socially desirable responses by providing a thick description of events in “real-time,” and how those events unfold over time (e.g., in change processes). Table 1. Overview Diaries. Unsolicited Diaries Solicited Diaries Definition Diaries are conducted voluntarily without observation of others Diaries are written for the purpose of trials, research, or to cover particular events Advantages Diarists are engaging voluntarily in journaling, often depicting a form of life-history data capturing the individual's reflections about life, experiences, and how the individual feels about particular events, and happenings in their life, that is, offering an “inside view, and helps to look on the “backstage” of the “backstage” (Goffman, 1959) Allows for the absence of the researcher in the field, especially suitable in extreme contexts (such as war) and covering a longer time period (months to years) Mitigate the risk of “retrospective bias” and “social desirability bias” which often associated with interview studies Diarists focus on certain aspects prompted by the initial guidance to record impression that are of interest to the research project, for example, focus on certain types of events, issues, and experiences (More) systematic technique to collect data Allows for the absence of the researcher in the field, especially suitable in extreme contexts (such as war) and covering a longer time period Helps to mitigate the risk of “retrospective bias” which often associated with interviews Points for consideration As there is no “one way” of keeping a personal diary, different diary technique and foci on detail among diarists Large amount of data which might be less relevant for the purpose of the research project Difficult to gain access and importance to adhere to anonymity and rules such as General Data Protection Regulation (GDPR), especially in data storage and handling Actors encouraged to keep to a diary for research purposes Actors are potentially less inclined to keep detailed diaries (e.g., managers, CEO) Diarists might feel “uncomfortable” and “observed” when journaling (e.g., on sensitive topics) Regular check in with diarists to potentially keep them motivated Importance to adhere to anonymity and rules such as GDPR, especially in data storage and handling Important examples Rauch and Ansari (2022) studying military operators Balogun and Johnson (2004) studying middle manager Studies on extreme contexts have illustrated how people “emotionally control” (de Rond & Lok, 2017; Fraher et al., 2017) their behavior in difficult situations, and are reluctant to speak about difficult experiences. One example is the study by De La Chaux et al. (2018) on maintaining social stability in Kenyan refugee camps. Fraher et al. (2017, p. 252) allude to the role of emotions in their study of U.S. Navy Seals working in Afghanistan and Iraq, arguing that the Seal candidate is forced to compartmentalize his emotions – and not fixate on them – to provide his best effort in the moment and to not obsess over the “what-ifs” […]. The social norms of working in extreme contexts such as in the military encourage a “culture of silence” (De Rond & Lok, 2016; Fraher et al., 2017), where participants are unable to not just express or display their emotional distress but also not share their experiences with colleagues. By observing or interviewing actors in such extreme contexts, one might conclude that actors are not emotionally affected by the experienced atrocities as they appear calm and controlled in getting on with their job, preferring not to display their emotions in public (Farny et al., 2019; Jarvis et al., 2019). As such, diaries can capture the experience of individuals in a way that is not possible by using traditional designs. They permit the examination of events and experiences in their natural and spontaneous context (Reis, Erber, & Gilmore, 1994), offsetting some of the problems associated with retrospective accounts (Bower, 1981). Empirical Examples from Diaries To show their methodological value, we provide illustrations of how diary studies can be used to enrich qualitative studies. Potentially, diaries can be used in the same manner as sole diary studies and as a support for quantitative studies (e.g., Amabile, Barsade, Mueller, & Staw, 2005; Schilit, 1987). However, we will focus on diaries as part of qualitative analysis. We draw on one example conducted for a larger project on “organizing in extreme contexts” in which we studied how actors cope with brutal reality in such contexts. In doing so, we draw on examples from our research drawing from unsolicited diaries obtained from medical personnel working for Doctors Without Borders (MSF). In the case of MSF, we analyzed a set of over 70 diaries of medical personnel ranging from nurses, doctors, and surgeons from different parts of the world with diverse backgrounds, gender and sexual orientations. The organization did not prescribe its personnel to write diaries. Instead MSF personnel engaged voluntarily in journaling. We interviewed all diarists (besides one) after we read through the diaries several times. In addition, one author joined several missions to extreme contexts such as Afghanistan, South Sudan, and Yemen to observe work in extreme contexts and get a feel of the context. This was done in light of our research interest in the realm of grand challenges, in contexts that scholars have described as people being “silent” or numb when it comes to the expression of their emotional experiences. In these situations, actors have deployed coping strategies when they are unable to openly voice, express, and display their emotional distress. Our observations regarding emotional experiences from field trips align with the findings of studies of contexts where actors prefer to remain silent rather than display their emotions and articulate their experience. Actors cope in different ways when faced with trauma, such as black humor, cynicism, planting vegetables, or engaging in substance abuse. By analyzing the diaries, we were able to observe that actors are actually not emotionally numb or in control of their emotions as it may appear from observing their behaviors in the field. Instead, they are highly affected emotionally by the experience on the ground but simply don’t wear them on their sleeve. As such, a surgeon illustrates in his diary his experience working in an MSF clinic in Afghanistan: I started two days ago and I remember he [Youssuf] was one of my first ones. I can’t describe what I have done. I know the medical procedure but in reality, what I did. I cannot. I should be able to but I cannot. I had no other option. Had the inner urge to puke. […] Youssuf was one of my longest patients. He had a severe infection. […] No surprise he had a severe infection. Every double here without infection is a medical miracle. Prosthetics in plastic bags. No wheelchair. No doubt no wheelchair. What do I even say? I am happy if we have enough pain pills to hand them […]. We have set him up with new prosthetics today. It was Christmas and birthday all in one. It is the best of all feelings here. It is the only feeling worth describing here. (Diary 56) In a follow-up interview with this surgeon, we probed further into this diary entry in relation to the patient which he called Youssuf: Of course I have never talked to anybody in the field about my sentiments towards Youssuf. Talking isn’t a way of solving anybody's problem in the field. We all do the work and we all need to find ways to come to terms with it. But to your question. No, I have never articulated or share this with anybody. We all performed procedures together but we have never talked about it from a human empathic point of view. We kept it always to the medical point of view. For me writing a diary which I still do to his very point was a way to find a space for myself but also a space to share my thoughts, both negative and positive ones. And I can tell you in the field there are more negative ones. (Interview 56) Looking at diaries in relationship to the interviews allowed us to access the inner thoughts, reflections, and feelings of the actors in the field. By looking at these diaries, we reveal how medical professionals are highly affected emotionally by what is happening in the field in contrast to what one may conclude from observing their behaviors in the field where they may appear as “cold rational agents.” We could also observe how actors described the need to change their way of “seeing” and “perceiving reality.” They described such a change as necessary “to survive” and cope with the traumatic reality at hand. A doctor in a field trip to Yemen reflected on the situation of civilians and the lack of access to basic food and medical supplies amidst the ongoing war: Once starvation was on the table and starvation was used as a weapon and tool of war, things radically changed for me. One needs to change to stay sane. No way to rationalize and find a justification why a human being would use and try to eradicate other humans in such a manner and cruel way […]. At this point I learned the hard way to look the other way. A different way! Which way this is I am still figuring out. For now, I just know it can't be the old way. (Diary 70) Despite the need to cope with the difficult situations on the ground, and “look the other way,” we identified a “culture of silence” (De Rond & Lok, 2016), where people do not share nor openly talk about their difficult experiences and emotional distress. Rather, they express themselves through writing personal diaries that reveal the internal conflicts they experience and the mechanisms they use to cope with the conflict. They engage in this form of “self-censorship” (Edmondson, 1999), not out of fear that speaking up will lead to being penalized by their superiors or concerns about personal consequences (e.g., Morrison, 2011). Rather they do so, as a coping mechanism to deal with the emotional distress they experience in extreme situations and for being able to “get the job done.” Speaking about emotional distress and sharing feelings even with colleagues only made matters worse, as in many cases, it led to emotional breakdowns. Overall, looking at the diaries while not being a researcher in situ, allows us to gain an insider’s understanding of their daily experiences and a peek into their innermost thoughts and feelings unobservable elsewhere. Methodological Guidance In the excerpts we showed, we have used the example of emotions illustrating the power of diaries. The use of diaries is not limited to the aforementioned concepts and ideas. Studying diaries can be also used for concepts such as cognition, sensemaking, and framing but also for studying identity, and interactions with technology. We see strong potential in using diaries revealing other aspects of organizational life and changes in the nature of work. For example, in following organizational norms, procedures, and rules, there may be a stark difference between the emotions experienced and the behavior displayed in which people may not express their innermost doubts and cynicism. We provide practical guidance informed by our own experience, and some important questions to be raised when engaging with diaries (see Table 2). We have characterized them alongside steps in empirical research which may vary depending on the scope of projects. Table 2. Questions to Consider When Working with Diaries. Data Collection Data Analysis Ethical Considerations How to get access to diaries? What is the purpose of data collection? Will diaries be the “main” data source? Why do people share diaries? What is the advantage of collecting interviews alongside diaries? Will you keep their employer in the loop? (How) to transcribe hand-written diaries? Which analytical approach will you follow? Can you use text analysis software? How do you analyze diaries? Will you use a sensitizing concept? Will you work with other data sources? How important are diaries vis-á-vis other data sources? Ethical approval by internal review board? How do you adhere to anonymity? Received “informed consent” by the diarists? Where and how you are going to store the diaries (long term)? How are you storing the physical diaries? What happens if one diarist would like the diary back/dies? How do you ensure research ethics with such sensitive data? How do you ensure GDPR in the project? Data Collection One reason why diary-based studies are less common in organization studies is partly due to the difficulty of access to diaries. Not everyone writes diaries nor may be vocal about their journaling, and even if they write diaries, they may be less willing to share their diaries with a stranger (researcher). This observation is in particular true for unsolicited diaries as these are not written as part of a research project, but instead considered to be a “sacred” space for the diarist’s private or even intimate reflections and inner thoughts. It might be easier to obtain in pre-arranged solicited diaries (e.g., see Ohly et al., 2010). One example is the study of procedures by Balogun et al. (2003) in which the authors instructed the diarists to focus on five particular questions in their journaling activities. Also, in a solicited diary study, Amabile and Kramer (2011) probed the everyday work experiences of professionals working of 238 professionals in 26 project teams. The study was based on a daily questionnaire in the form of a diary that was emailed from Monday through Friday. For gaining access to diaries, consider public sources such as the German diary archive which houses over 20,000 diaries by 4,000 diarists (Deutsches Tagebucharchiv, 2019), or the solicited diary database by Theresa Amabile based on her past work. From interactions with vocal actors at MSF, we had prior knowledge that a group of actors engages in journaling in order to cope with their emotional distress. Through support by key figures (including award-winning medics highly esteemed within the community) and personal relationships, we were able to build trust with an initial set of volunteers who became willing to share their diaries with us. It is important to emphasize that central figures do not necessarily need to be high ranked officials, or those with administrative influence such as human resources, that most likely does not keep track of who engages in private diary writing. On the contrary, it was important for actors in our setting that their private diaries will not “land on the desk of HR” (Interview 10). It is important to gradually build trust in order for people to become amenable to sharing data that might contain intimate aspects of their lives. We used a snowballing approach (Biernacki & Waldorf, 1981), that is, “a study sample through referrals made among people who share or know of others who possess some characteristics that are of research interest” (p. 141). We did not directly ask diarists to participate given the sensitive nature of data, but we relied on the unsolicited referrals of individuals encouraging other colleagues to participate by sharing their diaries. In subsequent projects, community support was of pivotal importance to gain access. Our experience of doing research in extreme settings, including time spent in Afghanistan visiting military establishments and semi-governmental organizations, gave us legitimacy in being able to relate and “speak the same language.” Our independent role as researchers also helped in gaining access, as we do not hold any institutional ties to the organizations, nor are involved in any consulting, education or any other activity. Participants saw us as neutral outsiders with no affiliation with the organization (including HR). As some of them acknowledged in private conversations, they were more willing to share their diaries with us than anyone from within their organization or the media. In our project, diary data have been the primary data source, which we have complemented with observations to sensitize us with the context, and interviews to follow-up on emerging topics and need for clarification. However, we made two key observations during the follow-up interviews: First, actors reacted differently to us knowing “everything” from their diaries including very personal sentiments pertaining to aspects outside of our research interest (e.g., family life, dreams, and desires) from treating us very welcomingly to being rather reserved. Some felt uncomfortable at first: “you know me through and through but I hardly know you.” As a common reaction at the beginning of each interview, most diarists asked the questions “Did you read all of it [Diary]”? often concluding that “you know more than my wife does” and “Don’t know what I can add to this anymore.” Second, many felt uneasy about revisiting particularly (harsh) experiences when prompted, as they found them “difficult to speak about.” This is echoed by De La Chaux et al. (2018) that people find it difficult to talk about difficult experiences despite journaling frequently about