Ethnography in industry: Methods overview (part one)

Editor’s note: PARC contributed a version of this post to UX Magazine. You can see that article here.

It’s really hard for companies to understand ethnography — even after they understand what objectives it can be used for.

In this second post in our series on ethnography [you can read the first one here], I thought it might be useful to provide an overview of data COLLECTION methods (and methodologies) that ethnographers use to understand a particular population or situation of interest; while specific needs vary, for our clients the general goal is to help them address a murky problem or innovate differentiated products.

Note the emphasis on data “collection” as opposed to data ANALYSIS. The latter is an important — though far less visible — other half of the equation, and is often where an ethnographer’s true expertise shines… But this is a different topic, so we’ll share more in an upcoming post.

Observation, observation, observation

Before joining PARC, I used popular observation methods to understand public health interventions. Since joining PARC, I have greatly expanded my definition, scope, and toolkit for “observation” because I have been faced with conducting ethnographic studies to satisfy the diverse goals of varied clients.

My colleagues and I still do what PARC has been credited with pioneering in the technology space, and that is human-centered observation: particularly through the use of video ethnography. We observe, record, and, when feasible, actually participate in the activity at hand. Video doesn’t lie — it adds a great deal of power to human behavior analysis — and we have an entire lab space devoted to video data sessions where our ethnographers join together to analyze human behavior recorded during ethnographic observation. Again, this observation methodology is an area of expertise that deserves more detail in a future post.

But the insight doesn’t stop with us. By showing client stakeholders examples of phenomena observed through ethnography – whether in the mobile domain, leisure settings, workplace, cityscapes, or elsewhere – we try to bring a “real world” view they might not otherwise see. In a recent workshop where we presented the end-user’s vantage point, our client commented, “Wow, we never really considered that user of our product — we have been so focused on our competitors’ product features that we didn’t consider this group of users.”

While the insights we show to clients are private to them, here’s a video of projector use, just one example of how we use video to show human behavior.

Asking, watching, and listening

Observation, though extremely powerful, is usually not enough. Ethnographers therefore can use any of the below methods depending on the situation or need to gain different slices of understanding a target group or situation of interest.

Semistructured or in-depth interviews. Ethnographers create a set of concepts or research questions on behalf of stakeholders. For us, the exploration plays out much like a conversation — answers tend to come out naturally, we may vary the order of the questions, and we can allow tangents if worthwhile. This flexibility (as opposed to sticking to a templated script makes more probable unexpected discoveries and really getting what people care about.

Show-and-tell. To avoid relying on imagination in unfamiliar situations, we often ask study informants to demonstrate the very things they are describing.

Think-aloud protocols. We ask study informants to fully explain what they are doing and thinking as they do it, so that we can better understand their objectives, thought processes, and decision-making processes.

In situ interviews. Often used in combination with more unobtrusive direct observation, we ask questions as people go about their usual activities so we can understand context that may not be obvious.  While we do this sparingly to avoid burdening study informants, we are always balancing the need to observe undisrupted behaviors with the need to understand what we are observing.

Shadowing. In some cases we are interested in the behavior of one individual over a length of time so we will shadow virtually every second of that person’s life for a set period of time (e.g., 1-8 hours a day).  We mike the subject and use wide-angle HD video cameras to capture as much as possible of his/her life within the given time period.

Focus groups. Used sparingly at PARC (at least in the way that market researchers use it!), this technique is more often a problem-solving vehicle or design endeavor. So we may use these in an ethnographic consultation where local informants are given the opportunity to participate in outcomes. Groups can range from the classic 8-12 participants from a particular demographic to subjects gathered at a study site for a particular set of questions or reactions.

Tracking

It may not be enough — or even feasible — to watch and ask. So we’ve adapted or developed many methods to assist in tracking and ultimately understanding human behavior, including:

  • surveys — one-off or repeated (e.g., daily)
  • diaries — paper or electronic
  • mobile- or desk-based experience sampling – which may be randomly timed or triggered by contextual data from logging, sensing, or instrumenting interactions in information, social, and virtual environments.

These methods can generate qualitative data (e.g., diaries) or quantitative data that might be analyzed using descriptive or inferential techniques and include machine-learning approaches.

Enter customization. We often have to build custom software to collect this data, which often requires advanced computational methods to analyze it. Since PARC has the advantage of social scientists working closely with other scientists, we regularly collaborate with our computer scientists to conduct “hybrid” studies that combine some of the above. And yes, we’ll have much more to say on this topic.

The science and art of ethnography is not in a preset formula for these individual methods. It’s in the selection, unique combination, customizations, and analysis — which together can yield the “deep” understanding that in turn inspires innovation, or fosters change.

There IS a method to the madness.

 

8 thoughts on “Ethnography in industry: Methods overview (part one)

  1. Pingback: Tweets that mention Ethnography in industry: Methods overview (part one) - PARC blog -- Topsy.com

  2. Todd Barnard

    [Editor's note -- Comment left after this PARC Forum talk in our "Ethnography in Industry" speaker series.]

    I assume that at some point in the future there will be an Ethnography vs. Ethnomethodology post in this series. I am of the opinion that using Ethnography for things like social search, social interaction design (SxD), and finding a social scientist to identify actionable insights is an error. What is called for is Ethnomethodology.

    From Nadine Schuurman’s 2008 paper Database Ethnographies Using Social Science Methodologies to Enhance Data Analysis and Interpretation:

    The goal of ethnography ‘is improvement of theory’ (Herbert 2000, 560), so it needs to generate conceptual tools that are useful to others. Herbert address the three chief criticisms of ethnography, namely: (i) ethnography relies on interpretation and it is not sufficiently scientific; (ii) an intense focus on a single or limited number of situations make it difficult to generalize; and (iii) representation is clouded by the failure of researchers to problematize their methods.

    Founded in 1960s by American sociologist Harold Garfinkel, ethnomethodology is the sociological perspective that analyzes the methods people use in order to understand or make sense of their social world…ethnomethodology purports that social order is illusive, it only exists as a mental construct and that social life, though often appearing orderly, may potentially be chaotic.

    Using the ‘documentary method’, people extract social facts from any given situation that appears to be representative of a pattern, and then utilize this pattern to interpret the meaning of these facts. Ethnomethodological studies have focused on the construction, organization and temporal order of social activities and interactions and, therefore, provided a greater understanding of the methods used to accomplish work. Ethnomethodological analysis of work and work settings has often been used as a framework for the design of computer-supported collaborative work systems, and throughout the 1990s these ideas were increasingly applied to human-computer interaction study as well.

    Although initially utilized in design critique, this approach has been progressively applied to design practice. It is now acknowledged that real-world failures of complex technology are often not attributable to technological shortcomings but to the failure to recognize the methods in which work and communication are actually organized.

    Ethnomethodology clarifies the gap between theory and practice by asking seemingly trivial questions that may inadvertently topple presumed internal logic. It attempts to avoid judgments of true or false or moral stances in favor of observation. For data drawn from different contexts to have value and applicability, they must be considered to be true – that is, their validity derives from their acceptances as ‘facts’. Actor-network theory (ANT) is a well-established means of understanding how certain practices and scientific ‘facts’ are established…

    Note the current Activity Stream specification has its roots in ANT.

    Also, I am surprised to not see any citations to former PARC researcher Lucy Suchman and her book Plans and Situated Actions which I believe borrows heavily from Garfinkel’s Ethnomethodology.

  3. James Glasnapp

    Thank you Todd for commenting, particularly on the importance of ethnomethodology and the points you make about “SxD” derived from Gentry’s very delightful talk, which can be viewed online here.

    We couldn’t agree more about the importance of ethnomethodology in order to understand interaction and “the how” people do things in various contexts and with various tools. We’re also focused on the practical applications of all ethnographic theories and methods, and that includes applying ethnomethodology to uncover a greater understanding.

    By the way, the paper you cited and linked to includes a number of references to Paul Dourish; he also delivered a talk in our series which can be viewed online here.

    Keep an eye out for an upcoming post we wrote about Suchman’s book and original “ethnography in industry” study at PARC!

  4. Pingback: Tweets that mention Ethnography in industry: Methods overview (part one) - PARC blog -- Topsy.com

  5. Gentry Underwood

    Point taken, but can we just say “ethnographic methods” and skip the five-dollar words? “Ethnography” is already esoteric enough to make most (non-academic) people go glossy-eyed. I fear “ethnomethodology” might mean losing them altogether.

  6. Todd Barnard

    @Gentry Underwood

    Their tongue-tying Greek etymology notwithstanding, there is a critical distinction between Ethnography and Ethnomethodology and the two should not be treated as interchangeable.

    … in the same sense that Rhinocerotoidea and Rhinoplasty are two distinct disciplines

  7. Pingback: Putting people first » Ethnography in industry

  8. Pingback: Ethnography in Industry: Methods for distributed & large data sets (part two) - PARC blog

Comments are closed.