UX Research Playbook — In-depth Interviews
In-depth interviews are ultimately based on 'interview techniques.'
Interviews are introduced as a technique for understanding the customer better. The reason is that they are conducted as a 1:1 conversation, not as a test where you read text and answer it. So even if something is unclear in the middle, it can easily be explained, and as the participant becomes immersed in answering, you can also obtain information neither party expected. As such, it is a very useful research methodology for understanding the customer. (User Interview Textbook)
And in user interviews, even after the interview is over, the presence of the interviewee remains in the researcher's memory, which naturally leads to inferences such as 'Ah! Wouldn't a user like this think this way?' — and through this process, building an 'cognitive empathy' with the user becomes smoother. (User Interview Textbook)
For reference, looking at the research matrix introduced earlier, you can see that interviews are positioned, as in the figure, between attitudinal and qualitative research.
And looking through the matrix proposed by Christian Rohrer in a bit more detail, you can see that an in-depth interview sits even further inside, between the qualitative and attitudinal areas.
This kind of research type allows you to expect a holistic approach — through close conversation with the participant — that reaches all the way to the user's background and surrounding environment.
For reference, just next to in-depth interviews lies the focus group interview. The words 'in-depth' and 'focus' seem similar in meaning, so I took this opportunity to compare which parts are alike and which are different.
As the words 'in-depth' and 'group' suggest, the techniques differ in participant sample size — '1:1' vs '1-to-many.' But this single difference can produce a much bigger gap than you'd think.
For example, in qualitative research, the most important factor — contact time — can differ by a factor of 3 to 4. That's the reason. A focus group interview is conducted with multiple people, led by a facilitator based on a few topics decided in advance. Because it's facilitator-led, the facilitator's bias can creep in unconsciously, which calls for caution. Also, in the nature of group meetings, only the people who like to talk keep talking, and on sensitive topics it can be hard to share honest opinions — so this needs to be considered up front, with appropriate encouragement and mediation by the moderator.
Earlier I said that one of the biggest differences between focus group interviews and in-depth interviews is *contact time. For reference, according to the content in 'A Guide to Qualitative Research,' 'how much time the researcher spends with the interview or observation subject is the most important precondition for good qualitative research.' That makes sense — when there are many opportunities for contact, 'rapport' forms with the participant, and once that happens, the 'out-group homogeneity bias' toward people or organizations not your own naturally decreases, and as a result the research can secure diversity and pluralism.
Ultimately, sufficient contact becomes the foundation of qualitative research, and through it you can expect the following five outcomes.
1 Cognitive empathy (the goal of much qualitative research)
Means how closely the researcher has come to understand the observed in the way they understand themselves — what people perceive — what it means to them — what motivates them.
2 Pluralism
Means how diversely the people or places described are represented. At the same time, this is also a signal that the researcher has achieved cognitive empathy.
3 Concreteness
Means how factually concrete (rather than abstract) the evidence presented is.
4 Tracking
Tracking means how much data the researcher collected to answer questions that newly emerged in the course of collecting data.
5 Self-awareness
Defined narrowly: how well the researcher themselves understands the impact of their own assumptions on the interview or observation subject. The core difficulty of interviews and observations as data collection methods is that data isn't simply collected — it's produced — and the data collector inevitably exists within the data itself.
So what is qualitative research? Personally, after gathering material from various sources and organizing it, I came to think it can be understood, as in the table, as a technique for grasping the meaning of 'human behavior' holistically through 'interviews' or 'observations,' and through that, naturally grasping the background and the context of the behavior.
Coming back,
Of course, that doesn't mean in-depth interviews come with only advantages. Because they are research conducted in conversational form, there can naturally be variation in outcome depending on the interviewer's skill. And because the result is drawn from a small number of participants, generalizing the research findings is not easy. As a result, it can take more time than expected, so continuous attention to areas requiring caution is needed... then again, what isn't like that...
Who, when
So who should use in-depth interviews, and when? As introduced earlier, they can be useful in the early strategic stages.
In particular, they can have high utility not only in organizations planning to launch a new product, but also in organizations that need to improve an existing product.
How
First, the in-depth interview process can be broadly divided into the stages of preparation — execution — analysis.
*TMI… For reference, if you abstract research up to the most basic types — purely as my personal view — you can broadly divide it into in-depth interviews (qualitative, exploratory research) and usability testing (quantitative, evaluative research). These basics, precisely because they are basics, can be both easy and hardest at the same time — like learning asreepa when studying drawing. Looking at the items above, there's a lot to do, but the consoling? part is that these forms (frames, ways of thinking) are repeatedly used in other research techniques as well, so if you build a solid foundation, it can be useful in many directions.
The first thing to do is research planning. Looking ahead at what's covered in chapter 17 of this book, it broadly consists of basic information, purpose and goals, and the operational plan.
For reference, the book 'User Interview Textbook' introduces a template for interview planning. In that book, the interview plan is divided into three categories — hypothesis verification, task analysis, and opportunity exploration. Since this post is about in-depth interviews, the slide above is composed using the hypothesis-verification template, which is the closest fit.
Next, in the participant recruitment and planning stage, you select the interview type and conduct method using the criteria introduced earlier, and define other items such as number of interviews, timeline, and incentives.
Next is writing the discussion guide. As the name implies, the discussion guide is just a guide, so it's helpful to use it as a reference during the actual interview, keeping in mind that you should not rely solely on the prepared questions and instead carry the flow of the conversation. (reference)
When writing the discussion guide, having a template seemed useful, so I went looking for one. Looking at its components, like the book describes, it consists of an introduction (where you list items to check beforehand without missing any), a background interview to build rapport, a main interview, and a closing.
In the closing portion above, an observer appears, who isn't mentioned in the book. They were probably omitted for length, so I went looking for additional content. For reference, 'A Guide to Qualitative Research' recommends having a participant observer (ethnographer) accompany you during in-depth interviews, so I organized related content additionally.
Why we discuss interviews and participant observation together
The main reason we discuss interviews and participant observation together is that the two methods share important characteristics. A skilled interview-based researcher is a competent listener, skilled at drawing stories out of anyone. Conversely, an excellent ethnographer is a sharp observer, skilled at sensing things others may miss — for instance, what a place looks like, what sounds you hear, what it smells like, what it feels like.
The evaluative criteria for the two methods actually differ, but they share some unique and important elements.
Care must be taken so that the researcher's individual preconceptions don't influence the research. This is a common problem across all kinds of data-collection methods. For example, in survey research, only answers to the questions the researcher has decided to ask are elicited, so the survey inevitably reflects the researcher's interests, point of view, and preferences. The unique element shared by interviews and participant observation is that the researcher doesn't only collect data — they 'produce' it — and the data collector is explicitly present within that data. Open-ended interviews are reactive interactions. The interviewer changes the wording, order, form, and content of questions or utterances from one interview to the next in response to what the respondent says or doesn't say. Therefore, the researcher's own words, which change from interview to interview, are also data that must be analyzed. The same goes for participant observation. Even the most passive researcher, just by being on site, naturally affects the observation results. Furthermore, when you consider that the ethnographer themselves writes every single one of the tens of thousands of words that make up the field notes, the observer too is unavoidably a part of the data itself.
When collecting in-depth interview or observation data, the researcher reacts in real time to events unfolding in front of them.
The researcher reacts to what someone says, asking a question or staying silent, becoming interested in or indifferent to a particular aspect of the unfolding event, deciding whether and how to convey what they observed. These unpredictable reactions are at the core of the data produced through qualitative research.
Next is research questions. For reference, research questions and interview questions are different.
Because interview questions are constructed based on research questions, this might be the most important and the most difficult part. The book 'UX Research' deals with this in a very fundamental way. Research questions aren't simply about asking and answering — they require constructing more fundamental questions that explain the purpose of the research and reveal to users your research design and focus.
Within the UX Research content, the topic of strength of evidence caught my eye. In summary, researchers should pay attention not to users' 'words' but to their 'actions.'
But for a moment I thought — uh? what? Because earlier, when classifying research types in the research matrix, the criteria 'user's words and actions' was used to divide types.
But thinking about it a little more, you can see that here 'words' isn't simply the literal wording itself; it's emphasizing that you also need to understand the background and reasons why the user — at a personal level — has no choice but to say something, or — historically/culturally — has no choice but to think and say it that way.
To give a slightly different example —
When discussing whether user research is necessary, many people quote Henry Ford and have the misconception that "don't ask the user — users don't know, they just want a faster horse." But here too, as in the earlier book, what's being emphasized is that 'words' are not the literal wording of what they say, but the importance of understanding the reasons why they have no choice but to say it that way.
In addition, 'UX Research' cites Tinbergen's four questions, emphasizing that to understand their behavior — to construct good questions — you need to consider the purpose of their behavior, the cause-and-effect of how that behavior works, how it has developed in their personal circumstances or environmental side as a way to overcome them, and how they adapt to the organization or society they belong to, and what behaviors and choices they make in service of that adaptation.
What is the purpose of the behavior?
This question is about the function of the behavior. What is the behavior currently used for, and what is its value? How does the behavior currently help the individual in their environment?
How does the behavior work?
This question is about cause-and-effect or control. How do causal factors control the behavior? What kind of stimuli elicit the behavior? What are the neurological, psychological, and physiological mechanisms that affect the behavior?
How did the behavior develop?
This question is about the development of behavior in the individual. What in the individual's daily life triggered this behavior/choice? What internal or external factors influenced its development? How does the developmental process work? In the course of development, what is the nature of the interaction between the individual and the environment?
How did the behavior evolve (adapt)?
This question is about the development of behavior in a population. What factors, over the course of a particular group's persistence, allow them to shape the development of their behavior?
And the book introduces a more detailed approach to constructing interview questions, along with examples. As shown in the items above, it explains the overall usage process and the reasons for it, and describes the cues for the emotions felt during that process — using concrete cases.
In addition, to infer the user's mental model, it introduces items that compare user experience, motivation for behavior, experience with others, and experience at a particular point in time.
For reference, when constructing questions, it would be even better to consider the structural requirements of the later data analysis stage in advance, so I looked up some material. For example, building a mental model requires content on cognitive space, behavior pillars, and unit behaviors, and the persona-derivation process organizes analysis content around usage patterns and interview keywords. So if you take that into account in advance when writing your questionnaire or description guide, it can help in constructing the questions.
How to conduct user interviews
Methods for digging out thoughts, Part 2
medium.com
There are points to be careful about when writing questions: it's better to avoid questions that ask too directly, ask about content that's so old it's hard to remember, or ask about future choices that are hard to predict.
After a long preparation process, when interview day finally arrives, you can put together a rough timeline alongside the previously written discussion guide, as shown above.
During the interview, rather than relying solely on the user's memory and words, if you can also use techniques like card sorting or user journey mapping that allow you to observe the actual usage process, it can help much more in interpreting and understanding their actions and the causes — not just their words — as discussed earlier.
Finally, the post-interview analysis process. Although it's not in the scope this time, briefly wrapping up this technique: it consists of the steps of compiling the conversation content and removing bias, the coding stage of grouping and creating themes, the analysis stage of drawing out themes and patterns, and finally the reporting stage of writing the results.
Fortunately, there's a case where analysis was actually performed, so I'll share that link.
Week 6: Graduation project research 6
1813097 Park Sua, Visual Communication Design Department, Graduation Project Studio I — This week we did the analysis on the research.
ahsuah927227517.wordpress.com
A post I'd been putting off for nearly two weeks — done. ㅋㅋㅋ
