The success of every digital product lies in understanding our end-users, especially in a direct face-to-face context. This is typically achieved through qualitative user research activities (such as user interview, contextual inquiry, think-aloud usability testing etc), where we seek to uncover people’s needs or receive useful feedback for design validation.
Think-aloud usability testings are a good method to gather useful feedback from end-users.
Despite our best efforts in research recruitment, each and every person is different with their backgrounds and motivations. This makes our findings interesting, but it can also be confusing and even frustrating at times, through a real psychological phenomenon known as the value-action gap.
What is the value-action gap?
The value-action gap refers to the discrepancy when the external actions of an individual does not correlate to their internal beliefs or thoughts. In short, it is the difference between what people think and what they do, something that happens a lot in real life. This is not surprising, given that we are all guilty of not practising what we preach from time to time, in one way or another.
"Nope, this doesn't happen to me."
The truth is, human behaviour is malleable to a certain extent, depending on the situation or context.
When certain actions are needed for a specific moment, people are shaped by their environmental context (where they are) or personal knowledge (what they know). They try to translate these factors into favourable outcomes, behaving in ways which might not be aligned with their initial thoughts.
To illustrate this, let’s take a look at the following example to see how environmental context and personal knowledge can contribute to the value-action gap.
Ben is a 35 years old data scientist who is committed to his well-being since he is predisposed to diabetes. He practises healthy eating and exercises daily, even going for annual check-ups. One day, Ben is invited to a Christmas buffet gathering with some of his closest friends. During this festive occasion, Ben decides to cheat on his health commitment and indulges in merry-eating and drinking with his friends.
“It’s alright, it’s only for today, I’ll do two rounds of exercise tomorrow.”
By dissecting Ben’s behaviour, we can see how the environmental context (Christmas, buffet, close friends) and his personal knowledge (confidence in his health rituals, check-ups) had led him to deviate from his beliefs. The fact that Ben went for the favourable outcome and acted against his commitment is a compelling case of our susceptibility to the value-action gap.
How does value-action gap affect user research?
For obvious reasons, having participants say (or do) what they don’t mean is confusing from the user research perspective. At its worst, the value-action gap might render the research effort useless, as it is not indicative of how the participant actually behaves in real life.
Imagine this scenario (adapted from a real-life example):
The design team of a job portal is enhancing a feature to facilitate better skill-matching with employers and job seekers. Here, they want to understand how employers use the current job portal in regard to:
- Overall recruitment pipeline (via introductory user interview)
- Job posting workflow (via think-aloud usability testing)
One employer, Recruiter A, mentions in the user interview that she prefers to download applicants’ resumes without specifically looking at the skills listed in their profiles. When the user researcher probes deeper, Recruiter A explains that she is skeptical over the authenticity of skills in the job portal as it can be fabricated by the applicants.
However, in the subsequent usability testing, the user researcher observes Recruiter A spending a considerable amount of time in the skills input section, a behaviour which confuses the design team.
Based on the above scenario, can we really consider Recruiter A’s observed behaviour (action) to be truthful, even though it contradicts her initial stance on skills (value)?
While this example of the value-action gap is unexpected, here’s how we can address it during qualitative user research.
What can we do?
For a start, we need to pre-empt the environmental context.
Frankly speaking, participating in user research can be an awkward experience, especially for first-timers. Most forms of qualitative user research take place in controlled locations and in formats which are unfamiliar to the participants. As such, some people might adopt a safe mentality for fear of doing something wrong. On the other hand, others might seek to make a positive impression via the Hawthorne effect.
Bearing this in mind, we want our participants to be as natural as possible. To do this, we need to build trust and rapport by making the research settings comfortable for them.
We don’t have to befriend our participants per se, but there are some methods to ease their preconceptions and make the subsequent interactions more authentic:
- Before the actual user session, meet the participant at a location that’s different from the testing/interview location.
Introduce yourself in a casual manner and explain the upcoming research in the most understandable way to put the participant at ease. - Make small talk with the participant.
Ask them about their day so far, what they are going to do after the user session etc. - Speak in their tone of voice* (as where and when possible).
If the participant speaks colloquial English, avoid speaking Queen’s English. Try to be relatable to your participant in order to mimic their normal environment.
*Note: You need to make sure that any terms/phrases used by the participant are translated correctly and consistently during the research synthesis stage.
Next, let’s address the personal knowledge factor.
As mentioned earlier, it is only human nature to go for favourable outcomes in everyday life. While we encourage our participants to be honest, there are times when they might subconsciously lapse and do (or say) something contradictory to their beliefs.
If this happens, we need to understand what is exactly going within the participant’s mind by clarifying things on the spot.
Using Recruiter A as an example:
- Ask the participant about his/her actions and gently remind them of their initial statement.
“I noticed that you have spent some time inputting skills for the job post. If I remember correctly, you initially mentioned your skepticism of skills in job portals. May I understand the reasons behind your actions?” - Understand if the contradictory action is part of the participant’s normal behaviour elsewhere.
“Do you always spend time inputting skills when you post a job in other job portals? If yes, can you elaborate why?” - Get them to clarify the gap factually.
“Thank you for explaining to me the reasons you input skills in other job portals. In this case, how do you clarify your initial perception of skills in job portals and what you have just mentioned to me?”
There are many ways we can approach participants, but the key idea here is to uncover their train of thought in real-life (as opposed to real-time which might be influenced by the current environment context). By doing so, we can establish if it is a one-off lapse or an outlier that can potentially lead to an interesting insight.
Conclusion
In summary, we need to be mindful that the value action gap differs in each and every person. Some people are prone to it, while others might be very consistent with what they say and do. Humans are very interesting creatures by nature, and it requires a certain effort and awareness to understand them.
By addressing the environmental context and personal knowledge factors, we as designers will be able to obtain more qualitative user research and translate them into more effective and impactful digital products.
Let’s research better, always!
