In this post, I’m going to discuss how I conduct Stakeholder Interviews as part of a larger UX research effort. The techniques I use are fairly tried-and-true, and one can certainly develop a similar recipe by cobbling together tools easily accessible through a few Google searches. I’m going to use a current client, NAS Insurance, as my case study.
For the past year I’ve been working with NAS Insurance to design and build a very complex internally-facing ERP system called Snap. With a large and talented engineering and product team, we’re dramatically increasing the speed and efficiency with which this $100M specialty insurance company processes, underwrites, and manages policies for a wide-range of small and medium-sized businesses across the country.
For our internally-facing Snap system, we have the luxury of working side-by-side with the underwriters and policy services staff so conducting research and usability studies is relatively easy. With that, we’re able to quickly fix problems or implement temporary workarounds. Our Snap users understand that our development schedule is decidedly aggressive, and they are forgiving when we make a UX mistake. However, the next phase of our efforts will be broker-facing portal, and independent wholesale insurance brokers (our customers) will not be so forgiving.
As we began research for the Broker Portal, we learned that NAS Insurance executive stakeholders had a variety of strategic ideas, some of which were in conflict, and we realized that we needed to more fully understand this in order to design the right product. For that purpose, I devised a way to capture their thoughts. In a later post, I will discuss how we conducted user research and integrated those learnings with those from the stakeholders to determine the Broker Portal MVP—or Minimally Valuable Product.
Step 1: Define the Purpose and Format
The first thing I did was write a simple purpose statement to guide the stakeholder interview process. It’s straightforward, and one could probably use a similar statement for a whole variety of UX projects. Here’s ours:
To fully understand the strategic goals of stakeholders with regard to a Broker Portal, and with that ensure we design with those goals in mind.
Given that each of our executives is busy and has his or her own working style, I settled on asking each to present a 10-20 minute lightning talk. A lightning talk is simply a short, informal presentation on a particular topic in order to share information with others.
These lightning talks were presented in-person to 6 members of the product and development team, and the executives were free to create a PowerPoint presentation or simply talk about their vision, ideas, and strategies as they relate to their various areas of responsibility, i.e., marketing, underwriting, operations, etc.
However, it was crucial that the lightning talks had some common structure so that the information we received could later be organized and analyzed more effectively. To that end, I asked the executives to prepare their talks by answering the same four questions:
- Strategic Goals: What is NAS’ primary strategic goal that the contemplated technical solution (i.e., Broker Portal) would enable us to meet? Are there secondary and tertiary goals that need to be met?
- Brokers’ Problems: What are the top problems we want to solve for Brokers in order to achieve our strategic goals? What gets in the way of our brokers quickly getting from application to policy issuance?
- Competition: What (or who) is the competition? Answer should extend beyond other carriers selling specialty insurance. We want to understand the threats to NAS’ business that might be met with technology.
- Measuring Success: What is the ultimate definition of success? In other words, what will be the impact of the solution 3 months after release? 1 year? 5 years?
Step 2: Conducting the Lightning Talk Sessions and Taking How Might We Notes
With the vice-president of IT and Development as our champion (it’s very important to have a high-level champion to facilitate getting things like this done), we gave 6 executives several days to review the questions and prepare their lightning talks. We then scheduled 6 separate 1-hour sessions over a two-week period.
While the executives were preparing, I used the time to train the product team on the How Might We… listening and note-taking method. This method enabled the team to use a common note-taking format and empowered them to listen for insights or pain points and reframe them as areas of opportunity. Rather than explain the process here, I recommend you watch this great video: https://www.youtube.com/watch?v=sRGk5oKXgCk
Since we had several members of the product and development team all listening and taking notes in the same way, we were able to very constructively and efficiently share our notes as you will see in Step 3.
The stakeholder lightning talks were very informative. Limiting the number of questions to four helped focus the executives, and each was able to keep his or her talk within the 20-minute timeframe. Some brought slides and bullet points while others simply talked from each question. The chief marketing officer even got busy with the whiteboard. Each talk was then followed by roughly 45-minutes of Q&A with each member of the product team furiously taking “How Might We…” notes on 8.5×11 pads of paper. (Some recommend using sticky notes, but we found that cumbersome, and we transcribed from the larger pads to sticky notes in Step 3.)
Step 3: Note Sharing and Affinity Mapping
Following each stakeholder lightning talk session, I pre-scheduled a second 1-hour meeting with just the product team so that we could share our notes and insights amongst ourselves. While it was helpful to meet immediately while the session was fresh in our minds, logistics sometimes required that we meet a couple of hours later or possibly the next day. This didn’t present much of a problem.
During the first 5-10 minutes we transcribed our initial notes to sticky pads—1 idea or concept per sticky (very important). Each sticky note was prefixed with “HMW” or “How Might We…” followed by an idea, concept, or a feature related to an insight or pain point heard during the lightning talk.
The next part required a big, blank vertical surface onto which we could post our sticky notes. In our case, we had an expansive glass wall, but a regular wall or whiteboard would have worked just as well.
After transcribing our notes, the first member of our team stood at the wall and read each note aloud as they stuck it to the wall. It was helpful to start as high on the wall as was reachable and place each note horizontally with a some space in-between. If two or more notes described roughly the same idea or concept then they would be placed vertically in relation to each other.
Each team member then took a turn doing the same thing. If a sticky note represented a new idea then it would be placed at the end of the row of sticky notes. If the note represented an idea that was the same or similar to one placed by someone else then it was stuck vertically underneath. The exercise quickly became very interactive, and the entire team would decide together whether each note presented was a new idea or related to something already added to the wall.
At the end of the note-sharing, the wall would typically have 15-20 columns of sticky notes with each column representing a distinct idea or area of opportunity. Some columns might have had one or two notes while others had over a dozen.
We would then take a good, collective look at the columns of notes. Sometimes, we’d combine columns that we determined described the same thing. In other cases, we’d divide columns of notes into separate ideas. Usually, we ended up with 8-12 columns, each representing a key point or area of emphasis that we heard in the stakeholder session. As each column took its final form we then gave it a label.
In this way, we were able to discover affinity among our notes, collectively process and organize what we heard in the stakeholder session, name each area of emphasis with a label, and build consensus among ourselves as a product team.
Step 4: Voting
Faced with a wall of notes representing up to a dozen areas of emphasis, we know that it’s typically not possible to build a Web application that can solve for everything in the first release—not unless we had massive, Google-sized resources at our disposal. With that in mind, we needed to identify the top priorities. This is where voting comes into play.
While we’d built some solid consensus by grouping our notes, it was important to recognize that each product team member had their own ideas about priorities. This might be based upon how they heard the stakeholder, the volume of notes in a particular category, or they might have their own ideas about what’s ultimately important to the business or the end users. Whatever the case or bias, the opinions of each product team member mattered. They were hired for their expertise, right?
With all this in mind, I gave each product team member, including myself, 3 votes represented by colored sticker dots. I then instructed the team to place a dot on any column labels they deemed the 3 highest priorities. In fact, they could even place all 3 dots on a single label if they felt that was the only thing that really mattered. With this method we were able to build consensus about the strategic priorities for the broker portal as expressed during a particular stakeholder session.
Step 5: Aggregating Multiple Stakeholder Sessions
As mentioned in this case study, we conducted 6 stakeholder interviews, and each session included 6 team members generating roughly 15 sticky notes per session. If you do the math then you start wondering if 3M came up with this idea.
After each affinity mapping and voting session with the product team, I would take a picture of the sticky-noted wall and copy the notes into a spreadsheet that mirrored the columns. Yes, this is a lot of typing and a bit laborious, and, yes, there are virtual sticky note Web applications that can handle some of this (http://www.realtimeboard.com), but as the UX guy I need to internalize everything that was said in the stakeholder sessions and BE the expert. All that typing will do the trick. Of course, not all projects require this amount of effort, and a startup may only have a couple of co-founders to interview so don’t sweat it too much.
With the notes from each stakeholder interview in a spreadsheet I was then able to aggregate similar ideas expressed among various executives and create super categories. In some cases, we had an idea or category that was exclusive to a single stakeholder, but typically I’d find that a multiple executives emphasized the same things.
By combining product team votes across those categories and factoring how many executives emphasized a particular idea, I was able to build a convincing list of strategic priorities that reflected the input of the entire executive team. Certainly, this priority list is subject discussion among the stakeholders, and certain individuals (i.e., the CEO) might decide to rearrange those priorities, but the product and the executive teams are now able to get on the same strategic page with firm data and insight.
