USAA Digital: Generative Research

A research study within the Chief Design Office (CDO) — identifying where human-centered design criteria falls through the cracks in product development at USAA.

This research fueled: USAA’s d:school: Bring Your Own Epic.

Details are left out due to confidentiality.

RESEARCH REPORT:

WHY THIS RESEARCH STUDY?
After creating a program to teach Product Managers the core tenants of design thinking, our team built on our offering to create a “team session”. Having launched efforts at IBM Design, our team had a strong hypothesis that a “team session” would work, but held many assumptions on how.

CHALLENGES WE ENCOUNTERED:

Multiple initiatives (SAFe, HCD, Adaptive Planning and Disciplined Execution) created ‘ways of working’ noise
Human-centered decision making not evident in products
An outdated understanding of design = is only visual design

PROBLEM TO SOLVE:
How might we ... identify where human-centered product development falls through the cracks while reducing ways of working noise?

IMPACT:

This report influenced Enterprise Executives 2020 corporate strategy on Cross-Functional Teaming, impacting 5,000+ employees.
This report influenced CDO Leaders 2020 strategic focus on Integration ... “we are more than champions of Design. We will create new expectations of cross-functional teaming and partnership”, impacting 300+ employees.
This report provided focus and fuel for Bring Your Own Epic (USAA’s d:school) that launched in July 2019.

MY PROCESS:
Exploratory Research ︎ Generative Research ︎ Synthesis ︎ Storytelling

🎤 Exploratory Research

We conducted a short sprint with CDO Directors and Producers, allowing us to tighten our focus, align on the right target segment and additional data to triangulate.

Sample Discussion Guide:

Describe a day when you felt fulfilled and were delivering value.
Co-design: who makes decisions impacting your work? who sets success metrics or milestones?
Co-design: sketch a project from beginning to end

Methods used:
• interviews
• co-design

Exporatory research enabled us to go wide on the problem space.

Stakeholder maps helped us understand who works with design and their relationships

Q: Why conduct interviews ... can’t you just send out a survey?

A: Focus groups and surveys, while they have their place, don’t lead to break through ideas. With our team launching a new offering, we needed to be steeped in the nuanced needs of employees at USAA – not just develop a carbon copy program from another company.

Q: What is exploratory research?

A: It begs to answer the question: “what is the research we should do next that will help our team generate lots and lots of ideas”.

🎤 Generative Research

Once we reduced our riskiest unknowns about who to talk to & what to talk to them about, we could now take a deeper-dive talking to CDO’s IT partners.

We observed the behaviors of our participants within their work environment and used physical artifacts as prompts for conversation and learning.

We used sketching to represent how they approach problem solving, who they collaborate with and the type of workflows in their day-to-day jobs.

Methods used:
• research plan, discussion guide
• contextual inquiry
• mind mapping
• user-journey mapping
• think-aloud

Diverse-perspectives was key in defining our initial research plan.

Co-designing with a mixture of methods helped facilitate more engaged and comfortable conversations.

Mental models & journey maps helped us understand what people do and why they do it.

Debrief guide ensure teams begin to mindmeld heading into synthesis.

Q: So what is generative research then?
A: It provides an opportunity for the product team to dig even deeper with target users to learn their feelings, dreams, needs and desires, in order to later develop data-rich concepts.

📝 Synthesis

After conducting 11 interviews our team transcribed interviews word-for-word to implant the raw data into our hearts & minds.

Next, we made observations, themes and insights by triangulating data from exploratory research, generative research and other industry standards, with the goal of making connections with disparate information.

A few challenges common during synthesis is staying grounded in your initial research objectives 🕳 🐇 and making sure you are identifying insights that are more than “common knowledge”.

Methods used:
• affinity diagramming
• triangulation

Bottom’s up analysis inspired by ac4d.

We immersed ourselves in the qualitative data for ~2 weeks — undistracted.

Synthesis — the key link between problem finding and problem solving. - Jon Kolko

Getting feedback on our insights from design partners.

🎭 Storytelling

Once insights were complete, we threaded a narrative to really make them stick — designers often make the mistake of presenting insights without any storyline or hook.

We had several metaphors that summarizing how churn occured between cross-functional teams ... but the Bermuda Triangle perfectly described the invisible tension between IT, business and design.

A simple napkin sketch from an early interview turned out to be the driving narrative we would thread with our insights to transfer the empathy we had gained to other colleagues across USAA.

Methods used:
• mood boards

We explored a range of metaphors.

Early napkin sketch during an interview

Bermuda Triangle mood boards

View the Full Report

(Password protected)

MY ROLE:
I was a design researcher, alongside senior researcher Melissa Chapman, defining a research plan, interviewing participants, conducting synthesis and creating a research narrative.

KEY LEARNINGS:

Conducting rigourous research in a vacuum is NOT a meaningiful outcome. You must transfer the empathy and knowledge gained through storytelling for true change to occur.
Creating artifacts (or boundary objects) during interivews helps facilitate more engaged and comfortable conversation.
There are many benefits to working on internal-facing products. End-users are easier to access and lack of resources means that stakeholders are comping at the bit for insights.