Hosting a User Shadowing Workshop
Empowering a team to be more empathetic, user-centric decision makers
Summary
This observational research initiative was conducted to develop a comprehensive understanding of Operations Agents' workflows and needs within the Telehealth Operations Dashboard (TOD) product ecosystem. Through structured observations and teamwork, the effort yielded refined user personas and actionable insights that directly influenced product strategy and roadmap prioritization – successfully achieving its objectives. We also established a foundation for future user research efforts including “user feedback loops” which increased user-centricity and accelerated our design delivery time.
Our Team
I initiated and led this effort for my product design team.
3 product designers: Participated across every step
Operations Agent Supervisor: Co-coordinator, domain expert
8 Operations Agents: Users who were observed
User Research Partner: Consulted and observed sessions
Product Manager: Engaged in next steps to follow up on findings
Our Why
My team was assigned ownership of the Operations Agents experience (in addition to the Clinician/Provider experience); however, we had limited knowledge about their day-to-day responsibilities so the goal was to deepen our collective understanding of our users. Learn their roles, responsibilities, challenges, and opportunities to support them.
Definition of Success:
Personas: Refined role based user personas to scale our understanding and improve user research team wide
Product Strategy: Designers will be able to identify and understand core user challenges facing Agents within and beyond our product, beyond what they’re asking for, to influence product direction
Our How
Getting Started
Leadership and stakeholder alignment meetings to get buy-in and allot the time and prioritization of my team and the operations agents bandwidth to proceed.
Who to observe: I wanted a strategic sampling across all key Operations Agent roles so I met with the Operation Agent Supervisor who was my partner throughout this initiative. We observed representatives from each of these teams:
Level 2 support
Behavioral Health
Clinical Health
Pharmacy Support
Workforce Mgmt
Align method: 2 hr observations using Teams across 2 days
Align risks: No screen recording to protect patient privacy (PHI)
Align timing: At the end of the quarter given the Agent’s expected dip in activity due to the upcoming holidays
Data collection methods: I created a structured workshop observation template, persona template, and presentation template; all completed in partnership with my design team.
Participation
Attend shadowing sessions: asking questions and taking notes
Debriefing workshop: Collate notes and discuss our findings
Synthesized findings: Created 5 personas and 21 product suggestions
Present findings: Share with other designers and design leadership
I invited representatives from Product Management and Engineering from both the Operations Agent product team and the Clinician product team
Retro: I invited designers to share what they thought, felt, and learned from the experience and if they would want to do it again
Persona Example (1 of 5)
This image is intentionally blurry to protect Amwell’s user information
Workshop Screenshot
This image is intentionally small example to protect participant and user insights
Outcomes
Team and user empathy and understanding: Team agreed they felt more connected to users and would like to do this again
5 Refined Personas: Designers synthesized findings into 5 Operations Agent Personas (as opposed to the 1 persona we had for all agents when we took ownership of this product)
Direct product influence: 21 “Quick Wins & Friction Points” were identified and prioritized by perceived user impact and technical feasibility by the Designers. I worked with the Product Manager to introduce these changes into our roadmap
Improved collaboration & user-centricity: I introduced bi-weekly “user feedback loops” with Agents to collect generative and formative research going forward – to deliver better, more informed designs with them, not just for them
Scalable templates: The research and workshop templates can be leveraged in the future
Reflection
Workshop preparation: I could have conducted pilot observations one week before my team to prepare the workshop and present it to my team before their sessions started to better prepare, as a note taking template essentially – this was their first chance doing something like this and they didn’t have a lot user research experience so it may have accelerated our mutual understanding of why and how to think about this initiative
Great team bonding: I felt more connected to my design team, our users, and our product
Future looking: Plan to do this again with Clinicians. Aim at doing this quarterly and involving other designers, product managers, and engineers across the team