Clinical AI Chatbot

User Research

Background: eo, a cannabis healthcare company, uses a large language model AI chatbot to educated people about medical cannabis. The bot answers questions, provides initial guidance, and onboards potential patients who might benefit from eo's care services. Unfortunately, AI chatbots can feel weird to use, especially in clinical settings. When a chatbot feels bad to use, people are less likely to follow through to the end of the conversation. In eo's case, this might lead to patients missing out on critical information and overlooking eo's care offerings.

Challenge: How might eo ensure that their chatbot feels comfortable, intuitive, and safe to use?

The Test

To improve the use experience of eo's chatbot, first I needed to understand how users felt.

Like the start of many other projects I've worked on at eo, this project began with userlytics, an online user testing tool. I designed the test to shine light on how intuitive the chatbot was to use, how relevant the conversations and recommendations were to the users, and how comfortable they felt disclosing personal information. I included a mix of multiple choice questions and open ended responses which testers answered after spending a few minutes playing around with the chatbot. I tested 100 people.

Analysis

After a few days, all 100 responses came in, and I began the arduous journey of analyzing the data. The multiple choice responses could be analyzed as quantitative data, which was easy enough. I ran the numbers in excel.

The hard part was analyzing the testers' open-ended responses. I did a first pass of theme tracking and idea categorization in excel, then mapped out key findings in a Figjam file using some techniques I learned in Human Centered Design class in college.

Synthesis

I illustrated my findings in a slide deck and presented it to the CEO, Chief Experience Officer, Head Designer, Head Developers, and the rest of the team at an all-hands meeting.

Here are some highlights from that presentation:

I ended the presentation with this list of action items:

Various teams across the company took these ideas to heart and made improvements to the chatbot, which is still live today.