We've heard many stories about using a chatbot in customer support or the travel industry, but what is it like to create one from scratch?
I recently led the design for Voyager, an AI-powered travel assistant chatbot within the Hopper app. As the third-largest online travel agency in North America, this was our first venture into Conversational AI. While incorporating conversational AI in the travel industry may seem straightforward on top of my experience in conversational design, there were new lessons learned during its development. This project also ignited my curiosity in exploring the possibilities of Gen AI for product innovation!
Why did we build an AI-powered Voyager and what to build? 🪄
Our goal was ambitious - to transform the user experience starting from trip planning, grasp user interest in AI chatbot interactions, and enhance conversion rates. Typically, users visit Hopper to compare prices after deciding on a destination. But with the AI travel assistant, we can identify user intent early on and offer continuous assistance throughout the process.
As for the MVP (minimum viable product), we focused on creating a basic feature set enabling seamless conversations during travel planning and guiding users from the initial chat to search results for hotels, flights, or rental cars. The users would be able to ask some travel-relevant questions like, what’s the best season to travel to Yellowknife town for azure light? (This was the real question I asked!) and Voyager, the chatbot will provide succinct answers. If it detects the user's intention, it will further show the relevant booking information for the users to book.
What were the challenges?
My team was able to develop the essential conversational design prototype efficiently. However, we faced challenges while training the AI model and strengthening the design to ensure a positive user experience while mitigating any risks. Tuning the interaction was particularly challenging, but we persevered to create the best possible outcome.
Content quality: It’s the key to determining whether the users will continue to use it. We need to ensure the chatbot's content remains pertinent and conduct meaningful dialogues which is positioned as a “professional travel companion” in trip planning.
Destination hallucination: We made the chatbot to detect the users' intentions, and then display the relevant booking and provide the flight/hotel/car rental information for the users to book. But sometimes AI hallucinated the departure or destinations during the conversations, which impacted user experience.
Driving conversion: While we try to incorporate useful travel information, It’s a challenge to know the best time to guide the conversations to provide the flight/hotel/car rental information for the users to book but not sound pushy. The transition experience from instant chat to the search result is also critical.
Key Learnings: Enhancing the model and understanding the users' mindset
1. Improving conversation quality through testing:
To enhance the conversation quality, we conducted thorough testing of various prompts and implemented an evaluation process to assess the model. The discussions would be redirected back to travel planning if users went off-topic or touched on inappropriate content, like competitor details. We also trained the model to respond from Hopper’s perspective and debated answer format for better information richness without being too lengthy.
We fine-tuned the AI model to better grasp user inputs, prompting the bot to ask additional questions to establish travel intentions before displaying search results, thus avoiding presenting hallucinated destinations.
Establishing an evaluation process for answer quality:
We were enthusiastic about gathering direct feedback from the pioneers and introduced a straightforward feedback feature for users to share their thoughts with us. Additionally, we frequently examined user dialogues to assess Voyager's effectiveness in assisting users.
2. Creating a seamless user experience in conversational AI design with rich interface:
Concerned about users' familiarity with Voyager, I designed user-friendly conversation prompts to help them kickstart interactions and understand how to engage with Voyager efficiently. Exploring different methods for onboarding new users through A/B testing in the future would be beneficial.
Throughout the conversations, Voyager would present the booking options visually and the users can easily navigate between the conversation and search result page. We were not able to present more detailed booking option information inside the chat in the MVP scope, but it’s critical to display booking options efficiently in the chat without switching between pages, maintaining a seamless chat interaction flow.
3. Acknowledging the non-linear nature of trip planning:
Certainly. We hope Voyager can assist us in converting bookings. However, we understand that users usually do not book tickets immediately after engaging with Voyager for the first time. Therefore, we plan to delve deeper into analyzing user behavior data to better understand how different stages of trip planning affect decision-making and conversion rates.
4. Leveraging data for personalization
AI technology offers a key benefit in personalization. To enhance this aspect, we should consider integrating it with the customers' booking and behavior data. This integration and ML allow for crafting customized generative responses and offering personalized customer assistance.
Wrap up:
Creating Voyager was both challenging and rewarding, sparking my interest in the world of conversational AI. 🔥 I was especially excited when the team overcame obstacles like improving conversation quality and minimizing errors, resulting in a more reliable and beneficial Voyager. While there are areas for UX enhancement, like incorporating a rich interface with more details into the chat flow, this marks an exciting beginning in enhancing user experience in conversational AI and envisioning the potential of further integrations. This experience has strengthened my passion for AI innovation, and I am enthusiastic about developing an effective framework for building AI-integrated products. 🪄