{"id":62486,"date":"2024-05-24T21:37:45","date_gmt":"2024-05-24T21:37:45","guid":{"rendered":"https:\/\/explore.local\/2024\/05\/24\/conversational-voice-ai-for-ld-coaching-role-playing-and-more\/"},"modified":"2025-04-04T20:32:07","modified_gmt":"2025-04-04T20:32:07","slug":"conversational-voice-ai-for-ld-coaching-role-playing-and-more","status":"publish","type":"post","link":"https:\/\/degreed.com\/experience\/blog\/conversational-voice-ai-for-ld-coaching-role-playing-and-more\/","title":{"rendered":"Conversational Voice AI for L&D: Coaching, Role Playing, and More"},"content":{"rendered":"\n

Okay, listen. I wrote this entire blog post about an AI executive coach we\u2019ve been experimenting with and thought, \u201cAnyone reading this will probably just want to try it out first.\u201d<\/p>\n\n\n\n

Check it out<\/a>! It\u2019s not perfect, but it is kind of fun. (We\u2019ll pay for the token usage, so don\u2019t chat all night.) We\u2019ll be leaving the sign-up open for a few days.<\/p>\n\n\n\n

When you\u2019re done chatting, come back and read the rest of this post. And with that, back to our regular programming:<\/p>\n\n\n\n

Yes, the robots can talk.<\/strong><\/h2>\n\n\n\n

OpenAI announced on May 13<\/a> that a new conversational mode will be released in the next few weeks.<\/p>\n\n\n\n

These improved capabilities becoming ubiquitous will make chatting with AI verbally a regular way we interact with technology. So, what will it mean for L&D? And how did we get here? After all, voice assistants aren\u2019t new. So what\u2019s the big deal?<\/p>\n\n\n\n

We conducted a few experiments to find out, gathering input from L&D pros along the way. Join us for a look at the results\u2014and the implications. It\u2019s looking more and more like conversational AI will have a significant impact on some key aspects of L&D.<\/p>\n\n\n\n

\"\"<\/a><\/figure>\n\n\n\n

First, a Quick Technology Summary<\/h2>\n\n\n\n

Voice assistants like Apple Siri and Amazon Alexa have been around for a while. They use natural language processing (NLP) to take a request and match it to canned responses. This means they\u2019re helpful for checking the weather, but also, as Microsoft CEO Satya Nadella said in 2023<\/a>, they\u2019re \u201cdumb as a rock.\u201d They don\u2019t have the dynamic or generative abilities of a large language model (LLM) like that used by ChatGPT.<\/p>\n\n\n\n

ChatGPT changed the game. In 2022, Whisper<\/a> was introduced as a complement to ChatGPT-3.5, providing users the ability to convert audio into text. This allowed users to speak to ChatGPT, which could then read responses back at the click of a button. Audio and voice became usable, but the technology still lacked the ability to interrupt, be interrupted, or have a real conversation without strict instructions to take turns.<\/p>\n\n\n\n

Newer startups have enabled more conversational interactions on top of LLMs. They introduced the ability to automatically detect turn taking, allowing for interruptions and speaking freely back and forth. They also added nice interjections like \u201cmmm hmms,\u201d which occurred while the AI was listening. Further, they analyzed vocal expressions. Some latency still happened in these experiences as they were essentially a multi-step process; an LLM like ChatGPT-4 creates a response and then a voice agent, a separate technology, speaks the response.<\/p>\n\n\n\n

Then this month, OpenAI announced that ChatGPT-4o, the most recent version of the company\u2019s generative AI chatbot, will be able to natively understand and reply conversationally. This means that there will no longer be a voice agent reading responses from the LLM. The LLM will speak. If you tell the AI to \u201cslow down\u201d or \u201cpretend to be a character,\u201d it will. It will also be naturally vocally expressive and understand users\u2019 vocal expressions. Because it will all be built into the same system, it will be faster than anything that\u2019s come before.<\/p>\n\n\n\n

To date, a text model of GPT-4o has been released. The most advanced voice capabilities (including a controversial Scarlett Johansson-sounding voice) have not been released as of this publication. That means that right now, today, you can talk to GPT-4o, which is fast, but you have to tell the system if you want to interrupt (because it\u2019s still using the old voice input-readout technology). You can read details of what is currently available<\/a>.<\/p>\n\n\n\n

Log in today and you might see a screen similar to this when starting voice mode in ChatGPT:<\/p>\n\n\n\n

<\/p>\n\n\n\n\n\n

Whew! That was a lot. Let\u2019s talk about what this could all mean for L&D after the floodgates open.<\/p>\n\n\n\n

Our Hypothesis: Faster, More Authentic Interactions for Practice and Reflection<\/h2>\n\n\n\n

We had previously explored chat-based scenarios (via typing) as a form of practice and role play. These interactions were fun at first, but the effort required to make them feel real (while you knew it was an AI) was hard to sustain. It was also weird to play out scenarios via chat that you\u2019d be more likely to have in a real spoken conversation (like a call).<\/p>\n\n\n\n

<\/p>\n\n\n\n

We wanted to add voice capabilities to see if it would make the scenario feel more real and make it easier to engage.<\/p>\n\n\n\n

Experiment No. 1: AI Coach<\/h2>\n\n\n\n

We used GPT-4 Turbo as our LLM and added a conversational layer on top. We then instructed the assistant to act as an executive coach. Prior research has shown GPT-4 to be the most effective at role playing<\/a> (from a limited evaluation of other models).

This video captures my first experience with this combination:<\/p>\n\n\n\n

\n