Voice enabled chatbot demo

Thought I’d share a quick video of an updated version of a chatbot that I’d created originally quite a while back, when Rasa was brand new.

Recently I updated it to work with Rasa 2.0 and to voice enable it for demo purposes, I was able to integrate it with a simple voice UI tool I’d put together Simple Speech Loop which here is using Wav2Vec 2.0 via Huggingface and TTS based on a model I trained using public domain audio from LibriVox.

I’m planning to update the repo for the bot in the next few weeks, so you can see how it works. The basic idea is to turn a variety of questions into queries that run against a single table for the chatbot’s topic (kings and queens of the UK, just for demo purposes)

2 Likes

@nmstoker this is so awesome! Lots of folks are interested in creating voice enabled bots, I think sharing this demo and updating the code is really going to help the community - in fact even had someone asking about this today, so I will be sharing this demo with them.

Really look forward to your updates! Great work! :slight_smile: :rocket: :rasahero:

2 Likes

Hi @Emma - thank you so much for the kind words.

I’ve put a “first cut” of the updated code here: GitHub - nmstoker/lockebot: LockeBot: a demonstration of implementing a basic question answering bot with use of Rasa NLU and a database

It still needs a bit of a clean-up but it should work with Rasa X now (I’ll expand a bit on the steps needed to do that but it’s standard stuff).

The steps to locally voice-enable aren’t there yet but that should be added fairly soon too.

For the voice part, it integrates with the regular REST connector via SimpleSpeechLoop: it’s largely a matter of me describing how to hook that up and to update the TTS/ASR I used in the demo which isn’t quite the same as in the github repo (although obviously people can plug in whatever they want there)

2 Likes