Rasa Voice Interface - future plans?

Having seen the blog post on building a voice assistant I’ve long been meaning to try it out and finally did this weekend :slightly_smiling_face:

It’s great, thanks a lot for putting it together @juste I had quite a few road bumps on the way to getting it working, and am interested in finding out more about the general direction for the Voice Interface.

Q. Are the repo and/or blog post expected to be updated much?

There’s a few points where it might do with some improvement.

I don’t know if others had these issues but I couldn’t get the TTS working in the same process as DeepSpeech and the way it’s architected, it seems like it risks problems whenever any of the three underlying projects get updated (especially regarding python dependencies). I ended up running the TTS demo server as a separate process and calling that via requests (replacing most of the TTS code in the connector).

There is a bug with CORS, and another one with Sanic (for which there’s a workaround), there’s that duckling error too (although it seems it’s fine to ignore), the dependencies are a bit of a nightmare (and barely touched on in the blog) and a few details are missing (eg installing Sara skips the requirements.txt step). Nothing impossible to figure out but it might help others if they could be updated.

Q.Are there any plans to simplify the voice demo?

Sara is cool but it does seem to require various things that are peripheral (eg the duckling docker image just adds needless complexity; it has a bunch of dependencies for things that Sara uses but aren’t really necessary for voice, eg for the newsletter sign-up etc)

One small point on the blog, in case it helps others, the original link I’d arrived at from Twitter was for the AMP version and in that version none of the script sections from GitHub showed up correctly. As there were a number of other instructions missing in the main text (eg directories to do things in) I initially thought the script sections mentioned had been missed out the blog in error. It was only when I got to the connector code section that I was stuck without the complete script and spotted the AMP thing :slightly_smiling_face:

For anyone interested, here’s a video of my installation working: https://youtu.be/C2LYAU39rA4

3 Likes

Hi @nmstoker, The video you’re sharing looks cool, but why did I end up with this problem following the tutorial?

And what appears on the left is not audible.

Hi @Neary-li

I’m sorry but I think it will be quite difficult to get to the bottom of why that error is coming up.

As mentioned I also had quite a few difficulties getting it working: the repo was pretty old last time I checked and it’s also quite a complex setup overall so there could be a huge number of things that have gone wrong, either with your setup of it or with the code no longer working unchanged with current browsers and/or backend infrastructure.

If you’re keen to use it, I would suggest trying to get in touch with someone from RASA on the forum to see if they have plans to maintain the demo - if it’s being maintained then that should reduce the chances that it has become incompatible with current tools (on the RASA side and with modern browsers).

If you can’t do that but wish to proceed anyway then I think you’d need to start debugging the front-end code. From what I can see there may be a problem with it listening to the mic, so it might be that your browser is blocking permission or that the way that works has changed (eg where the error message mentions Recorder not defined with startUserMedia)

I know that hasn’t solved your issue but I hope it’s at least a help in where you might start. Best of luck!

Kind regards, Neil