Dealing with silence in voicebot for rasa

Rasa Open Source
1

I am building a voicebot with Automatic speech recognition as the frontend and rasa nlu working as the bot. I want to handle situations such as -

  • silence: Silence steps:
    • intent: silence
    • action: utter_silence
    • intent: silence
    • action: utter_silence
    • intent: silence
    • action: utter_bye_silence

for this whenever a silence for 4 seconds is detected by ASR, it send ‘silence’ keyword to Rasa. Now rasa should reply with " Sorry. Can you please repeat" for the two consecutive silence and then the third consecutive silence should give a reply " ok i will call you later ".

Problem - incorporating this in story is not helping. Silence can occur anywhere in the conversation. Please suggest a solution for this.

2

Hello @sandeeppandey456 ,

Can you please let us know what you are using for ASR. Also i m unable to understand why are you detecting the silence and sending it to rasa. For most of the ASR, the noise and the speaker verification can be done. For indepth analysis you can plot on MEL spectogram.

So my suggestion is rather than sending keyword to rasa. Just create a list in ASR where you count no of times of silence(variable a) and no of seconds of silence (variable b). Whenever the values equal to your desired condition you can utter a local response rather than sending to rasa. This way it is faster and more accurate.