Rasa 3.1: Two Stage Fallback hangs after default_ask_affirmation

karan · November 23, 2022, 1:50am

I followed the guide from here.

rules.yml:

rule: Implementation of the Two-Stage-Fallback steps:
- intent: nlu_fallback
- action: action_two_stage_fallback
- active_loop: action_two_stage_fallback

domain.yml responses: utter_ask_rephrase:

text: Can you please rephrase the question?

pipeline:

name: FallbackClassifier threshold: 0.7

policies:

name: MemoizationPolicy
name: RulePolicy
name: UnexpecTEDIntentPolicy max_history: 5 epochs: 100
name: TEDPolicy max_history: 5 epochs: 10 constrain_similarities: true

After confirming an intent (during the default_ask_affirmation), I get the below error and it hangs:

2022-11-23 09:44:33 DEBUG    rasa.core.processor  - Received user message '/affirm' with intent '{'name': 'affirm', 'confidence': 1.0}' and entities '[]'
2022-11-23 09:44:33 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 10 events.
2022-11-23 09:44:33 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'ed50f33f7f274678b167b68a15650a8e'.
2022-11-23 09:45:33 ERROR    asyncio  - Task exception was never retrieved
future: <Task finished name='Task-14' coro=<RestInput.on_message_wrapper() done, defined at /Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/rest.py:33> exception=KeyError('action_two_stage_fallback') created at /Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/rest.py:71>
source_traceback: Object created at (most recent call last):
  File "/Users/karan/miniforge3/envs/rasa3-1/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/__main__.py", line 119, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/cli/shell.py", line 125, in shell
    rasa.cli.run.run(args)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/cli/run.py", line 126, in run
    rasa.run(**vars(args))
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/api.py", line 56, in run
    rasa.core.run.serve_application(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/run.py", line 225, in serve_application
    app.run(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/mixins/runner.py", line 145, in run
    self.__class__.serve(primary=self)  # type: ignore
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/mixins/runner.py", line 578, in serve
    serve_single(primary_server_info.settings)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/server/runners.py", line 206, in serve_single
    serve(**server_settings)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/server/runners.py", line 155, in serve
    loop.run_forever()
  File "connection_task", line 15, in connection_task
    from sanic.http import Http, Stage
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/http.py", line 146, in http1
    await self.protocol.request_handler(self.request)
  File "handle_request", line 112, in handle_request
    """
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/response.py", line 424, in stream
    await self.streaming_fn(self)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/rest.py", line 71, in stream
    task = asyncio.ensure_future(
Traceback (most recent call last):
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/rest.py", line 47, in on_message_wrapper
    await on_new_message(message)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/channel.py", line 89, in handler
    await app.ctx.agent.handle_message(message)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/agent.py", line 420, in handle_message
    return await self.processor.handle_message(  # type: ignore[union-attr]
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/processor.py", line 146, in handle_message
    tracker = await self.run_action_extract_slots(message.output_channel, tracker)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/processor.py", line 172, in run_action_extract_slots
    extraction_events = await action_extract_slots.run(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/actions/action.py", line 1200, in run
    intent_is_desired = SlotMapping.intent_is_desired(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/shared/core/slot_mappings.py", line 112, in intent_is_desired
    + SlotMapping._get_active_loop_ignored_intents(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/shared/core/slot_mappings.py", line 93, in _get_active_loop_ignored_intents
    form_ignored_intents = domain.forms[active_loop_name].get(
KeyError: 'action_two_stage_fallback'
2022-11-23 09:45:33 ERROR    asyncio  - Task exception was never retrieved
future: <Task finished name='Task-6' coro=<SignalRouter._dispatch() done, defined at /Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py:122> exception=ClientResponseError(RequestInfo(url=URL('http://localhost:5005/webhooks/rest/webhook?stream=true&token='), method='POST', headers=<CIMultiDictProxy('Host': 'localhost:5005', 'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Python/3.8 aiohttp/3.8.1', 'Content-Length': '68', 'Content-Type': 'application/json')>, real_url=URL('http://localhost:5005/webhooks/rest/webhook?stream=true&token=')), (), status=503, message='Service Unavailable', headers=<CIMultiDictProxy('Content-Length': '79', 'Connection': 'close', 'Content-Type': 'application/json')>) created at /Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py:195>
source_traceback: Object created at (most recent call last):
  File "/Users/karan/miniforge3/envs/rasa3-1/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/__main__.py", line 119, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/cli/shell.py", line 125, in shell
    rasa.cli.run.run(args)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/cli/run.py", line 126, in run
    rasa.run(**vars(args))
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/api.py", line 56, in run
    rasa.core.run.serve_application(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/run.py", line 225, in serve_application
    app.run(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/mixins/runner.py", line 145, in run
    self.__class__.serve(primary=self)  # type: ignore
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/mixins/runner.py", line 578, in serve
    serve_single(primary_server_info.settings)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/server/runners.py", line 206, in serve_single
    serve(**server_settings)
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/server/runners.py", line 151, in serve
    loop.run_until_complete(app._server_event("init", "after"))
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/app.py", line 1581, in _server_event
    await self.dispatch(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py", line 193, in dispatch
    return await dispatch
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py", line 163, in _dispatch
    retval = await maybe_coroutine
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/app.py", line 1144, in _listener
    await maybe_coro
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/app.py", line 1196, in dispatch_delayed_tasks
    await app.dispatch(name, context={"app": app, "loop": loop})
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py", line 195, in dispatch
    task = asyncio.get_running_loop().create_task(dispatch)
Traceback (most recent call last):
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/signals.py", line 163, in _dispatch
    retval = await maybe_coroutine
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/sanic/app.py", line 1202, in run_delayed_task
    await prepped
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/run.py", line 135, in run_cmdline_io
    await console.record_messages(
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/console.py", line 195, in record_messages
    async for response in bot_responses_stream:
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/rasa/core/channels/console.py", line 141, in _send_message_receive_stream
    async with session.post(url, json=payload, raise_for_status=True) as resp:
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/aiohttp/client.py", line 640, in _request
    resp.raise_for_status()
  File "/Users/karan/miniforge3/envs/rasa3-1/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 503, message='Service Unavailable', url=URL('http://localhost:5005/webhooks/rest/webhook?stream=true&token=')

Help please. I have tried:

Any help would be appreciated. Thank you!

ZahraDehghani99 · November 23, 2022, 4:44am

Hi @karan ,

I don’t know the exact cause of the error but I think you should change your rule in this way:

  - rule: Implementation of the custom two_stage_fallback
    steps:
    - intent: nlu_fallback
    - action: action_two_stage_fallback

I mean, delete the active_loop from your rule. I implemented my action_two_stage_fallback in this way with above rule and it worked for me.

karan · November 24, 2022, 7:27am

Unfortunately it doesn’t work for me :/… is your domain and config file the same ?

karan · November 25, 2022, 12:40am

Solved it!

Use return [UserUtteranceReverted()] instead of return [] at the end of the custom action class DefaultAskAffirmation.

Other settings are as described here.

Thanks everyone!

Topic		Replies	Views
Why does TwoStageFallbackPolicy keep asking for second affirmation even when the intent is correct? Rasa Open Source	7	1361	August 11, 2019
Two stage fallback Rasa Open Source	6	1297	October 28, 2021
The conversation is getting hanged after two stage fallback and after restart button Rasa Open Source	0	537	October 12, 2019
Two Stage Fallback unexpected behaviour Rasa Open Source	6	1197	July 30, 2019
Two-stage-fallback does not work when entities occur Rasa Open Source	2	285	July 7, 2021

Rasa 3.1: Two Stage Fallback hangs after default_ask_affirmation

Related topics