Hi all, I am currently facing an issue with my stories. So currently I have 7 stories. All of them starts with user greeting the bot and being verified.
## if existing client chosen to create new ticket story 1
* greet
- action_verify_user
- slot{"valid_user": "Yes"}
- introduction_form
- form{"name": "introduction_form"}
- slot{"command": "create"}
- outlet_menu_form
- form{"name": "outlet_menu_form"}
- form{"name": null}
- problem_form
- form{"name": "problem_form"}
- form{"name": null}
- action_restart
## if existing client chosen to view existing tickets story 2
* greet
- action_verify_user
- slot{"valid_user": "Yes"}
- introduction_form
- form{"name": "introduction_form"}
- form{"name": null}
- slot{"command": "view"}
- view_ticket_form
- form{"name": "view_ticket_form"}
- form{"name": null}
- display_existing_ticket
- utter_display_existing_ticket
- action_restart
## if existing client chosen to enter new authorisation code and is successful story 3
* greet
- action_verify_user
- slot{"valid_user": "Yes"}
- introduction_form
- form{"name": "introduction_form"}
- form{"name": null}
- slot{"command": "entercode"}
- authorisation_form
- form{"name": "authorisation_form"}
- form{"name": null}
- action_validate_user
- slot{"authorisation_code_status": "code_pass"}
- action_restart
I won’t put all the story but just this 3 examples. So the problem that I am facing is with the stories intent. When I tried running the rasa shell in debug mode, I gave an intent that didn’t start with “greet”. I gave a “thanks” intent and the story continued instead of going to action fallback.
This is really weird. I figured the problem seems to lay with the stories since “Rasa shell nlu” match the intent correctly. I then went to experiment it by removing the stories one by one till I was able to get an action fallback due to not meeting the “greet” intent in the stories.
I just don’t seem to get what the problem was. Wondering if anyone could lend me a helping hand.
For my policies.yml, I am just using the default one from the guide. Still reading about it to see if it affects the stories.
If you are having multiple stories and you only want your bot to run stories based on those,
try
rasa train --augmentation 0
When you train a model, by default Rasa Core will create longer stories by randomly glueing together.
You can alter this behaviour with the --augmentation flag. Which allows you to set the augmentation_factor.
This helped me and drastically improved my accuracy as stories are no longer glued together. And all my non-Action stories are just a simple Intent-utter pair.
Thank you for the reply @gcgloven. Was reading up on agumentation but have no idea where to place them at!
Just an update, I just ran rasa train --agumentation 0 and it works! Now when I tried to start off with a non “greeting” intent, it would run action fallback which is what I wanted. And when I start off with a “greeting” intent, my stories would then continue to flow
Now I am wondering, what is AugmentedMemoizationPolicy. Only saw MemoizationPolicy in the docs! Would this be the equivalance of doing rasa train --agumentation 0?
Another thing I am wondering is do you think I should seperate out my stories. Since all of them are starting with the following:
About the story management part:
If your planned action flow follows this Greet story, I would suggest you keep it this way. With augmentation-- 0, you can now add in other stories such as generic Intent-Utter pairs.
A story is your bot’s reply logic flow. If you wish to have greet to stay the same, just keep the Greet story under one.
Alright will go and read it up! I am actually very surprise with augmentation. I read that augmentation is that Rasa Core will create longer stories by randomly gluing together. So if my stories all starts with Greet intent, I just don’t see how is it possible that when I gave a Thanks or Goodbye intent, my story would still be running.
That’s great ! I don’t really get what you mean by generic intent-utter pairs. Example my bot domain would be to allow users to create and view tickets. So having an generic intent would mean I play create and view into a generic intent like Inform intent?
Alright, thanks for the help once more ! I will keep all my Greet stories under greet.md!
@gcgloven so sorry to disturb but is augmentation suppose to be unstable? Because just now when I run it, it was working as how i want it to be. But now when I re train it, even my Greet intent goes to action fall back. Really weird!
Did you try it with zero max history?
another reason your conversation falls to default fall back is that your confidence level for that particular intent is very low. You also need to build up your training data.
Currently, I am not sure how many training examples you have.
Can you try rasa test in command prompt to see your overall performance for each intent?
If the number of your training examples is larger than 1k or very high, you may consider supervised embedding for your pipeline.
For my pipeline I am using pretrained_embeddings_spacy because I have very few training data.
I ran rasa shell --debug and what I saw was that they recoginise the intent was
Received user message 'hi' with intent '{'name': 'greet', 'confidence': 0.7605575743457091}'
They predicted the right action to follow next but however it still goes to action fall back. Yes I will go and try the rasa test now. So althought they predicted the intent was a Greet intent but still they went to action fall back really got me wondering!
Oh hi there again, so I went to test and make everything back to default, including clearing the database. I realise that it actually works. I was able to have the right Greet intent that follows the stories, as well as having fallback actions on intents that did not match the stories.
So I guess the problem was when I was going deep down into the conversations. Anyway is it possible for me to pm you to ask more questions about it if you don’t mind … haha !
Thanks!
