Hi, I have an end-to-end model for (very specific) intent classification, and I like to execute a standard pipeline if my model does not recognise the intent. Is this feasible in Rasa?
1 Like
Hi @dbrick. Welcome to our community forum!
One way to handle this is through a combination of NLU fallback and a custom action.
The intent classifier will always assign an intent with some confidence. In your domain, you can define fallback behaviour which gets triggered if the model confidence falls below a threshold that you set in your configuration file. Please see the documentation here: Fallback and Human Handoff
Can you say more about the standard pipeline that you want to execute? If it’s a sequence of steps that you can write programmatically, then you can handle this with a custom action: Custom Actions
Please let me know if this solves your problem or if you require further assistance.
2 Likes
Hi @Aciel, Thank you so much for your prompt reply and my apologies if I was not clear enough.
I’ve created a custom component and it works well when it is the only line in the pipeline. I would like to exit the pipeline earlier when the confidence of my classifier is above some threshold. Not sure if the threshold argument is applicable here. In the code bellow the rest of the pipeline is executed even if the component returns a threshold above 0.5.
- name: custom.name_nlu.nameClassifier
- name: WhitespaceTokenizer
- threshold: 0.5
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
...Thanks for clarifying, @dbrick.
Looking at the example config that you’ve shared: are you not using any of our featurizes / tokenizers as input to your custom component? Does your component operate at the string level?
Also, did you mean:
- name: custom.name_nlu.nameClassifier
- threshold: 0.5
- name: WhitespaceTokenizer
...
where the threshold applies to your classifier rather than the WhitespaceTokenizer?