Problem: If a user replies with a message length over a certain character limit, I would like to have an auto-reply. Fallback is not triggering for most long messages when it should as the bot is incorrectly confident of the intent, so I want to have a rule to reply with a specific utterance instead.
I can only think to:
make a new intent entirely with long NLU examples and hope that works (doesn’t seem like it would)
or 2) create a custom action that will check tracker.latest_message[‘text’] after every single message and return a Boolean slot if the message is “too_long”. This seems cumbersome and inefficient, so I’m really hoping there is a better way to handle it.
Hi @zjntmo, you could write a custom NLU component which could set an entity whenever the message is above a certain character limit. You can then map a slot to this entity and write a rule based on that. Don’t forget to reset the slot in the end of the rule because otherwise that rule will get triggered after every turn.
Can you post your config, I am curious to know why the confidence of intent is high for long messages?
Also, if you are curious to try an alternative, you can try a branch sigmoid_loss with model_confidence=cosine. We are working on making the wrong predictions less confident and this is a step in that direction. We’ll soon be releasing this but you can already try it if you wish to by installing Rasa open source from that branch and then running an experiment.
@dakshvar22 My model is not currently set up to use entities as it caused confusion with a few API returns (I’m not sure how - I just started working on this model as part of a team). The model is not always overly confident about long messages, but it is often enough to be problematic.
Current config file. I’m also still trying to understand the basics of the pipeline setup (like, why do the Rasa example show “CountVectorsFeaturizer” twice in the same pipeline, one right after the other?), so I’m sure there’s optimization to be done there. I’ve wondered if having the LexicalSyntaticFeaturizer, RegexFeaturizer, and EntitySynonymMapper are detrimental since we are not using Entities, but I don’t know enough about them to dare remove anything.
We just finished upgrading from Rasa 1.8 to 2.2, so I don’t think I’d be able to convince the team to try a staging branch - but thanks for the suggestion! I’ll start digging into creating custom components and see if I can solve it that way.