On a fresh install of rasa, running rasa data validate, the following UserWarning is emitted:
UserWarning: The example ‘good afternoon’ was found labeled with multiple different intents in the training data. Each annotated message should only appear with one intent. You should fix that conflict The example is labeled with: goodbye, greet
Since “good afternoon” can be interpreted as both an initial greeting (greeting intent) , and as part of ending a conversation (goodbye intent), a warning is em
Context:
(venv) $ rasa --version
Rasa Version : 2.7.1
Minimum Compatible Version: 2.6.0
Rasa SDK Version : 2.7.0
Rasa X Version : None
Python Version : 3.7.10
Operating System : Darwin-20.5.0-x86_64-i386-64bit
Python Path : …/venv/bin/python
I’m updating my example text for “goodbye” to “have a good afternoon”, but what is the correct way to allow the same utterance to match depending on context (e.g. IF greeting has already happened, then it must be goodbye. If goodbye has already happened, then assume it is greeting.
Basically. are we “in a conversation”
Yes → then this likely to be a “goodbye” intent.
No → then this is likely to be a “greeting” intent
In my opinion, sample data that comes “out of the box” installs should provide complete examples, without UserWarnings.
I would be glad to open a pull request, to fix the file in “/cli/initial_project/datga/nlu”, but is that the right fix? I’m still learning my way around Rasa but wouldn’t it be better to provide a default example that includes something that can disambiguate “good afternoon” during convo vs. to start convo?
Yes this is a good point, in fact this is a mistake on our end. You should not have identical examples for different intents, as our intent classifiers are context-independent (meaning what you described “allow the same utterance to match depending on context” is not currently possible). End to end training, which is currently experimental, may be one way around this, alternatively you can write a custom classifier or custom action to disambiguate certain utterances.
A PR would be great – the fix would be to remove the example from one of the intents. Thank you for your contributions as you get to know Rasa!
I’m wondering if the “right” fix is to create an intent, “salutation” (applies to both arrivals and departures), and then use a rule to say something appropriate to the start of a conversation only if the salutation is in the context of the start of a conversation.
rules:
- rule: Say `hello` when the user starts a conversation with intent `salutation`
conversation_start: true
steps:
- intent: salutation
- action: utter_greet
- rule: Say `goodbye` when the user provides a salutation during a conversaition
conversation_start: false
steps:
- intent: salutation
- action: utter_goodbye
Per the documentation, maybe the second rule is not necessary? Just stories that show salutation followed by a closing remark (e.g., “Have a great day!” or “It was nice chatting with you.”, etc.)
The salutation intent would include both initial greetings and wrap up greetings:
nlu:
- intent: salutation # both initial and final salutations
examples: |
- good afternoon
- hello
- goodbye
- hi
- bye
- later
- gotta go
Question: Are there other inherent conditions like “conversation_start” or is it unique? …or does it exist at all, I’m wondering if this example assumes a slot was set by an action server or something. I can’t fin any documentation on the “conversation_start” condition except the example in the documentation cited here.
@fkoerner do you think that is good solution? (I have not tested yet.)
Thanks for any suggestions or insight you can provide on this.
Hi @TomV, please excuse the delay in my response, I was out.
That’s definitely a viable solution, though I’m not sure I’d agree that “hello” ever means “farewell” . Strictly speaking you might want to have three intents, one for definite greeting, one for definite farewell, and one for ambiguous salutations. You could even use conditional response variations for this.
In general I’d like to strongly advise against designing stories such that the actions depend on the text of an intent, or the later predictions depend on the actual text of the response, as the policies don’t have access to the text and won’t be able to adjust predictions accordingly (disregarding end to end training, for the moment). In this case, I think the later actions don’t depend on the actual text of the intent or the response, so it should be fine.
Question: Are there other inherent conditions like “conversation_start” or is it unique?
conversation_start is indeed unique, other conditions can be slot values (set at any point in the conversation) or currently active loops
. Strictly speaking you might want to have three intents, one for definite greeting, one for definite farewell, and one for ambiguous salutations. You could even use