story: site accessibility path different site
steps:
intent: greet
action: utter_greet
action: utter_help
intent: site_accessibility_issue
action: action_set_focussed_intent
action: utter_ask_application_context
intent: choose_current_application
entities:
current_application: false
slot_was_set:
current_application: false
action: action_verify_select_application
intent: choose_app
action: utter_express_sorry
action: action_serve_curated_answers
action: utter_ask_bot_helpful
story: site accessibility path current site
steps:
intent: greet
action: utter_greet
action: utter_help
intent: site_accessibility_issue
action: action_set_focussed_intent
action: utter_ask_application_context
intent: choose_current_application
entities:
current_application: true
slot_was_set:
current_application: true
action: action_verify_select_application
action: utter_express_sorry
action: action_serve_curated_answers
action: utter_ask_bot_helpful
You can clearly see that in these 2 paths, the deciding takes place when someone chooses whether or not the Application is current or a different application to select for the next course of actions / steps.
The issue definition:
The problem arises when someone decides to go for current_application: true
and the program instead of predicting the path “site accessibility path current site”, abruptly ends with no output. However if the user selects current_application: false, then path “site accessibility path different site” is correctly predicted.
On a reverse scenario if I juxtapose the stories upside down, i.e.
a) - story: site accessibility path current site
b) - story: site accessibility path different site
instead of
a) - story: site accessibility path different site
b) - story: site accessibility path current site
the prediction behavior reverses selecting the path “site accessibility path current site” correctly and abruptly ending without selecting “site accessibility path different site”.
So now the question is, how can I sequence the stories where there are common entities and bifurcations on a deciding factor like this case?
any angle of thought would be highly appreciated, because I am too much biased on my own presumptions.
I just run a data validity and it gave me some futureWarnings and couple of userWarnings :
/root/.virtualenvs/chatbotrasaproject/lib/python3.6/site-packages/rasa/shared/utils/io.py:93: UserWarning: The example ‘good afternoon’ was found labeled with multiple different intents in the training data. Each annotated message should only appear with one intent. You should fix that conflict The example is labeled with: goodbye, greet.
/root/.virtualenvs/chatbotrasaproject/lib/python3.6/site-packages/rasa/shared/utils/io.py:93: UserWarning: The example ‘I have some issues’ was found labeled with multiple different intents in the training data. Each annotated message should only appear with one intent. You should fix that conflict The example is labeled with: mood_unhappy, site_accessibility_issue.
/root/.virtualenvs/chatbotrasaproject/lib/python3.6/site-packages/rasa/shared/utils/io.py:93: UserWarning: The example ‘I had some issues’ was found labeled with multiple different intents in the training data. Each annotated message should only appear with one intent. You should fix that conflict The example is labeled with: mood_unhappy, site_accessibility_issue.
/root/.virtualenvs/chatbotrasaproject/lib/python3.6/site-packages/rasa/shared/utils/io.py:93: UserWarning: The example ‘What does the organization or company or Informa work on?’ was found labeled with multiple different intents in the training data. Each annotated message should only appear with one intent. You should fix that conflict The example is labeled with: organization_function, organization_structure.
Would that be some of the contributing factors? Although I agree I need to fix these, which I will, eventually.
I don’t think these are contributing factors, the warnings are all to do with your NLU data.
I do notice that you have a very small number of epochs for TEDPolicy. I’d suggest training for at least 100 epochs, probably more like 200 or 300.
You might also consider using rules for this case. Rules are useful for parts of the story that you know always want to be handled a certain way. This could look something like this:
@fkoerner ,
1 dumb question, should I move all the conditions mentioned in the stories and create rules for each corresponding story? SO, suppose I have similar pairs of stories which deals with different questions but similar path patterns ( “…path current site” and “… path different site” ) are quite similar, like:
story: site accessibility path different site
story: site accessibility path current site
story: generic dissatisfaction path different site
story: generic dissatisfaction path current site
… and so on
on each story path there is at least one deciding intent (in this specific question description mentioned as: site_accessibility_issue ) which is then processed in actions.py method action_serve_curated_answers to call intended API to fetch answer. The action action_serve_curated_answers is a common method to handle several story paths.
So the bottom line question is :
How would the stories be rewritten if these rules are the way you said? would there be only 2 rules generally written as you mentioned that would work for all the story patterns? or there would be different rule pairs for each story-pairs?
I am being too demanding, I understand, but this to admit that I am learning this rule usage for the first time, from amazing you…
also to emphasise that the rasa train is halted stating “Contradicting rules or stories found” because I kept the stories.yml as is, ignorant of the changes to be made there.
@tuhinsom there are no dumb questions, we’re here to help and if something’s unclear we want to improve that!
should I move all the conditions mentioned in the stories and create rules for each corresponding story
The answer to this is case-by-case. Rules should be used carefully: if a rule applies in a situation it will always take effect. If it almost applies (meaning, for example, there is only one small difference), the system can’t generalise. So you are probably best off using a combination of rules and stories. This is also why rules should be kept short.
How would the stories be rewritten if these rules are the way you said?
It sounds like you have a contradiction, could you share this? Then we can look at how the stories should be changed to account for the rules.
would there be only 2 rules generally written as you mentioned that would work for all the story patterns? or there would be different rule pairs for each story-pairs?
Again, you want the rules to be short. This means you shouldn’t turn a whole story into a rule. But rules are snippets and can be pieced together combinatorially, provided that the rules apply. The RulePolicy will use the longest applicable rule for prediction, regardless of which rule has or hasn’t been applied in the past.
Hi @tuhinsom, I see the issue now. I think your problem is that the the action taken after:
- action: action_verify_select_application
is dependent on quite a few turns in the past. All three of:
generic dissatisfaction path current site, searching issue path current site, and site accessibility path current site contain the following:
but only the site accessibility story predicts the action utter_express_sorry, which is dependent on the intent site_accessibility_issue quite a few turns ago. One way to disambiguate these would be to set a slot with the user’s dissatisfaction reason when they utter one of the intents: site_accessibility_issue, searching_issue, generic_dissatisfaction. You could call this slot dissatisfaction_reason. Then, you can write a rule for the different combinations of current_application + dissatisfaction_reason. Don’t forget to add the slot to your domain!
@fkoerner , I now see where you are getting at.
I was trying to achieve the same conditional treatment by looking at the intent and taking the next step in actions:
class ServeCuratedAnswers(Action):
def name(self):
return "action_serve_curated_answers"
def run(self,dispatcher,tracker,domain):
if tracker.get_slot("focussed_intent") == None :
latest_intent = tracker.latest_message['intent'].get('name')
else :
latest_intent = tracker.get_slot("focussed_intent")
ent = tracker.latest_message['entities']
# for name in confvars.options('DEFAULT'):
API_endpoint = getSysConfig('DEFAULT','CURATION_ENDPOINT')
entities = []
if len(ent) > 0 :
for eachEnt in ent:
entities.append({ eachEnt['entity']: eachEnt['value']})
if latest_intent == 'site_accessibility_issue':
message = API_endpoint + 'Sorry, you faced the issue, let us find some solution for you';
response = callCloudAPI(API_endpoint, 'post', {'entities': entities})
resp_msg = [{'response':json.loads(response.text),'bot_message':message}]
elif latest_intent == 'generic_dissatisfaction':
message = 'Sorry, you feel dissapointed, please read the following article on the issues:'
response = callCloudAPI(API_endpoint, 'post', {'entities': entities})
resp_msg = [{'response':json.loads(response.text),'bot_message':message}]
I appreciate your point of view and will try to adapt.
Will come back here by letting you know the outcome of the changes I make.
Oh I see! @tuhinsom, in general it’s best to let the rasa system predict the next action. Otherwise, if prediction is “hidden” from the system, it will fail to generalise, and also distinguish between cases. As a general rule of thumb, you should only output an action from a custom action if this does not branch and isn’t expected to affect prediction.
Let me know how this goes, I’m confident we can figure this out
@fkoerner ,
after all the changes I made I am stuck on the blocker on my newly bought Mac M1 chip issue with rasa currently. Not sure when I will get to check it. Although an option is there to setup. a docker of linux and try to setup rasa on it anew, but that would downgrade the performance, which is the primary objective of buying a Mac M1 with 16 core neural engine.
1 question on this, did you leave out the slot - reason_dissatisfaction: part from the trailing rules in your example, deliberately?
However, I need the action_serve_curated_answers action at the end of every rule or story to execute with proper parameter value (reason_dissatisfaction value is all I need there to call the correct api), utter_express_sorry is not enough to end the conversation in any case. But I understand what you mean.
I am stuck on the blocker on my newly bought Mac M1 chip issue with rasa
Yep, we’re aware of a number of users blocked by this. We’re working on it!
did you leave out the slot - reason_dissatisfaction: part from the trailing rules in your example
I did this because these rules apply whenever reason_dissatisfaction isn’t accessibility, i.e. it could be something else, like searching_issue or generic_dissatisfaction
@fkoerner ,
I have stuck into the following issue (as I was trying to do rasa train) after I upgraded the TensorFlow to 2.4.0-rc0 from 2.3.1 which supports M1 chip:
File “/Users/tuhinsom/devel/python_projects/virtual-assistant/chatbotrasaproject/lib/python3.8/site-packages/rasa/utils/tensorflow/temp_keras_modules.py”, line 37, in TmpKerasModel
@ training.enable_multi_worker
AttributeError: module ‘tensorflow.python.keras.engine.training’ has no attribute ‘enable_multi_worker’
and I checked that the tensorflow library tagged 2.4.0 training.py file actually did not have the method enable_multi_worker
Hi, yes, there is currently no way to run Rasa on an M1 mac because it is not yet compatible with tensorflow 2.5. We’re working on this, and will update this post when done
