Hello.
I’m trying to communicate with Rasa (v. 1.1.8) using the HTTP API. So, I find that while the “intent” (when sending a message) is correctly detected, on the contrary, “prediction” is incorrect (conserning the given stories, of course).
At the same time, sending the exact same message to the RestInput channel makes the right prediction and Rasa returns the expected response.
Hi @c.kalleas. By ‘prediction’ do you mean the prediction of the dialogue model or something else? Could also provide an actual example of this situation?
Hello again.
As “prediction” i mean the use of “http://localhost:5005/conversations/{conversation_id}/predict” method.
To be more specific, i made an experiment on the “rasa init” project. At first, i used the “rasa shell” environment. There, i had - as a user - the perfect conversation with Rasa given below : User: hi! Rasa: Hey! How are you? User: very good Rasa: Great carry on!
Then, using a Rest API client, i tried to run the same user senario with Rasa HTTP Api.
Curl:
curl -X POST -i http://localhost:5005/conversations/user/messages --data ‘{ “text”: “hi”, “sender”: “user” }’
Response:
{ “sender_id”: “user”, “slots”: {}, “latest_message”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “latest_event_time”: 1565064305.811917305, “followup_action”: null, “paused”: false, “events”: [{ “event”: “action”, “timestamp”: 1565064305.7650516033, “name”: “action_listen”, “policy”: null, “confidence”: null }, { “event”: “user”, “timestamp”: 1565064305.811917305, “text”: “hi”, “parse_data”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “input_channel”: null }], “latest_input_channel”: null, “active_form”: {}, “latest_action_name”: “action_listen” }
Curl:
curl -X POST -i http://localhost:5005/conversations/user/predict
Response:
{ “scores”: [{ “action”: “utter_greet”, “score”: 1.0 }, { “action”: “action_back”, “score”: 0.0 }, { “action”: “action_deactivate_form”, “score”: 0.0 }, { “action”: “action_default_ask_affirmation”, “score”: 0.0 }, { “action”: “action_default_ask_rephrase”, “score”: 0.0 }, { “action”: “action_default_fallback”, “score”: 0.0 }, { “action”: “action_listen”, “score”: 0.0 }, { “action”: “action_restart”, “score”: 0.0 }, { “action”: “action_revert_fallback_events”, “score”: 0.0 }, { “action”: “utter_cheer_up”, “score”: 0.0 }, { “action”: “utter_did_that_help”, “score”: 0.0 }, { “action”: “utter_goodbye”, “score”: 0.0 }, { “action”: “utter_happy”, “score”: 0.0 }], “policy”: “policy_0_MemoizationPolicy”, “confidence”: 1.0, “tracker”: { “sender_id”: “user”, “slots”: {}, “latest_message”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “latest_event_time”: 1565064305.811917305, “followup_action”: null, “paused”: false, “events”: [{ “event”: “action”, “timestamp”: 1565064305.7650516033, “name”: “action_listen”, “policy”: null, “confidence”: null }, { “event”: “user”, “timestamp”: 1565064305.811917305, “text”: “hi”, “parse_data”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “input_channel”: null }], “latest_input_channel”: null, “active_form”: {}, “latest_action_name”: “action_listen” } }
Curl:
curl -X POST -i http://localhost:5005/conversations/user/execute --data ‘{ “name”: “utter_greet” }’
Response:
{ “tracker”: { “sender_id”: “user”, “slots”: {}, “latest_message”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “latest_event_time”: 1565064408.6396431923, “followup_action”: null, “paused”: false, “events”: [{ “event”: “action”, “timestamp”: 1565064305.7650516033, “name”: “action_listen”, “policy”: null, “confidence”: null }, { “event”: “user”, “timestamp”: 1565064305.811917305, “text”: “hi”, “parse_data”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “input_channel”: null }, { “event”: “action”, “timestamp”: 1565064408.6396431923, “name”: “utter_greet”, “policy”: null, “confidence”: null }, { “event”: “bot”, “timestamp”: 1565064408.6396431923, “text”: “Hey! How are you?”, “data”: { “elements”: null, “quick_replies”: null, “buttons”: null, “attachment”: null, “image”: null, “custom”: null }, “metadata”: {} }], “latest_input_channel”: null, “active_form”: {}, “latest_action_name”: “utter_greet” }, “messages”: [{ “recipient_id”: “user”, “text”: “Hey! How are you?” }] }
Curl:
curl -X POST -i http://localhost:5005/conversations/user/messages --data ‘{ “text”: “very good”, “sender”: “user” }’
Response:
{ “sender_id”: “user”, “slots”: {}, “latest_message”: { "intent": { ** “name”: “mood_great”,** ** “confidence”: 0.9697415233** ** },** “entities”: [], “intent_ranking”: [{ “name”: “mood_great”, “confidence”: 0.9697415233 }, { “name”: “mood_unhappy”, “confidence”: 0.1109128371 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “affirm”, “confidence”: 0.0 }, { “name”: “greet”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “very good” }, “latest_event_time”: 1565064470.4238843918, “followup_action”: null, “paused”: false, “events”: [{ “event”: “action”, “timestamp”: 1565064305.7650516033, “name”: “action_listen”, “policy”: null, “confidence”: null }, { “event”: “user”, “timestamp”: 1565064305.811917305, “text”: “hi”, “parse_data”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “input_channel”: null }, { “event”: “action”, “timestamp”: 1565064408.6396431923, “name”: “utter_greet”, “policy”: null, “confidence”: null }, { “event”: “bot”, “timestamp”: 1565064408.6396431923, “text”: “Hey! How are you?”, “data”: { “elements”: null, “quick_replies”: null, “buttons”: null, “attachment”: null, “image”: null, “custom”: null }, “metadata”: {} }, { “event”: “user”, “timestamp”: 1565064470.4238843918, “text”: “very good”, “parse_data”: { “intent”: { “name”: “mood_great”, “confidence”: 0.9697415233 }, “entities”: [], “intent_ranking”: [{ “name”: “mood_great”, “confidence”: 0.9697415233 }, { “name”: “mood_unhappy”, “confidence”: 0.1109128371 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “affirm”, “confidence”: 0.0 }, { “name”: “greet”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “very good” }, “input_channel”: null }], “latest_input_channel”: null, “active_form”: {}, “latest_action_name”: “utter_greet” }
Curl:
curl -X POST -i http://localhost:5005/conversations/user/predict
Response:
{ "scores": [{ ** “action”: “action_listen”,** ** “score”: 0.3164482713** ** }, {** ** “action”: “utter_happy”,** ** “score”: 0.1764522195** ** }**, { “action”: “utter_greet”, “score”: 0.1322206855 }, { “action”: “utter_goodbye”, “score”: 0.1122527719 }, { “action”: “utter_did_that_help”, “score”: 0.1109326184 }, { “action”: “utter_cheer_up”, “score”: 0.0957360938 }, { “action”: “action_deactivate_form”, “score”: 0.0115207061 }, { “action”: “action_revert_fallback_events”, “score”: 0.0101715438 }, { “action”: “action_default_ask_rephrase”, “score”: 0.0079753846 }, { “action”: “action_default_ask_affirmation”, “score”: 0.0078029218 }, { “action”: “action_back”, “score”: 0.0066379979 }, { “action”: “action_restart”, “score”: 0.0063005914 }, { “action”: “action_default_fallback”, “score”: 0.0055482313 }], “policy”: “policy_1_KerasPolicy”, “confidence”: 0.3164482713, “tracker”: { “sender_id”: “user”, “slots”: {}, “latest_message”: { “intent”: { “name”: “mood_great”, “confidence”: 0.9697415233 }, “entities”: [], “intent_ranking”: [{ “name”: “mood_great”, “confidence”: 0.9697415233 }, { “name”: “mood_unhappy”, “confidence”: 0.1109128371 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “affirm”, “confidence”: 0.0 }, { “name”: “greet”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “very good” }, “latest_event_time”: 1565064470.4238843918, “followup_action”: null, “paused”: false, “events”: [{ “event”: “action”, “timestamp”: 1565064305.7650516033, “name”: “action_listen”, “policy”: null, “confidence”: null }, { “event”: “user”, “timestamp”: 1565064305.811917305, “text”: “hi”, “parse_data”: { “intent”: { “name”: “greet”, “confidence”: 0.973082304 }, “entities”: [], “intent_ranking”: [{ “name”: “greet”, “confidence”: 0.973082304 }, { “name”: “affirm”, “confidence”: 0.0679882169 }, { “name”: “mood_great”, “confidence”: 0.0 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “mood_unhappy”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “hi” }, “input_channel”: null }, { “event”: “action”, “timestamp”: 1565064408.6396431923, “name”: “utter_greet”, “policy”: null, “confidence”: null }, { “event”: “bot”, “timestamp”: 1565064408.6396431923, “text”: “Hey! How are you?”, “data”: { “elements”: null, “quick_replies”: null, “buttons”: null, “attachment”: null, “image”: null, “custom”: null }, “metadata”: {} }, { “event”: “user”, “timestamp”: 1565064470.4238843918, “text”: “very good”, “parse_data”: { “intent”: { “name”: “mood_great”, “confidence”: 0.9697415233 }, “entities”: [], “intent_ranking”: [{ “name”: “mood_great”, “confidence”: 0.9697415233 }, { “name”: “mood_unhappy”, “confidence”: 0.1109128371 }, { “name”: “deny”, “confidence”: 0.0 }, { “name”: “affirm”, “confidence”: 0.0 }, { “name”: “greet”, “confidence”: 0.0 }, { “name”: “goodbye”, “confidence”: 0.0 }], “text”: “very good” }, “input_channel”: null }], “latest_input_channel”: null, “active_form”: {}, “latest_action_name”: “utter_greet” } }
As you can see, from the score list of the last command, the action that rasa has predicted is “action_listen” and not the “utter_happy” although, in step 4, the exported intent was “mood_great”.
So, am i doing something wrong using the HTTP Api?
@c.kalleas did you get the resolution for this?
Hello. Unfortunately, no. In any case, what I want to point out is the different behavior of HTTP Api compared to RestInput Channel (in terms of prediction).