HTTP API not predicting correctly

Rasa Open Source
1

Hi there.

I’m attempting to write a test script to make sure that I don’t break the expected dialog flow during development. My script uses the HTTP API and is called test.py. It reads “stories” from a YML file that describe the expected actions for each human message.

I’m currently using Rasa 1.0.6 and starting the server with:

rasa run -m models --enable-api --cors

My issue is that the predicted actions returned by the HTTP API don’t match the actions that I get with using “rasa shell” or “rasa interactive” or even the web socket connector through a browser. They all perform as expected.

Here is my test script: test.py

And here is the input file with one story: test.yml

My script outputs as follows:

Starting...
------------------------------------------------------------------------

Story to test: Happy flow, saying yes to more

    Message: /welcome
    TEST OK
    Actions: ['action_clear_topic', 'utter_welcome', 'utter_name', 'utter_agent_purpose', 'utter_ask_problem']

    Message: I am sad
    TEST OK
    Actions: ['action_check_topic', 'utter_confirm_topic']

    Message: yes
    TEST FAILED
    Expected actions: ['utter_ask_type_of_help']
    Actual actions: ['action_clear_topic', 'utter_ask_problem']

    Message: talk to somebody
    TEST FAILED
    Expected actions: ['action_check_type_of_help', 'action_lookup_resource', 'utter_resource_found', 'utter_ask_another']
    Actual actions: ['action_check_type_of_help', 'utter_state_problem_first']

    Message: yes
    TEST OK
    Actions: ['action_clear_topic', 'utter_ask_problem']

Finished

So why is my script (test.py) getting results from the HTTP API that do not match the web socket connector or other ways of accessing the model such as the shell? It’s almost as if it’s using the wrong model.

Thanks - I just can’t see what’s wrong with the way I access the HTTP API as it was working for me before. Am I using it correctly? Also, should I use the Python API instead? If so, how?

2

Bump! Can anyone help with this please?

3

Hey @andymayer, we refer to the API you’re using as the HTTP API. I’ll use this name as to avoid confusion when linking to the docs.

Have you tried using the rest input channel? Does that yield the same results as when using rasa shell?

4

I’ve changed the thread title to “HTTP API”. Sorry for the confusion.

Anyway, I have found the solution! It turns out that I was using the HTTP API correctly to make predictions and the problem was that line 123 of test.py that predicts the next actions should be indented as part of the IF statement (duh! I’m new to python):

        if type(test_item) is not str and test_item.get('message'):

           message = test_item.get('message')[0]
           print("\n    Message: "+message)

           # Predict next actions
           actual_actions = get_actions(conversation_id, message)

My test script works perfectly now :slight_smile: :slight_smile:

Here is the output:

Starting...   
------------------------------------------------------------------------

Story to test: Happy flow, saying yes to more

Message: /welcome
TEST OK
Actions: ['action_clear_topic', 'utter_welcome', 'utter_name', 'utter_agent_purpose', 'utter_ask_problem']

Message: I am sad
TEST OK
Actions: ['action_check_topic', 'utter_confirm_topic']

Message: yes
TEST OK
Actions: ['utter_ask_type_of_help']

Message: talk to somebody
TEST OK
Actions: ['action_check_type_of_help', 'action_lookup_resource', 'utter_resource_found', 'utter_ask_another']

Finished
5

By the way, I will be developing this concept further as a way of writing scripted tests for Behaviour-Driven Development (BDD) of Rasa chatbots.

https://cucumber.io/docs/bdd/

Is there much interest in this for Rasa? Is anybody else doing this? Are there better ways than my script?Do get in touch with any thoughts, ideas or suggestions.