How do I test my custom actions?

Rasa Open Source
1

Hey there!

I’m a bit lost here.

I’m building an assistant that will rely heavily on functions inside my action server. So far, I have four custom actions. Ideally, I’d like to ensure that these custom actions always behave the way I intend for them to behave, especially as I start to develop the assistant further.

I wrote a list of things I’d like to assert and wrote a mock domain and tracker. So far, the tests only assert that the action names are correct, as shown below:

action_hello_world = ActionHelloWorld()

assert(action_hello_world.name() == 'action_hello_world')

I don’t quite know how to execute the run function of each of these classes and assert that the returned events and response that will be sent to the user (through dispatcher.utter_message) are correct.

I tried passing my mock tracker and domain into the function to assert that my events list will return empty , but the code raises a TypeError when it reaches the line which calls the dispatcher as shown below:

Code:

action_hello_world = ActionHelloWorld()
assert(action_hello_world.run(tracker=tracker, domain=domain, dispatcher=CollectingDispatcher)==[])

Error:

 File "/app/actions/actions.py", line 48, in run
 text=f"Hello world!") 
 TypeError: utter_message() missing 1 required positional argument: 'self'

It’s obvious that I’m doing something wrong here. What is the best way to test custom actions on the action server without running into these kind of errors with the SDK?

Thanks in advance!

2

@snek, it’s simple. On your domain, you have to declare your action’s name on action section:

actions:
  -  action_hello_world

On your stories the same:

- story: testing custom action
  - intent: greet
  - action: action_hello_world

On your actions.py:

class ActionHelloWorld(Action):
    def name(self) -> Text:
        return "action_hello_world"

    def run (self,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any], 
    ) -> List[Dict]:

        dispatcher.utter_message("Hello from custom actions :D")

Retrain your model and test :smiley:

3

@marcos.allysson Thanks for the reply, but I think you missed the point. Could be my fault since I may not have been super clear. My actions work, I have a functional assistant. I just have lots of logic going on inside class ActionHelloWorld(Action) such as communicating with my database and fetching things, and it returns different responses based on what response it gets back from the post request.

What I am trying to do here is not testing the conversation logic of Rasa (since I already did that and all tests run successfully), but to create unit tests for my action server to ensure that all functions in my endpoint behave as needed even with corner cases. :slight_smile:

2 Likes
4

We are working on adding some custom action unit test examples into the financial demo bot. It’s a work in progress but you’ll find it in the db branch. Here are the files to look at:

Greg

2 Likes
5

Hey Greg,

That was super helpful!

6

can i use mock object

7

I’m using requests-mock · PyPI and it’s working like a charm.

8

Greg, how can we export the tracker to use it in custom actions test?

Thanks in advance!

9

Try the rasa export command.

1 Like
10

Hi @stephens, is there a differente way get the Tracker data (the conversation, JSON, etc.) other than using “rasa export”?

My Tracker is “in memory”, and “rasa export” doesn’t seem to work for this situation.

I would like to create tracker examples to use in my tests, as you have done in https://github.com/RasaHQ/financial-demo/tree/main/tests/data

How did you create those JSON files? Is there a way to do that without using “rasa export”?

11

You can also setup RabbitMQ as an event broker and read the events from Rabbit.