Automatic testing against REST API using stories from tracker

Rasa Open Source
1

Hello

Our team is really in need of a testing solution that takes into consideration the response from the action server and database.

One solution we are exploring is to interact with the bot via the REST API, download the tracker story (in Markdown), and then use that tracker story as a test.

We would parse the Markdown file, execute all the user actions one after another (against the REST API), and then at the end compare the returned Markdown contents from the tracker against the file we originally had (which would be the source of truth).

However, we are encountering a problem:

However, in the dsl parser from the core the * form: (user action) lines are explicitly ignored:

in lib/python3.6/site-packages/rasa/core/training/dsl.py:

                elif re.match(r"^[*\-]\s+{}".format(FORM_PREFIX), line):
                    logger.debug(
                        "Skipping line {}, "
                        "because it was generated by "
                        "form action".format(line)
                    )

Is there a specific reason for this?

I experimented with removing the form: from the line and then having only e.g. * answer_time: [1 nov 2019](time:2019-11-01T00:00:00.000-04:00)

However, this breaks the parser, and the intent is now qualified as '1 nov 2019'.

2

I don’t have an answer to your issue (sorry), but I just started using Botium (https://botium.com) last week and that will allow you to build and run tests against your bot hitting the rest api. You can configure it to run on localhost (for development) or put in an actual production url.

Check it out, it’s pretty cool.

1 Like