Evaluating model with rasa X deployed at docker

[Deprecated] Rasa X Community Edition
1

Hello.

I’m using

rasa X - 0.31.3

rasa - 1.10.8

rasa_sdk - 1.10.2

Now as we developed our bot and all infrastructure around it, we have to setup automatic testing. Basically my goal is to get the bot to predict each NLU example correctly with at least 0.8 probability.

I know that there are command that you can run from CLI to run cross fold validation for NLU data. However deployment of docker doesn’t expose NLU data directly, instead having it in the database. Therefore I’m not able to run automated testing at my server.

I know that expected solution is to deploy new models to the repo and then running automatic test, however I would like to know if there are other solutions, since I don’t want to push changes every time I change something in the bot.

Also I would like to know what is the best practices for my case, i.e. getting NLU data to be predicted correctly with 0.8 probability. I’m fine with overfitting till some point, so if there are configurations for number of epochs, I might consider using them.

2

Rasa X is designed to be used with git as part of a git workflow.

The Rasa X HTTP API can be used to export your training data.

There’s a good blog post on testing here. We have example CI/CD pipelines which automate the testing. A good one to review is our Sara rasa-demo repo.

Greg