Is there a way to compare multiple pipelines that require different Docker images?

Apologies in advance, if I’m not getting something really basic right. :slight_smile:

I have a file to install the Docker image - docker pull rasa/rasa:1.10.0-full, but I was hoping to compare the performance of the default recommended pipeline for an English dataset, with the performance of another pipeline which needs another Docker image like rasa:1.10.0-spacy-en or rasa:1.10.0-mitie-en.

How do I go about it? Thanks in advance!

To compare the performance of two different bots, you can use the rasa test command (see docs). The command requires a trained model, some data and the config file. It will write all results by default to a folder results.

So you can create multiple docker files that inherit from the desired rasa image and execute the rasa test command. Once this is done you can compare the results in the different results folders.

Does that help?

Hello @Tanja, yes it does. Thank you.