Num stories does not exist

(base) C:\Users\TizianoLabruna\Downloads\Rasa\test-bot>rasa test core --evaluate-model-directory
Traceback (most recent call last):
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\utils\", line 136, in read_file
    with open(filename, encoding=encoding) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'models\\20191128-113632.tar.gz\\num_stories.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\tizianolabruna\anaconda3\lib\", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\tizianolabruna\anaconda3\lib\", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\TizianoLabruna\Anaconda3\Scripts\rasa.exe\", line 9, in <module>
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\", line 76, in main
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\cli\", line 81, in test_core
    test_core_models_in_directory(args.model, stories, output)
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\", line 26, in test_core_models_in_directory
    number_of_stories = io_utils.read_json_file(story_n_path)
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\utils\", line 144, in read_json_file
    content = read_file(filename)
  File "C:\Users\TizianoLabruna\AppData\Roaming\Python\Python37\site-packages\rasa\utils\", line 139, in read_file
    raise ValueError(f"File '{filename}' does not exist.")
ValueError: File 'models\20191128-113632.tar.gz\num_stories.json' does not exist.

Hi, anybody knows how to solve this issue? I’m trying to evaluate two different config files.

Thank you, Tiziano

What’s the content of your models folder?

Hi @Tobias_Wochinger, thank you for replying.

Here a screenshot of my models folder:

You trained the multiple models with rasa train core --config <config-1> <config-2>, didn’t you?

Yes I did

can you please try moving the run folders to a different directory (so that there are no .tar.gz in there) and try again?

I tried both creating a new folder inside models and moving the 3 run folders there, and moving that folder outside models (in the main directory). But still getting the same error…

Ok, i tried to reproduce you error, but couldn’t. I did

rasa train core -c config.yml config2.yml --out comparison_models
rasa test core --evaluate-model-directory -m comparison_models

Does that work for you? If yes, it would be great if you could give me some hints how to improve the CLI documentation / docs to make this easier understandable.

For reproducing the error the commands to run are:

rasa train core --config config.yml config2.yml
rasa test core --evaluate-model-directory

So yeah, the relevant difference with your commands is that I don’t specify a folder for the models. Specifying it, it works correctly.

Personally, it doesn’t seem very clear to me from the documentation, that you need to write -m followed by the directory of the models; I thought that it automatically recognised the folder models, if not specified.

Thank you, Tiziano


thanks for your feedback! I did some further digging, and that’s definitely some bug in the code. I created an issue for it since I don’t have time to work on it right now. Feel free to tackle it yourself - I’d be happy to help you doing the PR. "rasa test core --evaluate-model-directory" should have directory as default · Issue #4896 · RasaHQ/rasa · GitHub