Trying to validate data with python script

Getting Started with Rasa
1

Hello,

I was reading the documentation on how to prevent my model from regressing using “rasa data validate”.
I saw that it was possible to do the same thing, but via a python script.
Here is the example provided:

import logging
from rasa import utils
from rasa.core.validator import Validator

logger = logging.getLogger(__name__)

utils.configure_colored_logging('DEBUG')

validator = Validator.from_files(domain_file='domain.yml',
                             nlu_data='data/nlu_data.md',
                             stories='data/stories.md')

validator.verify_all()

Except it gives me this error message:

from rasa.core.validator import Validator
ModuleNotFoundError: No module named 'rasa.core.validator'

I found an other Validator here :

from rasa.validator import Validator

It doesn’t have the methode : verify_all()

Rasa 1.10.3
Python 3.6.9

2

I’ve got some results with:

import logging
import asyncio
from rasa.validator import Validator
from rasa.importers.rasa import RasaFileImporter

file_importer = RasaFileImporter.load_from_config(
'config.yml', 'domain.yml', ['data/nlu.md', 'data/stories.md']
)

loop = asyncio.get_event_loop()
domain = loop.run_until_complete(file_importer.get_domain())
nlu_data = loop.run_until_complete(file_importer.get_nlu_data())
story_graph = loop.run_until_complete(file_importer.get_stories())

logfile = './nlu_model.log'
logging.basicConfig(filename=logfile, level=logging.DEBUG)
validator = Validator(domain, nlu_data, story_graph.story_steps)

validator.verify_intents()
validator.verify_utterances()
validator.verify_example_repetition_in_intents()
#validator.verify_story_structure(max_history=5)
#validator.verify_utterances_in_stories()
#validator.verify_nlu()
#validator.verify_intents_in_stories()

But the last four methods generate those errors.

validator.verify_story_structure(max_history=5)

Traceback (most recent call last):
  File "poc.py", line 23, in <module>
    validator.verify_story_structure(max_history=5)
  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/validator.py", line 223, in verify_story_structure
    augmentation_factor=0,
  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/core/training/generator.py", line 167, in __init__
    self.story_graph = story_graph.with_cycles_removed()
AttributeError: 'list' object has no attribute 'with_cycles_removed'

validator.verify_utterances_in_stories()

Traceback (most recent call last):
  File "poc.py", line 22, in <module>
    validator.verify_utterances_in_stories()
  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/validator.py", line 173, in verify_utterances_in_stories
    for story in self.story_graph.story_steps:
AttributeError: 'list' object has no attribute 'story_steps'

validator.verify_nlu()

  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/validator.py", line 243, in verify_nlu
    intents_are_valid = self.verify_intents_in_stories(ignore_warnings)
  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/validator.py", line 103, in verify_intents_in_stories
    for story in self.story_graph.story_steps
AttributeError: 'list' object has no attribute 'story_steps'

validator.verify_intents_in_stories()

Traceback (most recent call last):
  File "poc.py", line 25, in <module>
    validator.verify_intents_in_stories()
  File "/home/joseph/.venv/.py3env/lib/python3.6/site-packages/rasa/validator.py", line 103, in verify_intents_in_stories
    for story in self.story_graph.story_steps
AttributeError: 'list' object has no attribute 'story_steps'
3

This seems wrong. I think it’s supposed to be

validator = Validator(domain, nlu_data, story_graph)
4

Thanks, it works well.
The current documentation mentions verify_all().
Is this still on ?

5

Thanks for the hint, I created an issue for it: Docs for data validation mention not existing methods · Issue #6070 · RasaHQ/rasa · GitHub

6

but this is validating it without file or with empty file

7

@kulveersingh What do you mean?