Error when running rasa test even when intent is correct

Rasa Open Source
1

I noticed that when running test on some test stories, if the intent involves entity extraction, then even if the predicted intent is actually correct, it will have errors.

For example, The correct intent in test story is as follows:

  • intent: choose_num_reps

    user: |-

    2

    entities:

    • reps: ‘2’

The error msg is:

  • intent: choose_num_reps # predicted: choose_num_reps: 1

As you can see the prediction is actually correct but it’s throwing errors.

2

Hi @YZZ

In the test story you must annotate the entity in the user text. e.g.

user: |
  [2](reps)

See the examples here: Testing Your Assistant

3

So this way of entity annotation [Madrid]{ "entity" : "location"} will cause an error then? Because I have the same issue, when annotating this way. Should it always be [Madrid](location) ?

What if you are working with roles/groups when it comes to entities, for example: {"entity":"number","role":"people"} or {"entity":"number","role":"nights"} ?

Many thanks!

4

Hi @losimons

You should be able to use the json format like in your example. What issue are you running in to?

5

Hi @jjuzl thank you for you reply. I’m using Rasa X to generate the test stories and in the auto generated test stories there is no entity annotated. I guess this is something from the Rasa X side that need to be fixed?

6

Check that the entity is actually being extracted. It should use the format [Madrid](location) in the test stories.

7

Hi @jjuzl I can see that the entity is actually extracted successfully, it’s just not shown in the generated test stories