Hello everyone!
I am running the stories tests (rasa test) that I have written and despite the fact that the output shows me that everything has passed OK (or so I think), I see that the TEDPolicy_errors.json file shows two texts that do not predict entity, as an example, this would be one of them (the others are related to this same user input)
If the language is a problem, I have no problem changing it to English, I put it like this (Spanish) since for now the development is in Spanish, but I plan to change it in the future.
{
"text": "quiero hacer reserva para 4 comensales",
"entities": [
{
"start": 26,
"end": 27,
"value": "4",
"entity": "comensales"
}
],
"predicted_entities": []
}
The story test is:
- story: Reserva mesa deny path 1
steps:
- user: |
quiero hacer reserva para [4](comensales) comensales
intent: hacer_reserva_comensales
entities:
- comensales: "4"
- slot_was_set:
- comensales: "4"
- action: ValidateComensales
- slot_was_set:
- comensales: "4"
- slot_was_set:
- mesa_disponible: true
- action: utter_ask_reservar_mesa
- user: |
No
intent: deny
- action: ReservarMesa
- slot_was_set:
- comensales: null
- slot_was_set:
- mesa_reservada: null
- slot_was_set:
- mesa_disponible: null
The intent (nlu.yml file): (I know I can put this a bit better but right now I’m just looking for basic functionality)
- intent: hacer_reserva_comensales
examples: |
- quiero hacer reserva para [1](comensales) persona
- quiero hacer reserva para [2](comensales) personas
- quiero hacer reserva para [3](comensales) personas
- quiero hacer reserva para [4](comensales) personas
- quiero hacer reserva para [5](comensales) personas
- reserva para [1](comensales) persona
- reserva para [2](comensales) personas
- reserva para [3](comensales) personas
- reserva para [4](comensales) personas
- reserva para [5](comensales) personas
- reserva mesa [1](comensales) persona
- reserva mesa [2](comensales) personas
- reserva mesa [3](comensales) personas
- reserva mesa [4](comensales) personas
- reserva mesa [5](comensales) personas
The domain.yml file:
entities:
- comensales
...
slots:
comensales:
type: text
influence_conversation: true
mappings:
- type: from_entity
entity: comensales
The story test passes, but I don’t quite understand the problem, or at least, what I would understand is that the value “4” is not being recognized with the entity “comensales” even though the story itself is functional (both rasa shell and rasa interactive show correct values for this slot/entity)
2022-07-08 13:43:21 INFO rasa.core.test - Finished collecting predictions.
2022-07-08 13:43:21 INFO rasa.core.test - Evaluation Results on CONVERSATION level:
2022-07-08 13:43:21 INFO rasa.core.test - Correct: 7 / 7
2022-07-08 13:43:21 INFO rasa.core.test - Accuracy: 1.000
...
2022-07-08 13:43:22 INFO rasa.core.test - Evaluation Results on ACTION level:
2022-07-08 13:43:22 INFO rasa.core.test - Correct: 29 / 29
2022-07-08 13:43:22 INFO rasa.core.test - F1-Score: 1.000
2022-07-08 13:43:22 INFO rasa.core.test - Precision: 1.000
2022-07-08 13:43:22 INFO rasa.core.test - Accuracy: 1.000
2022-07-08 13:43:22 INFO rasa.core.test - In-data fraction: 0
- DIETClassifier_confusion_matrix.png has only the diagonal line
- DIETClassifier_histogram.png has no wrong bars
- DIETClassifier_report.json has not “confused_with” match and all “precision” values are 1.0
- failed_test_stories.yml is empty (all good)
- intent_confusion_matrix.png has only the diagonal line
- intent_histogram.png has no wrong bars
- intent_report.json has not “confused_with” match and all “precision” values are 1.0
- stories_with_warnings.yml is empty (all good)
- story_confusion_matrix.png has only the diagonal line
- story_report.json has all “precision” values to 1.0
And then, the error files:
TEDPolicy_errors.json (454 Bytes) TEDPolicy_report.json (469 Bytes)
Here comes the question: am I writing the user input correctly in the test? Looking at the documentation, I see these two input examples (adapted to my entity):
- Testing Your Assistant
- Example 1:
- [4]{“entity”: “comensales”} (same result as 4
- Example 2:
- [4][{\“entity\”: \“comensales\”}] (this directly gives the following error running the command)
InvalidEntityFormatException: Incorrect training data format (‘{\“entity\”: \“comensales\”}’). More info at Training Data Format line 1 column 2 (char 1)
I can’t put a link to the project on GitHub because until I defend the development it has to be private (university stuff), but if you require something specific I could share it
Regards, Raúl.