I noticed that when running test on some test stories, if the intent involves entity extraction, then even if the predicted intent is actually correct, it will have errors.
For example, The correct intent in test story is as follows:
- reps: ‘2’
The error msg is:
- intent: choose_num_reps # predicted: choose_num_reps: 1
As you can see the prediction is actually correct but it’s throwing errors.