Precision-recall curve in rasa

Rasa Open Source
1

Is it possible to implement a precision recall curve in rasa? Thanks

2

@revet Hi, I guess rasa have inbuilt one, did you visited this link: Testing Your Assistant Or you want something else, please elaborate more.

3

@nik202 Hi, thanks! I want something like the attached picture

Screen Shot 2021-09-12 at 15.45.18
Screen Shot 2021-09-12 at 15.45.18718×556 34.4 KB

4

I suggest you use Tensorboard to make comparisons and choose an optimal configuration. This is doable on DIET, ResponseSelector, and TED like so for example:

- name: DIETClassifier
  // other parameters
  evaluate_on_number_of_examples: 200
  evaluate_every_number_of_epochs: 5
  tensorboard_log_directory: ./tensorboard/DIET
  tensorboard_log_level: epoch

Try to set evaluate_on_number_of_examples to about 20% of your total number of examples (of course, this means these examples will not be used for training and you will have to give a bit more examples). You can use this script I wrote to count the number of examples you have.

I would also recommend setting a random_seed: If you want to accurately compare two Pipeline Components or Policies across multiple trainings, you could set a Seed for DIET, ResponseSelector, and TED like so for example:

- name: DIETClassifier
  random_seed: 1
  // other parameters
1 Like