Rasa test not showing results

tiziano · February 12, 2020, 4:41pm

Hi,

I followed this guide for evaluating my model.

I first split my data into training and test set, then evaluated some config files for finding the best one. When I found I wanted to test it on the test set, so I first trained a model, and then tested it using this command:

rasa test nlu -u train_test_split/test_data.md --model models/nlu-convert_default-20200212-153255.tar.gz -v

The test went fine and created the confmat and hist files, but the problem is, I need to know the values of the performance of the model (f1-score, accuracy, loss…), and I can’t find them either in the command line nor in the folder.

How can I solve it?

Thank you, Tiziano

tiziano · February 13, 2020, 9:37am

Anyone please?

Tanja · February 18, 2020, 12:09pm

The rasa test nlu command should create a results folder with the following files:

confmat.png
hist.png
intent_report.json

and optionally also a CRFEntityExtractor_report.json if you have the component in your pipeline. The *_report files contain the f1-scores, accuracy, etc. How does your result folder look like? What version of Rasa are you using?

tiziano · February 18, 2020, 12:15pm

Yes I see these files, but as I said in no one of them it is reported the values of, for example, f1-score and loss. I would like to have some more results… The same for the Core testing. I’m only getting the plot and the results.json (which is the data used in the plot), but nothing else such as the errors that the model did, or so…

Tanja · February 18, 2020, 12:20pm

Those files should contain something like this

  "mood_great": {
    "precision": 1.0,
    "recall": 1.0,
    "f1-score": 1.0,
    "support": 8,
    "confused_with": {}
  },

So you should see an entry per intent for the intent_report.json and some overall metrics at the bottom of the file. Errors should be reported as well per default. E.g. you should see a file called intent_errors.json. If you want to see successes of the model you can switch that on via --successes.

What else do you want to see?

tiziano · February 18, 2020, 12:32pm

I do see the intent errors, but it doesn’t happen the same for the stories.

This are all the results I get:

I would like to see for which stories it went wrong…

tiziano · February 19, 2020, 11:11am

Anybody?

Tanja · February 19, 2020, 12:11pm

When executing rasa test or rasa test core you should see a file called failed_stories.md in the results folder. It should contain the incorrect predicted stories.

tiziano · February 19, 2020, 12:32pm

I agree that I should, but as I showed in the previous message, this is not what happens…

Tanja · February 21, 2020, 8:27am

Can you execute rasa test --debug and paste the log output here? Thanks.

tiziano · February 24, 2020, 10:15am

It’s hundreds and hundreds of lines, it doesn’t even fit in my command line history… how can I paste all the log here?

Below are the last few lines of the log:

2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=cmss10:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 0.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=cmss10:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to cmss10 ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/cmss10.ttf') with score of 0.050000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=cmex10:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 0.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=cmex10:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to cmex10 ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/cmex10.ttf') with score of 0.050000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 0.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 0.33499999999999996
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 1.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 1.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to DejaVu Sans ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf') with score of 0.050000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=italic:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 1.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 10.434999999999999
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 10.15
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 1.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 0.15000000000000002
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 0.43499999999999994
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=italic:variant=normal:weight=normal:stretch=normal:size=10.0 to DejaVu Sans ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans-Oblique.ttf') with score of 0.150000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=normal:variant=normal:weight=bold:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 0.33499999999999996
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 11.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 0.0
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans:style=normal:variant=normal:weight=bold:stretch=normal:size=10.0 to DejaVu Sans ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans-Bold.ttf') with score of 0.000000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans Mono:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 0.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 1.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 0.33499999999999996
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 1.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans Mono:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to DejaVu Sans Mono ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSansMono.ttf') with score of 0.050000.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans Display:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0.
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-BoldItalic.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Display' (DejaVuSansDisplay.ttf) normal normal 400 normal>) = 0.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmmi10' (cmmi10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmss10' (cmss10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneral.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif Display' (DejaVuSerifDisplay.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmb10' (cmb10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeOneSym' (STIXSizOneSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmr10' (cmr10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans Mono' (DejaVuSansMono-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmex10' (cmex10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFourSym' (STIXSizFourSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeTwoSym' (STIXSizTwoSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralBolIta.ttf) italic normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Italic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeFiveSym' (STIXSizFiveSymReg.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmsy10' (cmsy10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-Oblique.ttf) oblique normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Sans' (DejaVuSans-BoldOblique.ttf) oblique normal bold normal>) = 11.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUniIta.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXSizeThreeSym' (STIXSizThreeSymBol.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'DejaVu Serif' (DejaVuSerif-Bold.ttf) normal normal bold normal>) = 10.335
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'cmtt10' (cmtt10.ttf) normal normal 400 normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXGeneral' (STIXGeneralItalic.ttf) italic normal 400 normal>) = 11.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: score(<Font 'STIXNonUnicode' (STIXNonUni.ttf) normal normal regular normal>) = 10.05
2020-02-24 10:08:49 DEBUG    matplotlib.font_manager  - findfont: Matching :family=DejaVu Sans Display:style=normal:variant=normal:weight=normal:stretch=normal:size=10.0 to DejaVu Sans Display ('/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSansDisplay.ttf') with score of 0.050000.
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:49 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:08:49 DEBUG    matplotlib.backends.backend_pdf  - Embedding font /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf.
2020-02-24 10:08:49 DEBUG    matplotlib.backends.backend_pdf  - Writing TrueType font.
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:54 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:08:58 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:02 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:09:02 DEBUG    matplotlib.backends.backend_pdf  - Assigning font /b'F1' = '/usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf'
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 652.0
2020-02-24 10:09:08 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03,
       2.e+04, 3.e+04, 4.e+04, 5.e+04, 6.e+04, 7.e+04, 8.e+04, 9.e+04])
2020-02-24 10:09:08 DEBUG    matplotlib.backends.backend_pdf  - Embedding font /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans.ttf.
2020-02-24 10:09:08 DEBUG    matplotlib.backends.backend_pdf  - Writing TrueType font.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/dialogue/stories.md' is 'unk'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/dialogue/stories_basic.md' is 'unk'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/dialogue/stories_extra.md' is 'unk'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/dialogue/stories_test.md' is 'unk'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/dialogue/stories_training.md' is 'unk'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/nlu.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.acquaintance.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.age.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.annoying.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.answer_my_question.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.bad.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.be_clever.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.beautiful.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.birth_date.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.boring.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.boss.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.busy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.can_you_help.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.chatbot.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.clever.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.crazy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.fired.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.funny.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.good.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.happy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.hobby.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.hungry.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.marry_user.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.my_friend.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.name.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.occupation.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.origin.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.ready.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.real.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.residence.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.right.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.sure.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.talk_to_me.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.there.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/agent.what_can_do.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.bad.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.good.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.no_problem.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.thank_you.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.welcome.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/appraisal.well_done.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/confirmation.cancel.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/confirmation.no.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/confirmation.yes.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.hold_on.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.hug.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.i_do_not_care.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.sorry.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.what_do_you_mean.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/dialog.wrong.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/emotions.ha_ha.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/emotions.wow.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.bye.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.goodevening.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.goodmorning.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.goodnight.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.hello.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.how_are_you.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.nice_to_meet_you.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.nice_to_see_you.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.nice_to_talk_to_you.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/greetings.whatsup.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.angry.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.back.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.bored.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.busy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.can_not_sleep.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.does_not_want_to_talk.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.excited.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.going_to_bed.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.good.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.happy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.has_birthday.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.here.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.joking.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.likes_agent.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.lonely.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.looks_like.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.loves_agent.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.misses_agent.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.needs_advice.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.sad.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.sleepy.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.testing_agent.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.tired.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.waits.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.wants_to_see_agent_again.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.wants_to_talk.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu/smalltalk/user.will_be_back.md' is 'md'.
2020-02-24 10:09:09 DEBUG    rasa.model  - Extracted model to '/tmp/tmpucmn1ob4'.
2020-02-24 10:09:09 INFO     absl  - Using /tmp/tfhub_modules to cache modules.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/963a7a83964f4719a4b9ca735dfa0690_agent.acquaintance.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/1442bfe71c54467cb04cfb10078cf3a4_agent.age.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/d16830a2da09450186c2864fa8c195f0_agent.annoying.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/741c320392dd4bf1ab082226e4bf28c5_agent.answer_my_question.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/7b9b758329554fcab81d2123df6ac45f_agent.bad.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/e6089ddd6b634f4da054b09dbe22c9a0_agent.be_clever.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/a6366bff828b417488c0d35bd97005da_agent.beautiful.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/0777936f25ce43ffb261da43085b0dff_agent.birth_date.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/9673ecb3e83b4bd09fd79a21ca8ebeff_agent.boring.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/03106926e4bc47228888c4576c0d04b1_agent.boss.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/6f66406652ae4820bcfce66a63459f35_agent.busy.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/048ba04552a14b738be0069d46a12c35_agent.can_you_help.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/80a8b3c160724b96bc6e66bc1c2a6607_agent.chatbot.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/747d596fc992419da53d39e89a5ba44f_agent.clever.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/395ff84008264088af05faa7879334b7_agent.crazy.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/4517b3d28265476f8ef9390bc21f4036_agent.fired.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/7b3c61ecb660416980bfb5eb30028b26_agent.funny.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/80fb9cc62d094c3694fe3eb5af4406b9_agent.good.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/96033c1c4cbb473f8c2adf669ab87a32_agent.happy.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/9c05f139c60148d998ccdfb8e8825696_agent.hobby.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/555fc9f8c8cf4227a9c3065f9b8105b6_agent.hungry.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/401449c02d2a4140930977a948798a5b_agent.marry_user.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/19291c19435e49eaa910f0f82f65d69d_agent.my_friend.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/fe6f95405b4e4f76aeb69053e058645d_agent.name.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/05f75923dd2f4bf2a36a456f49bdc1cf_agent.occupation.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/a68e561ad1eb42358c3db82d418ce222_agent.origin.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/1dec57b672b94d9e9198a1fc4a874ad8_agent.ready.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/528a44acbb0c4edc88df92a7605057d7_agent.real.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/13fd78c6981f40199db1340f0135e47d_agent.residence.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/5d69e4410b4b46e7b19934d9af1f4a83_agent.right.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/8040e627fb8e44f1a231d4ed0ef5dd25_agent.sure.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/11f4513a4ee240cd90db2505b78ef3fe_agent.talk_to_me.md' is 'md'.
2020-02-24 10:09:17 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/bb575c6fcecd487b91258f558e1e8d23_agent.there.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/c984eef37e4d42449e491f309a74483e_agent.what_can_do.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/bc10854fe292499fae47b75ba8e10c0d_appraisal.bad.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/a953bc0c7c0544db950ae42447754993_appraisal.good.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/506b822765e54f51a0c0c2c5f18c5ddc_appraisal.no_problem.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/1827fdde628448ad960019724be5ae6b_appraisal.thank_you.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/89836f76ef43459292a11bb4a99a4678_appraisal.welcome.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/d99793e9b06746729e07500e9d8243b2_appraisal.well_done.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/2b6297cb39a1483090be372d10b1b9d4_confirmation.cancel.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/bc6ee961cd6d46009ac285f2e7f30773_confirmation.no.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/f6a24ff9ab904837b3a9414cc7efe3fd_confirmation.yes.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/81c58c16f050446389c835c088dea9cc_dialog.hold_on.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/32bb99f68b6544bb9189962451804359_dialog.hug.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/67a5717632a54648b7a9c04bfe54920f_dialog.i_do_not_care.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/d1239360c23a4b4f8f884cf72ae4e85c_dialog.sorry.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/4507fe31e2ca47458aefde94df0a4af1_dialog.what_do_you_mean.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/b2247b0919ad4c82993b95a2600a8c6b_dialog.wrong.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/ef6d9ade21874666b99824dbc58ae8ea_emotions.ha_ha.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/c010f69e51d94e819495602f4d0646d8_emotions.wow.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/1a13d2e4cc674f43ab5995b81e92ef2c_greetings.bye.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/91f0c39a766a4e7bb5f12d77fb196b17_greetings.goodevening.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/6d6814d4c1dc437ab09bc3c3e2c076ae_greetings.goodmorning.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/00435c6d7488412287ff70a84d9728bd_greetings.goodnight.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/513885ad7a8d49c0b621077332525ee1_greetings.hello.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/0c77bea8689a4492972a4c79f618bbdb_greetings.how_are_you.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/02b22ce5495f4b6aacdfd28e3682f1a4_greetings.nice_to_meet_you.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/dba4dfd3f81c40de8b6c4df42f1de58d_greetings.nice_to_see_you.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/eff5525128d349a1997755f01ae2a928_greetings.nice_to_talk_to_you.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/537d4d0c29854573a98d87f7ca399472_greetings.whatsup.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/9d9ae95206504394b266cdcaedca9d67_nlu.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/8f06fc3daa5745448fd226e6522d8c1e_user.angry.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/7e0a3eedcdc346c8abdd708a068ae3e7_user.back.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/663e80b0484d43be970c3e4c7dedb3f9_user.bored.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/21c39a43303d43dcb187427eabbd0394_user.busy.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/23609ddc99d44674a4dcfea301637301_user.can_not_sleep.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/df3fd4cec5ce427da10fc8939421b7eb_user.does_not_want_to_talk.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/0cdf631161e34261a1f8800c135d4baa_user.excited.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/8eb3ce7845b946898d7ab582b2aa014b_user.going_to_bed.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/83b70f9cc5eb4e2bb9080ea23d4ebe23_user.good.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/b966525d40a343518ac29c8c85986669_user.happy.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/23820ec18454462c978863954d00c689_user.has_birthday.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/5d7007e64fcf436a9c9b9894deed14a7_user.here.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/9da48924d3dd455b919258f03fce3ae1_user.joking.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/476c1275935f45e99aa9da9411b0a6d9_user.likes_agent.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/f8d0051b78404bd5b1587089876a0d3d_user.lonely.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/5bf2735082e94d38a9ffc1cc578df5d6_user.looks_like.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/aff0b4d3c2a34cff9d2cd5d512ca9d5a_user.loves_agent.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/1614f49559da470684476b2bed04912e_user.misses_agent.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/b9ee9684a9a24ccba39d01255bd8c31f_user.needs_advice.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/0385c388aaeb4d009c5499b1a1e1c277_user.sad.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/26949dc8c4224db18cfbe31bd01b3feb_user.sleepy.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/d73a615cb4524cc9804749d8c5c4e6bd_user.testing_agent.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/ebc13cba2ff840c8bc2bb4ef4ee1ef2e_user.tired.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/f19f41c26faa4cc5a1cc568c5464a0ea_user.waits.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/34acb7c27e0e4ea6ba931f683bb9bcd4_user.wants_to_see_agent_again.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/c2a79fd5491a4c73b7f70429f4fae537_user.wants_to_talk.md' is 'md'.
2020-02-24 10:09:18 DEBUG    rasa.nlu.training_data.loading  - Training data format of '/tmp/tmpxng80ty7/323955e4576e416584012f4aa99ae3d0_user.will_be_back.md' is 'md'.
2020-02-24 10:09:18 INFO     rasa.nlu.test  - Running model for predictions:
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2449/2449 [00:43<00:00, 56.62it/s]
2020-02-24 10:10:01 INFO     rasa.nlu.test  - Intent evaluation results:
2020-02-24 10:10:01 INFO     rasa.nlu.test  - Intent Evaluation: Only considering those 2449 examples that have a defined intent out of 2449 examples
2020-02-24 10:10:01 INFO     rasa.nlu.test  - Classification report saved to results/intent_report.json.
2020-02-24 10:10:01 INFO     rasa.nlu.test  - Incorrect intent predictions saved to results/intent_errors.json.
2020-02-24 10:10:01 DEBUG    rasa.nlu.test  -

These intent examples could not be classified correctly:
[{'text': 'what are you upto?', 'intent': 'agent.acquaintance', 'intent_prediction': {'name': 'greetings.whatsup', 'confidence': 0.5551027655601501}}, {'text': 'brilliant', 'intent': 'agent.clever', 'intent_prediction': {'name': 'appraisal.good', 'confidence': 0.4388422966003418}}, {'text': 'speak with me', 'intent': 'agent.talk_to_me', 'intent_prediction': {'name': 'user.wants_to_talk', 'confidence': 0.5329943895339966}}, {'text': 'what can you do for me?', 'intent': 'agent.what_can_do', 'intent_prediction': {'name': 'agent.acquaintance', 'confidence': 0.7208907604217529}}, {'text': 'good', 'intent': 'appraisal.good', 'intent_prediction': {'name': 'user.good', 'confidence': 0.49830812215805054}}, {'text': 'nice', 'intent': 'appraisal.good', 'intent_prediction': {'name': 'agent.good', 'confidence': 0.6217567324638367}}, {'text': 'amazing', 'intent': 'appraisal.good', 'intent_prediction': {'name': 'appraisal.well_done', 'confidence': 0.4023098647594452}}, {'text': 'marvelous', 'intent': 'appraisal.good', 'intent_prediction': {'name': 'appraisal.well_done', 'confidence': 0.50665682554245}}, {'text': 'splendid', 'intent': 'appraisal.good', 'intent_prediction': {'name': 'appraisal.well_done', 'confidence': 0.48590222001075745}}, {'text': 'so nice of you', 'intent': 'appraisal.thank_you', 'intent_prediction': {'name': 'appraisal.good', 'confidence': 0.5358421206474304}}, {'text': 'fantastic', 'intent': 'appraisal.well_done', 'intent_prediction': {'name': 'appraisal.good', 'confidence': 0.4823855757713318}}, {'text': 'brilliant', 'intent': 'appraisal.well_done', 'intent_prediction': {'name': 'appraisal.good', 'confidence': 0.4388422966003418}}, {'text': 'coool', 'intent': 'confirmation.yes', 'intent_prediction': {'name': 'agent.good', 'confidence': 0.7931346297264099}}, {'text': 'amazing', 'intent': 'emotions.wow', 'intent_prediction': {'name': 'appraisal.well_done', 'confidence': 0.4023098647594452}}, {'text': "I'm glad to see you", 'intent': 'greetings.nice_to_see_you', 'intent_prediction': {'name': 'user.happy', 'confidence': 0.5028986930847168}}, {'text': "It's boring", 'intent': 'user.bored', 'intent_prediction': {'name': 'agent.boring', 'confidence': 0.630631685256958}}, {'text': "It's good", 'intent': 'user.good', 'intent_prediction': {'name': 'appraisal.good', 'confidence': 0.5598580837249756}}, {'text': "I'm here", 'intent': 'user.here', 'intent_prediction': {'name': 'user.back', 'confidence': 0.6181103587150574}}]
2020-02-24 10:10:01 DEBUG    matplotlib.colorbar  - locator: <matplotlib.colorbar._ColorbarLogLocator object at 0x7fa618f60b00>
2020-02-24 10:10:01 DEBUG    matplotlib.colorbar  - Using auto colorbar locator on colorbar
2020-02-24 10:10:01 DEBUG    matplotlib.colorbar  - locator: <matplotlib.colorbar._ColorbarLogLocator object at 0x7fa618f60b00>
2020-02-24 10:10:01 DEBUG    matplotlib.colorbar  - Setting pcolormesh
2020-02-24 10:10:01 INFO     rasa.nlu.test  - Confusion matrix, without normalization:
[[22  0  0 ...  0  0  0]
 [ 0 21  0 ...  0  0  0]
 [ 0  0 20 ...  0  0  0]
 ...
 [ 0  0  0 ... 62  0  0]
 [ 0  0  0 ...  0  7  0]
 [ 0  0  0 ...  0  0 19]]
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:10 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:15 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:19 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:23 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([1.e-01, 1.e+00, 1.e+01, 1.e+02, 1.e+03])
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - vmin 1.0 vmax 88.0
2020-02-24 10:10:30 DEBUG    matplotlib.ticker  - ticklocs array([2.e-01, 3.e-01, 4.e-01, 5.e-01, 6.e-01, 7.e-01, 8.e-01, 9.e-01,
       2.e+00, 3.e+00, 4.e+00, 5.e+00, 6.e+00, 7.e+00, 8.e+00, 9.e+00,
       2.e+01, 3.e+01, 4.e+01, 5.e+01, 6.e+01, 7.e+01, 8.e+01, 9.e+01,
       2.e+02, 3.e+02, 4.e+02, 5.e+02, 6.e+02, 7.e+02, 8.e+02, 9.e+02,
       2.e+03, 3.e+03, 4.e+03, 5.e+03, 6.e+03, 7.e+03, 8.e+03, 9.e+03])

Tanja · February 25, 2020, 7:30am

Strange that it is showing so many debug messages from matplotlib. Can you maybe store your log output to a file an upload the file here? E.g. rasa test --debug > log.out 2>&1.

tiziano · February 25, 2020, 9:39am

Here you are log.out (1.6 MB)

Tanja · February 27, 2020, 9:15am

What Rasa version are you using? What is the exact command that you are executing? How does your directory structure look like? We only don’t write the the failed stories when we compare different models. Can you also try to run rasa test core --model <path-to-model.tar.gz>? Do you see any failed stories in that case?

tiziano · February 27, 2020, 10:06am

Ok, so here we have the solution!

I’m actually comparing different models and so, this is why I’m not getting the failed stories.

I’m doing this, because this is what is written in this guide. First it says to train different models for different config files, and then to compare the different models.

So, how can I compare config files without doing these 2 steps, in order to get the failed stories?

Thank you for your time

Tanja · February 27, 2020, 10:48am

Great, we found the issue! I’m afraid you need to execute two commands if you want to (1) compare models and (2) get the failed stories for now. So, please first compare the models and then check for which of the models you actually want to have the failed stories. Then you can get those by rasa test core --model <path-to-model.tar.gz>. I’ll forward the feedback and we might gonna add failed stories also the compare models output.

tiziano · February 27, 2020, 10:52am

It’d be nice! Thank you very much for your help

p.s. it would be also nice if this was specified in the docs, so that one wouldn’t have to struggle to find out where the gap was

Topic		Replies	Views
How to Test the Model Rasa Open Source	1	416	February 25, 2021
How to create a test set Rasa Open Source	1	596	December 12, 2019
Trying to understand model training and testing Rasa Open Source	0	272	February 15, 2022
Specify train/test file in rasa train/test nlu Rasa Open Source	1	1302	May 28, 2019
Model evaluation Rasa Open Source	2	764	November 12, 2018

Rasa test not showing results

Related Topics