What is the Meaning of the metric "it/s" when training or evaluating configurations?

Rasa Open Source
1

As the title suggests, I would like to know what the “it” in “it/s” as seen in the log while training or evaluating a configuration means. Iterations maybe? Please elaborate, what that metric tells us. I could not find anything by googling and searching the forum, sorry if this has been asked before.

Example from a recent log:

2020-11-22 17:04:37 INFO rasa.nlu.test - Running model for predictions: 100%|██| 83/83 [00:00<00:00, 182.72it/s]

Thanks in advance

EDIT: Since this can probably not be answered by the community (since it is not described in the docs), can anyone of the Rasa team (maybe @akelad) maybe answer this question?

Thank you. Sorry for being impatient, I am writing my masters thesis at the moment using Rasa and I can’t just use that as a metric without explanation ^^

3

Hi @tobi_fis, the “it” stands for “iteration”, but what that means depends on the context:

  • during training, one iteration will be one epoch
  • during evaluation, one iteration is one NLU example or one story

I hope this helps :slight_smile: