Hi, I wanted to try out some defferent configs for my chatbot just like in this video.
I tried configs from video on Sara chatbot and the results were the same as in the video, so my rasa instalation is probably ok.
My chatbot has around 30 intents with 10-20 examples per intent. No entities.
All configs from video that I used are avaible here
First I tried config-mega-basic and it gave me pretty acceptable results.
Hi @kaladin. Cool post! The number of examples per intent you need to make DIET work depends on what performance you expect, how you divide your assistant into intents, and the training data you provide. 10-20 examples per intent for 30 intents seems like a really small amount to me though.
I think only you can answer if you need 100+ examples per intent. Following Conversation-Driven Development will allow you to grow a dataset until your assistant performs as well as you desire. There are some good tips in 10 Best Practices for Designing NLU Training Data that should also help along with Rasa X, which makes it easy to collect and annotate those training examples.
one regret I have with the original benchmark that you list there is that the epochs are … rather small. It might be better to push them to 100 epochs to ensure that DIET converged (I just updated the gist). Should you ever run the benchmarks again I’d love to hear about it.
What Ty said is something to keep in mind too. I’m assuming that currently running a model against data that you came up with but odds are that your users are going to behave differently.
Starting out with 20-30 examples per intent can be fine (you need to start somewhere) but eventually you’ll want to collect more data. Preferably from real users. Not just because “more data is better” but also because your examples will cover more ground, language wise.
And as Ty mentioned, Rasa X should be a great startinging point for this. It’s free and it integrates with Rasa from the get-go.
