Is it possible to train the rasa model in parts or partially? Like I have 4 folders of data, I have to train the first 2 and update the data from the next 2 and I don’t want to train it again on the previous 2, to save time.
@Anurag02 Rasa trains on the data you have specified in nlu.yml and stories.yml. And then it creates .tar.gz file as a trained model in models folder. So, for your case, you can try with various versions of nlu.yml and stories.yml file (also config.yml and other files as per the model you choose) and train different model. Each of the model can be used for inference later on.
NOTE: Remember to keep only model you want to test in models folder.
how can i use those trained models to interface later on? or can i merge those models?
@Anurag02 You cannot merge models. You can keep one trained model in the modes folder and then directly test.
how can i test the both models simultaneously? because at the end i have use all the folders data for my bot
@Anurag02 Model will be one only. You can play with .yml files. Include all your nlu.yml files in one nlu.yml, similarly for other files too.
@Anurag02 model are trained based on core and nlu files basically i.e nlu, stories and rules basically and on the top domain.yml. You can not partial trained the model and merge again. It not recommended and even possible. If you have two projects then you can train the model separately but that need to run on different port i.e 5005 or 5006 respectively. Same applied to rasa action server. On the other hand If you have two different model for single project and you want to test then you can do that.
rasa shell -m models/model1.tar.gz
rasa shell -m models/model2.tar.gz
rasa train --finetune
Explain how it is work
From the docs:
In order to improve the performance of an assistant, it’s helpful to practice CDD and add new training examples based on how your users have talked to your assistant. You can use rasa train --finetune to initialize the pipeline with an already trained model and further finetune it on the new training dataset that includes the additional training examples. This will help reduce the training time of the new model.
By default, the command picks up the latest model in the models/ directory. If you have a specific model which you want to improve, you may specify the path to this by running rasa train --finetune <path to model to finetune>. Finetuning a model usually requires fewer epochs to train machine learning components like DIETClassifier, ResponseSelector and TEDPolicy compared to training from scratch. Either use a model configuration for finetuning which defines fewer epochs than before or use the flag --epoch-fraction. --epoch-fraction will use a fraction of the epochs specified for each machine learning component in the model configuration file. For example, if DIETClassifier is configured to use 100 epochs, specifying --epoch-fraction 0.5 will only use 50 epochs for finetuning.
You can also finetune an NLU-only or dialogue management-only model by using rasa train nlu --finetune and rasa train core --finetune respectively.
To be able to fine tune a model, the following conditions must be met:
- The configuration supplied should be exactly the same as the configuration used to train the model which is being finetuned. The only parameter that you can change is
epochsfor the individual machine learning components and policies. - The set of labels(intents, actions, entities and slots) for which the base model is trained should be exactly the same as the ones present in the training data used for finetuning. This means that you cannot add new intent, action, entity or slot labels to your training data during incremental training. You can still add new training examples for each of the existing labels. If you have added/removed labels in the training data, the pipeline needs to be trained from scratch.
- The model to be finetuned is trained with
MINIMUM_COMPATIBLE_VERSIONof the currently installed rasa version.
If you don’t know what incremental training (finetuning) is, look it up on Wikipedia.