All core usage in RASA

As we know python is not multi-threaded so entity extraction for a huge dataset containing 2.2k entities is taking almost 10 hours of training time. I am currently experimenting with multi-threading, multi-processing & overcoming GIL concepts. But so far no luck. Anyone here can help me with making CRFEntityExtractor into multi-threaded system ?

Any suggestion is appreciated. Thank you.

@Akshay23 can I ask why you have 2.2k entities?

Use case involving general data from daily life @akelad. Yes i know i could have reduced it by making it more general with mentioning entities & values. But that didn’t work correctly as we didn’t had enough training sentences.

Could you give some examples from the data set and what wasn’t working?