As we know python is not multi-threaded so entity extraction for a huge dataset containing 2.2k entities is taking almost 10 hours of training time. I am currently experimenting with multi-threading, multi-processing & overcoming GIL concepts. But so far no luck. Anyone here can help me with making CRFEntityExtractor into multi-threaded system ?
Any suggestion is appreciated. Thank you.
@Akshay23 can I ask why you have 2.2k entities?
Use case involving general data from daily life @akelad. Yes i know i could have reduced it by making it more general with mentioning entities & values. But that didn’t work correctly as we didn’t had enough training sentences.
Could you give some examples from the data set and what wasn’t working?
hi akelad, could you please answer my question here, thanks advance!