Hey, I’m Ruben from the Jovo team, happy to be here.
We are currently trying to compensate for the removal of projects, in essence we need a different model per user, so high scalability is key.
Currently, we have a Flask server, which dynamically generates a model and spins up an agent for it. Since an agent can take up memory of up to 240MB, we can’t/don’t want to run them in parallel, as this would be too costly with 20+ agents. Dynamically loading an agent upon request takes up to 12s, not an alternative. What we were trying to do now was to spin up agents asynchronously and then use shelve to serialize and persist the agent instance and load that on request, which should be much faster and way better to scale. Unfortunately, we get this error when trying to serialize the agent:
Serializing it did work neither with dill nor with json.dumps. I’m not a Python expert, in fact this is the first Python code I wrote in years, but a bit of debugging led me to believe that this error only comes up when using the DIETClassifier component. Is there a solution to that problem or do I need to look into other classifier components?
Nice to meet you as well Unfortunately cannot confirm. Tried it outside of the Flask scope and it still fails with the same error. Serialization fails for a trained/loaded DIETClassifier.
$ rasa --version
$ Rasa Version : 2.8.1
$ Minimum Compatible Version: 2.8.0
$ Rasa SDK Version : 2.8.1
$ Rasa X Version : None
$ Python Version : 3.8.11
$ Operating System : Linux-5.10.52-1-MANJARO-x86_64-with-glibc2.2.5
$ Python Path : /*/python/bin/python3.8