@koaning here is the requirements.txt if needed
ujson
tensorflow
rasa[spacy]
#en-core-web-lg @ https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.0.0/en_core_web_lg-3.0.0-py3-none-any.whlI’m wondering … maybe by manually installing your requirements.txt file you’re overwriting the Rasa version and are automatically upgrading to a newer version. I’d also recommend against installing tensorflow manually since this is handled by the rasa install.
@koaning yes I removed everything from the requirements.txt except the en_core_web_lg which is needed. I noticed that it is trying to load the interpreter from ‘models’. What is ‘models’? Is that a default path to look for models when executing “rasa run --enable-api”?
@Tanja any pointers here? I have seen some of your posts relevant to this issue.
@koaning I found the issue. When I tried the following pipeline, endpoint gets deployed correctly and works without any issues.
language: "en"
pipeline:
- name: WhitespaceTokenizer
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: CountVectorsFeaturizer
analyzer: "word"
min_ngram: 1
max_ngram: 4
- name: DIETClassifier
epochs: 100
entity_recognition: False
constrain_similarities: True
But when I run the following pipeline, I got the ERROR (in the screenshot in the previous post)
language: "en_core_web_lg"
pipeline:
- name: SpacyNLP
model: "en_core_web_lg"
- name: SpacyTokenizer
- name: SpacyFeaturizer
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: RegexEntityExtractor
- name: CRFEntityExtractor
- name: CountVectorsFeaturizer
analyzer: "word"
min_ngram: 1
max_ngram: 4
- name: EntitySynonymMapper
- name: DIETClassifier
epochs: 100
entity_recognition: False
So that means something is not working correctly with the Spacy language model when deployed in docker container. I don’t know what the cause is. I still want to investigate this issue and find solution.
I think it might be related to the spaCy 3.0 upgrade now that I think about it. In Rasa 2.5 we upgraded our spaCy dependency from 2.3 to 3.0. It’s likely related to that.
@koaning create issue?
Can you confirm the Rasa + spaCy version you’re using, both inside and outside of the container?
spacy version used outside of the docker is slightly different. I did “pip install rasa[spacy]” in docker but somehow that installs spacy 3.0.5 while the same code installs spacy 3.0.6 on local machine. Is this a huge version difference though?
Inside Docker:
Outside Docker:
@koaning I think the issue is that it is looking for the language model, en_core_web_lg, from inside of the container in the deploy job. The container used to train the model is not the same container that gets used in the deploy job. Maybe I need to save the language model in a persistent storage and mount it to the container that is used in the deploy job? I don’t know how to do that but would that be the solution?
Ah yeah that makes sense. If you really want the container to be stateless I suppose you could run this during the docker build (warning, container will get a fair bit bigger):
python -m spacy download en_core_web_lg
I gotta ask, any reason in particular to use the lg model? The md is much smaller.
@koaning lg model has way bigger vocabulary size therefore increases the model accuracy?
Yes and no. In theory it indeed has a larger vocabulary, but in practice this only matters if you assistant needs all of that vocabulary. SpaCy does a neat trick internally for vectors. In the medium model vectors are shared across words.
I can’t recall the perfect example, but I recall words like “dog” and “puppy”. Sure they’re different words, but for many intents and purposes they might as well point to the same word vector.
1 Like
Note, this thread has motivated me to write a blogpost on how to run Rasa NLU in a container. If you’re up for it, feel free to review here.
1 Like
@koaning thanks for sharing the draft of your blogpost. I think something like this would really help people who are trying to deploy their NLU endpoint. I was thinking about a similar blogpost but more comprehensive one to include gitlab ci/cd jobs. In practice, most of the time we tend to deploy things via AutoDevOps so it might be useful to cover end to end process. But I reviewed your draft, it looks pretty good to me as it is. It appears to be pretty hands-on so that is great. I’m glad I was able to motivate this blogpost and contribute as well.
@koaning by the way, my SpaCy model is now up and running correctly in k8s. I did two things to make it work; 1-) I used en_core_web_md as you suggested, 2-) I changed the order of my ci/cd jobs (build and push the container as the last job as opposed to previous configuration where I had them as first jobs to execute)
1 Like
Grand!
Yeah I was running the en_core_web_lg model on a Cloud Run container … it runs … but getting it from a cold to a warm state can take 10-20s. Which is a bit high in terms of latency. It makes sense though, it takes a while to boot up a 1-2Gb container. Kubernetes sounds like a better idea.
1 Like