# Load jieba self\_dictionay cost too much time

**URL:** <https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306>\
**Category:** Rasa Open Source\
**Created:** [September 6, 2019, 4:32am UTC](https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306 "2019-09-06T04:32:53Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sdu-2044](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/sdu-2044/32/3697_2.png) [@sdu-2044](https://forum.rasa.com/u/sdu-2044)\
**Post date:** [September 6, 2019, 4:32am UTC](https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306/1 "2019-09-06T04:32:53Z")

</div>

Hi，I am new to rasa. And I train a model use the config as follows

> language: “zh” pipeline:
> 
> - name: “JiebaTokenizer” dictionary\_path: “userdict/”
> - name: “RegexFeaturizer”
> - name: “CRFEntityExtractor”
> - name: “EntitySynonymMapper”
> - name: “CountVectorsFeaturizer”
> - name: “EmbeddingIntentClassifier”

In my case ,I put about 100M txt file in the dictionary\_path which contains large number of movie\_name and star\_name.So every time I restart the server or fetch a new model from a server by url I set in endpoints.yml, rasa will cost about 10 second to response for the next request.What is worse is that I deploy my model with docker for about 30 containers .So I will have 30 very slow request each time I train a new model . my user\_dict for jieba will change very offen,because I have to add some new move\_name and star\_name .So I have to train and fetch a new model very offen.Anyone can help me about this.

---

<div class="post-metadata">

**Author:** ![imLew](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/imlew/32/3744_2.png) [@imLew](https://forum.rasa.com/u/imLew)\
**Post date:** [September 13, 2019, 8:44am UTC](https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306/2 "2019-09-13T08:44:27Z")

</div>

I don’t think there is anything from the rasa side that you can do about this as you will have to retrain whenever you update the dictionary. If possible it’d be best to reduce the size of the file. Why is it so large?

---

<div class="post-metadata">

**Author:** ![sdu-2044](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/sdu-2044/32/3697_2.png) [@sdu-2044](https://forum.rasa.com/u/sdu-2044)\
**Post date:** [September 18, 2019, 3:45am UTC](https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306/3 "2019-09-18T03:45:31Z")

</div>

Thanks for reply.I think I have to find some other solution from other side

---

<div class="post-metadata">

**Author:** ![imLew](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/imlew/32/3744_2.png) [@imLew](https://forum.rasa.com/u/imLew)\
**Post date:** [September 23, 2019, 7:50am UTC](https://forum.rasa.com/t/load-jieba-self-dictionay-cost-too-much-time/15306/4 "2019-09-23T07:50:18Z")

</div>

Glad to hear it 👍If you think your solution could help others would mind outlining it here?