# How can we improve confidence score of intents

**URL:** <https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788>\
**Category:** Rasa Open Source\
**Created:** [October 11, 2018, 7:30am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788 "2018-10-11T07:30:31Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikingsvivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/vikingsvivek/32/752_2.png) [@vikingsvivek](https://forum.rasa.com/u/vikingsvivek)\
**Post date:** [October 11, 2018, 7:30am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/1 "2018-10-11T07:30:31Z")

</div>

Hey guys i am using the pipeline of tensor-flow with rasa-nlu version 0.13.6. I want to know how i can get more confidence in my intents like ‘product\_inform’,‘product\_utility’ and ‘product\_start\_inform’. While creating the model, confidence score is very low around 0.01-0.03.

Please suggest how we can improve the confidence score.

Attached the files.

[config.yml](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/1X/6a70b47bb1e25b8eb685404df459f07eda800d2c.yml) (245 Bytes) [chatbot\_domain.yml](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/1X/ada9cdd18ee9502bce3f1ce5866d524c864ec9d1.yml) (1.7 KB) [nlu\_model.py](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/1X/7c70e43179d47ba927f9d06d174ff84f138360be.py) (658 Bytes)

[data.json](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/1X/f223f2c3f76303cbc5343acb2b86fbab8c33d081.json) (19.1 KB) [stories.md](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/1X/05160acf68fbe6dc762a2e9f9949ec78b7fad2cc.md) (3.9 KB)

---

<div class="post-metadata">

**Author:** ![NikhilBansal21](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/nikhilbansal21/32/418_2.png) [@NikhilBansal21](https://forum.rasa.com/u/NikhilBansal21)\
**Post date:** [October 11, 2018, 12:47pm UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/2 "2018-10-11T12:47:04Z")

</div>

you have to increase your utterances in your training data. However tensorflow requires less as compared to spacy still try adding more utterances it can help in increasing confidence

---

<div class="post-metadata">

**Author:** ![ruiconti](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ruiconti/32/716_2.png) [@ruiconti](https://forum.rasa.com/u/ruiconti)\
**Post date:** [October 11, 2018, 5:36pm UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/3 "2018-10-11T17:36:24Z")

</div>

It seems you’re in the beginning of your project and you don’t have enough data. A good tool that can help you is [Chatito](https://github.com/rodrigopivi/Chatito). You can generate synthetic _datasets_ for any given intent.

---

<div class="post-metadata">

**Author:** ![vikingsvivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/vikingsvivek/32/752_2.png) [@vikingsvivek](https://forum.rasa.com/u/vikingsvivek)\
**Post date:** [October 12, 2018, 6:02am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/4 "2018-10-12T06:02:37Z")

</div>

Hi Nikhil, Please specify more utterance mean? DO we need to add more uttterence against a intent.

---

<div class="post-metadata">

**Author:** ![NikhilBansal21](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/nikhilbansal21/32/418_2.png) [@NikhilBansal21](https://forum.rasa.com/u/NikhilBansal21)\
**Post date:** [October 12, 2018, 6:15am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/5 "2018-10-12T06:15:05Z")

</div>

you will have to increase your training data for all of your intents. Bot will have higher confidence when they are being provided with large training data set .i.e in your data.json file

---

<div class="post-metadata">

**Author:** ![NikhilBansal21](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/nikhilbansal21/32/418_2.png) [@NikhilBansal21](https://forum.rasa.com/u/NikhilBansal21)\
**Post date:** [October 12, 2018, 6:16am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/6 "2018-10-12T06:16:19Z")

</div>

follow this –

> **[NLU Training Data](https://rasa.com/docs/rasa/nlu-training-data/)**
>
> Read more about how to format training data with Rasa NLU for open source natural language processing.

---

<div class="post-metadata">

**Author:** ![vikingsvivek](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/vikingsvivek/32/752_2.png) [@vikingsvivek](https://forum.rasa.com/u/vikingsvivek)\
**Post date:** [October 15, 2018, 5:46am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/8 "2018-10-15T05:46:43Z")

</div>

Hi Rangaraj,

While building the model we are getting low confidence score. Are the above logs captured when we train our model?

please suggest how we can check the same at our end?

Below is the logs while we build the model.

C:\Users\e\Desktop\Chatbot\>python nlu\_model.py C:\Users\e\AppData\Local\Programs\Python\Python36\lib\site-packages\h5py\__init_\_.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from .\_conv import register\_converters as \_register\_converters 2018-10-15 10:29:15.736598: I tensorflow/core/platform/cpu\_feature\_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 Epochs: 100%|█████████████████████████████████████████████████| 300/300 [00:04\<00:00, 71.71it/s, loss=0.107, acc=1.000] C:\Users\e01410\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa\_nlu\extractors\entity\_synonyms.py:85: UserWarning: Failed to load synonyms file from ‘./models/nlu/default/chatbotnlu\entity\_synonyms.json’ “”.format(entity\_synonyms\_file)) {‘intent’: {‘name’: ‘greet’, ‘confidence’: 0.9288316965103149}, ‘entities’: [], ‘intent\_ranking’: [{‘name’: ‘greet’, ‘confidence’: 0.9288316965103149}, {‘name’: ‘goodbye’, ‘confidence’: 0.20464512705802917}, {‘name’: ‘chatbot\_name’, ‘confidence’: 0.08284890651702881}, {‘name’: ‘product\_inform’, ‘confidence’: 0.06197477504611015}, {‘name’: ‘mood’, ‘confidence’: 0.05501033738255501}, {‘name’: ‘chatbot\_info’, ‘confidence’: 0.01574290543794632}, {‘name’: ‘forget\_password’, ‘confidence’: 0.0}, {‘name’: ‘product\_upgrade’, ‘confidence’: 0.0}, {‘name’: ‘product\_start\_inform’, ‘confidence’: 0.0}, {‘name’: ‘product\_utility’, ‘confidence’: 0.0}], ‘text’: ‘Hi’}

---

<div class="post-metadata">

**Author:** ![thinkingmonks](https://avatars.discourse-cdn.com/v4/letter/t/c6cbf5/32.png) [@thinkingmonks](https://forum.rasa.com/u/thinkingmonks)\
**Post date:** [October 15, 2018, 8:05am UTC](https://forum.rasa.com/t/how-can-we-improve-confidence-score-of-intents/1788/9 "2018-10-15T08:05:25Z")

</div>

> [@vikingsvivek](#):
>
> default/chatbotnlu\entity\_synonyms.json

Your intent is properly identified in Rasa NLU

```auto
{"user_input": {"intent": {"name": "product_inform", "confidence": 0.9743123054504395}, "entities": [{"start": 8, "end": 27, "value": "about your products", "entity": "products", "confidence": 0.7121597839662546, "extractor": "ner_crf"}], "intent_ranking": [{"name": "product_inform", "confidence": 0.9743123054504395}, {"name": "product_utility", "confidence": 0.08250807225704193}, {"name": "chatbot_name", "confidence": 0.03798557445406914}, {"name": "product_issue", "confidence": 0.0035776719450950623}, {"name": "forget_password", "confidence": 0.0}, {"name": "account_access_issue", "confidence": 0.0}, {"name": "greet", "confidence": 0.0}, {"name": "mood", "confidence": 0.0}, {"name": "product_start_inform", "confidence": 0.0}, {"name": "product_upgrade", "confidence": 0.0}], "text": "tell me about your products", "project": "nlu", "model": "current"}, "project": "nlu", "model": "current", "log_logger": {"unpersistable": true}, "log_level": {"name": "info", " __class_uuid__": "02e59486-f24d-46ad-8224-3acdf2a5732a"}, "log_namespace": "query-logger", "log_source": null, "log_format": "", "log_time": 1539585185.2487378}

```

But your rasa core is unable to map the intent to action because of your story.

My suggestion is split each your story into one intent and one action and it should solve your issue.