# Rasa NLU predicting random intent

**URL:** <https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765>\
**Category:** Rasa Open Source\
**Created:** [November 1, 2019, 10:34am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765 "2019-11-01T10:34:04Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 1, 2019, 10:34am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/1 "2019-11-01T10:34:05Z")

</div>

I am using rasa\_core==0.12.2 and rasa\_nlu[tensorflow]==0.13.7 NLU config is as follows -

```
pipeline:
- name: tokenizer_whitespace
- name: intent_entity_featurizer_regex
- name: ner_crf
- name: intent_featurizer_count_vectors
  analyzer: 'word'
  min_ngram: 1  
  max_ngram: 1 

```

- name: intent\_classifier\_tensorflow\_embedding epochs: 100 language: en

Dialogue config is as follows -

```
policies:
  - name: MemoizationPolicy
    max_history: 1
  - name: FallbackPolicy
    nlu_threshold: 0.60
    core_threshold: 0.3
  - name: FormPolicy

```

Bot gives random response even if that particular text never appeared in the nlu training data.

I tested following scenarios -

scenario 1 -

```
   User - What is hippa compliance?
   Bot - Hi I am a bot. How may I help you

```

scenario2 -

```
User - What is rasa nlu
Bot - I am fine

```

My expectation is that if bot is not trained on some conversation it should respond with fallback (Please try again I do not understand).

Why is bot giving random response?

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 4, 2019, 1:04pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/2 "2019-11-04T13:04:37Z")

</div>

Has anyone also faced the same issue?

---

<div class="post-metadata">

**Author:** ![JiteshGaikwad](https://avatars.discourse-cdn.com/v4/letter/j/77aa72/32.png) [@JiteshGaikwad](https://forum.rasa.com/u/JiteshGaikwad)\
**Post date:** [November 4, 2019, 1:28pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/3 "2019-11-04T13:28:50Z")

</div>

hey @r4sn4, just check what is the confidence score you get for the above queries, is it above your nlu threshold that you have defined in the fallback policy? 🙂

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 5, 2019, 5:24am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/4 "2019-11-05T05:24:22Z")

</div>

@JiteshGaikwadI have already checked threshold and it is coming to be above 0.7. Why threshold is coming out to be so high for words which never existed in my data? Even if it is resolving to random intent, it should resolve with low confidence score? On what basis it resolve to particular intent?

---

<div class="post-metadata">

**Author:** ![JiteshGaikwad](https://avatars.discourse-cdn.com/v4/letter/j/77aa72/32.png) [@JiteshGaikwad](https://forum.rasa.com/u/JiteshGaikwad)\
**Post date:** [November 5, 2019, 5:34am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/5 "2019-11-05T05:34:16Z")

</div>

@r4sn4 might be it’s able to fine atleast some of the words in your training data, I mean as in the above example you had asked

> What is hippa compliance?

> What is rasa nlu

since you are using `tensorflow_embedding` as the classifier, it creates the bag of words from the training data and the words like `what`, `is` are getting matched with the bag of words.

can you check with `sklearn classifer` and see the nlu results

---

<div class="post-metadata">

**Author:** ![JiteshGaikwad](https://avatars.discourse-cdn.com/v4/letter/j/77aa72/32.png) [@JiteshGaikwad](https://forum.rasa.com/u/JiteshGaikwad)\
**Post date:** [November 5, 2019, 5:36am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/6 "2019-11-05T05:36:56Z")

</div>

Check this post:

> **[Supervised Word Vectors from Scratch in Rasa NLU](https://medium.com/rasa-blog/supervised-word-vectors-from-scratch-in-rasa-nlu-6daf794efcd8)**
>
> We’ve released a new pipeline which is totally different from the standard Rasa NLU approach. It uses very little memory, handles…

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 5, 2019, 5:44am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/7 "2019-11-05T05:44:14Z")

</div>

if this is the case. then it should resolve to intents where ‘what is’ is present in the conversation, but instead it is pointing to intents which does not contain such words at all. with SKlearn classifier, accuracy of model degrades even further.

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 7, 2019, 6:40am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/8 "2019-11-07T06:40:10Z")

</div>

@juste_petr Hi… Any solution for this problem?

---

<div class="post-metadata">

**Author:** ![jonathanpwheat](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jonathanpwheat/32/4880_2.png) [@jonathanpwheat](https://forum.rasa.com/u/jonathanpwheat)\
**Post date:** [November 8, 2019, 12:50pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/9 "2019-11-08T12:50:55Z")

</div>

@r4sn4 - Does it respond properly for things it should? And it is just random for things it doesn’t know about? or is it random for everything?

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 11, 2019, 9:24am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/10 "2019-11-11T09:24:12Z")

</div>

@jonathanpwheat - It responds properly for conversation it is trained on. Whereas for conversations it does not know it responds with random intent.

---

<div class="post-metadata">

**Author:** ![jonathanpwheat](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jonathanpwheat/32/4880_2.png) [@jonathanpwheat](https://forum.rasa.com/u/jonathanpwheat)\
**Post date:** [November 11, 2019, 9:09pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/11 "2019-11-11T21:09:18Z")

</div>

I just went through setting up the default fallback. I think you need to add a fallback\_action\_name to your `config.yml` file like this

```auto
  - name: "FallbackPolicy"
    nlu_threshold: 0.3
    core_threshold: 0.3
    fallback_action_name: "action_default_fallback" 

```

Then add this to your actions.py file

```auto
class ActionDefaultFallback(Action):
    def name(self) -> Text:
        return "action_default_fallback"

    def run(
        self,
        dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any],
    ) -> List["Event"]:

        dispatcher.utter_template("utter_default", tracker)
        return [UserUtteranceReverted()]

```

in your `domain.yml` file, you create an `utter_default` template like this to handle your responses:

```auto
  utter_default:
    - text: "Sorry, I didn't get that. Could you please rephrase?"
    - text: "I didn't understand, could you rephrase that?"
    - text: "I'm sorry, but I didn't understand you. Could you please rephrase what you just said?"
    - text: "I'm afraid I didn't get what you just said. Could you rephrase that?"
    - text: "I didn't quite get that, could you rephrase your message?"
    - text: "Could you rephrase your message? I didn't get it, I'm sorry."

```

Also in your `domain.yml` file add the fallback action name to the bottom of your `actions:` section

`action_default_fallback`

I think that’s “all” I had to to do get a fallback response to work.

retrain it and type some garble in, and you should get one of the random responses back.

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 12, 2019, 5:35am UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/12 "2019-11-12T05:35:35Z")

</div>

default action is already present in my code.

---

<div class="post-metadata">

**Author:** ![jonathanpwheat](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jonathanpwheat/32/4880_2.png) [@jonathanpwheat](https://forum.rasa.com/u/jonathanpwheat)\
**Post date:** [November 12, 2019, 12:54pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/13 "2019-11-12T12:54:29Z")

</div>

Hmm, ok. I don’t mean to talk down to you so please don’t take this as such.

Do you have it setup the way I do though?

I seem to remember getting randomish responses before I had it all right.

Just asking because I didn’t see `fallback_action_name: "action_default_fallback"` in your initial `policies:` block above, I think you need that, and the naming convention is important as well.

Again, sorry if you know this already 🙂Maybe if you could post your config.yml, domain.yml and actions.py file to a gist so I (we) can check it out

---

<div class="post-metadata">

**Author:** ![r4sn4](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/r4sn4/32/279_2.png) [@r4sn4](https://forum.rasa.com/u/r4sn4)\
**Post date:** [November 14, 2019, 2:12pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/14 "2019-11-14T14:12:32Z")

</div>

I have upgraded rasa version to 1.3.6. Now not facing this issue.

---

<div class="post-metadata">

**Author:** ![jonathanpwheat](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/jonathanpwheat/32/4880_2.png) [@jonathanpwheat](https://forum.rasa.com/u/jonathanpwheat)\
**Post date:** [November 15, 2019, 5:28pm UTC](https://forum.rasa.com/t/rasa-nlu-predicting-random-intent/20765/15 "2019-11-15T17:28:45Z")

</div>

Oh, that’s great news.