Improper intent recognition

i Am trying to experiment on executing fallback intents and failed to do so. I has given some random text as “hjsdhjsd” and i am getting answer of some other question as response.

I had fallback intent defined in dilogue management as below.

from rasa_core.agent import Agent fallback = FallbackPolicy(fallback_action_name=“utter_unclear”, core_threshold=0.2, nlu_threshold=0.1)

agent = Agent(‘domain.yml’, policies=[MemoizationPolicy(), KerasPolicy(), fallback]).

But still wrong action is performing. When i dig into reason, I found that entity is being recognized improper through rasa nlu.

When I am trying to identify the entity for any random text like ‘hkjdfkd’, the intent is wrongly identified as DepositEnquiryForAgency . This output is thrown when I run nlu_model.py. Could you please help me on how to correct this. i am surprised to see that rasa nlu is giving confidence as 0.907 for random text. COuld someone please let me know where i am doing wrong??

{‘intent’: {‘name’: ‘DepositEnquiryForAgency’, ‘confidence’: 0.9075979430989889}, ‘entities’: [], ‘intent_ranking’: [{‘name’: ‘DepositEnquiryForAgency’, ‘confidence’: 0.9075979430989889}, {‘name’: ‘goodbye’, ‘confidence’: 0.062111495788272415}, {‘name’: ‘greet’, ‘confidence’: 0.030290561112738677}], ‘text’: ‘jhfdkjsh’}

Data file :

{

“rasa_nlu_data”: {

"common_examples": [

  {

    "text": "Hello",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "goodbye",

    "intent": "goodbye",

    "entities": []

  },

  {

    "text": "What's the pending deposit for federal",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 11,

        "end": 18,

        "value": "pending",

        "entity": "depositStatus"

      },

      {

        "start": 31,

        "end": 38,

        "value": "federal",

        "entity": "Jurisdiction"

      }

    ]

  },

  {

    "text": "hey",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "hello",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "hi",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "heya",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "howdy",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "hey there",

    "intent": "greet",

    "entities": []

  },

  {

    "text": "bye",

    "intent": "goodbye",

    "entities": []

  },

  {

    "text": "goodbye",

   "intent": "goodbye",

    "entities": []

  },

  {

    "text": "bye bye",

    "intent": "goodbye",

    "entities": []

  },

  {

    "text": "see ya",

    "intent": "goodbye",

    "entities": []

  },

  {

    "text": "see you later",

    "intent": "goodbye",

    "entities": []

  },

  {

    "text": "Show me what's the pending deposit  for FLSUI",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 19,

        "end": 26,

        "value": "pending",

        "entity": "depositStatus"

      },

      {

        "start": 40,

        "end": 45,

        "value": "FLSUI",

        "entity": "Jurisdiction"

      }

    ]

  },

  {

    "text": "for Failed",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 4,

        "end": 10,

        "value": "Failed",

        "entity": "depositStatus"

      }

    ]

  },

  {

    "text": "Failed",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 1,

        "end": 6,

        "value": "Failed",

        "entity": "depositStatus"

      }

    ]

  },

  {

    "text": "Oh, sorry, in Held",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 14,

        "end": 18,

        "value": "Held",

        "entity": "depositStatus"

      }

    ]

  },

  {

    "text": "tell me failed deposit for flsui",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 8,

        "end": 14,

        "value": "failed",

        "entity": "depositStatus"

      },

      {

        "start": 27,

        "end": 32,

        "value": "flsui",

        "entity": "Jurisdiction"

      }

    ]

  },

  {

    "text": "How about federal",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 10,

        "end": 17,

        "value": "federal",

        "entity": "Jurisdiction"

      }

    ]

  },

  {

    "text": "How about flsui",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 10,

        "end": 15,

        "value": "flsui",

        "entity": "Jurisdiction"

      }

    ]

  },

  {

    "text": "How about failed",

    "intent": "DepositEnquiryForAgency",

    "entities": [

      {

        "start": 10,

        "end": 16,

        "value": "failed",

        "entity": "depositStatus"

      }

    ]

  }

]

}

}

Hi, try to increase the threshold value of RASA nlu and RASA core. It will work

But here my question is Rasa NLU is identifying confidence as 0.9 for my random text… So i need to have threshold more than 0.9 to go for fallback intent. Would it be a good idea to have threshold more than 0.9?

What version of rasa_nlu do you use? Do you have oov turned on for count_vector_featurizer in your nlu pipeline?

i am using rasa-nlu 0.13.0. I am new bee to AI and bots. Could you please guide me where to turn on count_vector_featurizer??

Below is my config file

{ “pipeline”:“spacy_sklearn”, “path”:"./models/nlu", “data”:"./data/data.json" }

Try to add more examples and train it online and generate proper stories. The number of examples for each intent you have added are very less.

If I turned on OOV_token for “intent_featurizer_count_vectors” the above problem mentioned will be solved?

pipeline:
- name: "intent_featurizer_count_vectors" #Creates bag-of-words representation of intent features
     OOV_token: oov

Sorry, I thought you use different pipeline. Try using tensorflow_embedding pipeline (http://www.rasa.com/docs/nlu/choosing_pipeline/)