Using rasa 2.8.26 to buid model, but the suggestion return value not make sense

Rasa Open Source
1

we use rasa 2.8.26 in our project. we trained a model by below training data:

nlu:
   -  intent: 9489fc71-bb03-4d65-9e14-e8bc8200d317
      examples: |
         -  Wireless connection drops every 5 minutes
         -  No connection to the internet with my PDA
         -  Wireless is not very stable
         -  Wireless connection unavailable
         -  Wireless connection isn't working
         -  No wireless connection
         -  Wireless connects with message "limited or no connectivity"
   -  intent: 32bc13ef-b92e-42ac-a01a-4736468d3d76
      examples: |
         -  I have problem using AWS service
         -  AWS service issue
         -  We have some issues for AWS

but when we try to parse, and type any text , for example : {“text”:“abcdefg”}

you could see " abcdefg" never appear in training sample data above.

but rasa gives a very high confiendence return value , for example:

{

"text": "abcdefg",

"intent":{

"id": 8652920554070762147,

"name": "32bc13ef-b92e-42ac-a01a-4736468d3d76",

"confidence": 0.9999998807907104

},

"entities":[],

"intent_ranking":[

{

"id": 8652920554070762147,

"name": "32bc13ef-b92e-42ac-a01a-4736468d3d76",

"confidence": 0.9999998807907104

},

{

"id": 1744349216196580737,

"name": "9489fc71-bb03-4d65-9e14-e8bc8200d317",

"confidence": 1.4354384347825544e-7

}

],

"response_selector":{

"all_retrieval_intents":[],

"default":{

"response":{"id": null, "responses": null, "response_templates": null, "confidence": 0.0, "intent_response_key": null,…},

"ranking":[]

}

},

"project": "555500001-en",

"model": "555500001-en.tar.gz"

}

we dont think this is make any sense.

if anyone could help us on this situation ? thanks in advanced.

here is our config, we copied from offical document.

recipe: default.v1
language: en

pipeline:
   - name: WhitespaceTokenizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
     analyzer: char_wb
     min_ngram: 1
     max_ngram: 4
   - name: DIETClassifier
     epochs: 500
     constrain_similarities: true
   - name: EntitySynonymMapper
   - name: ResponseSelector
     epochs: 500
     constrain_similarities: true
   - name: FallbackClassifier
     threshold: 0.3
     ambiguity_threshold: 0.1
2

Please read Handling Out-of-scope Messages

3

thanks for reply. I’ve read the text you posted, but in that post, I couldn’t find any solution. rasa gives a high confidence (>0.99) for an intent that doesn’t seem relevant, and I can’t filter it with fallback intent. At the same time, we can’t predict what request content the user will randomly enter, so we have no way to set an ‘out of scope intent’ to handle it. " You will need to define an out_of_scope intent in your NLU training data and add any known out-of-scope requests as training examples, for example:" how do we know all possiblity out of scope request text and list it here?

4

actually we use rasa-nlu in this project before, the version is 0.13.7. the pipeline concept already be there in that old version. and we use intent_classifier_tensorflow_embedding as the main classifer but when we try to upgrade rasa frame work to 2.8 . we meet above issue. the response from new RASA does not make sense but still give a very high conferdence