Single line story is not performing good

Rasa Open Source
1

Why Rasa is not performing good on single line stories ?? Policy.yml

policies:
  - name: EmbeddingPolicy
    epochs: 2000
    attn_shift_range: 5
  - name: FormPolicy

## story 01
*agent.acquaintance
    - utter_agent.acquaintance
    
## story 02
*agent.age
    - utter_agent.age
    
## story 03
*agent.annoying
    - utter_agent.annoying
    
## story 04
*agent.answer_my_question
    - utter_agent.answer_my_question

## story 05
*agent.bad
    - utter_agent.bad
    
## story 06
*agent.be_clever
    - utter_agent.be_clever
    
## story 07
*agent.beautiful
    - utter_agent.beautiful
    
## story 08
*agent.birth_date
    - utter_agent.birth_date
    
## story 09
*agent.boring
    - utter_agent.boring
    
## story 10
*agent.boss
    - utter_agent.boss
    
## story 11
*agent.busy
    - utter_agent.busy
    
## story 12
*agent.can_you_help
    - utter_agent.can_you_help

## story 13
*agent.chatbot
    - utter_agent.chatbot
    
## story 14
*agent.clever
    - utter_agent.clever
    
## story 15
*agent.crazy
    - utter_agent.crazy
    
## story 16
*agent.fired
    - utter_agent.fired
    
## story 17
*agent.funny
    - utter_agent.funny
    
## story 18
*agent.good
    - utter_agent.good
    
## story 19
*agent.happy
    - utter_agent.happy
    
## story 20
*agent.hobby
    - utter_agent.hobby
    
## story 21
*agent.hungry
    - utter_agent.hungry
    
## story 22
*agent.marry_user
    - utter_agent.marry_user
    
## story 23
*agent.my_friend
    - utter_agent.my_friend
    
## story 24
*agent.occupation
    - utter_agent.occupation
    
## story 25
*agent.origin
    - utter_agent.origin
    
## story 26
*agent.ready
    - utter_agent.ready
    
## story 27
*agent.real
    - utter_agent.real
    
## story 28
*agent.residence
    - utter_agent.residence
    
## story 29
*agent.right
    - utter_agent.right
    
## story 30
*agent.sure
    - utter_agent.sure

## story 31
*agent.talk_to_me
    - utter_agent.talk_to_me

## story 32
*agent.there
    - utter_agent.there
    
## story 33
*appraisal.bad
    - utter_appraisal.bad
    
## story 34
*appraisal.good
    - utter_appraisal.good
    
## story 35
*appraisal.no_problem
    - utter_appraisal.no_problem
    
## story 36
*appraisal.thank_you
    - utter_appraisal.thank_you
    
## story 37
*appraisal.welcome
    - utter_appraisal.welcome

## story 38
*appraisal.well_done
    - utter_appraisal.well_done
    
## story 39
*dialog.hold_on
    - utter_dialog.hold_on

## story 40
*dialog.hug
    - utter_dialog.hug
    
## story 41
*dialog.i_do_not_care
    - utter_dialog.i_do_not_care
    
## story 42
*dialog.sorry
    - utter_dialog.sorry
    
## story 43
*dialog.what_do_you_mean
    - utter_dialog.what_do_you_mean
    
## story 44
*dialog.wrong
    - utter_dialog.wrong
    
## story 45
*emotions.ha_ha
    - utter_emotions.ha_ha
    
## story 46
*emotions.wow
    - utter_emotions.wow
    
## story 47
*greetings.bye
    - utter_greetings.bye
    
## story 48
*greetings.goodevening
    - utter_greetings.goodevening
    
## story 49
*greetings.goodmorning
    - utter_greetings.goodmorning
    
## story 50
*greetings.goodnight
    - utter_greetings.goodnight
    
## story 51
*greetings.hello
    - utter_greetings.hello
    
## story 52
*greetings.how_are_you
    - utter_greetings.how_are_you
       
## story 53
*greetings.nice_to_meet_you
    - utter_greetings.nice_to_meet_you
    
## story 54
*greetings.nice_to_see_you
    - utter_greetings.nice_to_see_you
    
## story 55
*greetings.nice_to_talk_to_you
    - utter_greetings.nice_to_talk_to_you
    
## story 56
*greetings.whatsup
    - utter_greetings.whatsup

## story 57
*user.angry
    - utter_user.angry
    
## story 58
*user.back
    - utter_user.back
    
## story 59
*user.bored
    - utter_user.bored
    
## story 60
*user.busy
    - utter_user.busy
    
## story 61
*user.can_not_sleep
    - utter_user.can_not_sleep
    
## story 62
*user.does_not_want_to_talk
    - utter_user.does_not_want_to_talk
    
## story 63
*user.excited
    - utter_user.excited
    
## story 64
*user.going_to_bed
    - utter_user.going_to_bed
    
## story 65
*user.good
    - utter_user.good
    
## story 66
*user.happy
    - utter_user.happy
    
## story 67
*user.has_birthday
    - utter_user.has_birthday
    
## story 68
*user.here
    - utter_user.here
    
## story 69
*user.joking
    - utter_user.joking
    
## story 70
*user.likes_agent
    - utter_user.likes_agent
    
## story 71
*user.lonely
    - utter_user.lonely
    
## story 72
*user.looks_like
    - utter_user.looks_like
    
## story 73
*user.loves_agent
    - utter_user.loves_agent

## story 74
*user.misses_agent
    - utter_user.misses_agent
    
## story 75
* user.needs_advice
    - utter_user.needs_advice

## story 76
*user.sad
    - utter_user.sad

## story 77
*user.sleepy
    - utter_user.sleepy

## story 78
*user.testing_agent
    - utter_user.testing_agent

## story 79
*user.tired
    - utter_user.tired
    
## story 80
*user.waits
    -utter_user.waits

## story 81
*user.wants_to_see_agent_again
    - utter_user.wants_to_see_agent_again

## story 82
*user.wants_to_talk 
    - utter_user.wants_to_talk
    
## story 83
*user.will_be_back
    - utter_user.will_be_back
    
## story 84
*confirmation.yes
    - utter_confirmation.yes

## story 85
*confirmation.cancel
    - utter_confirmation.cancel

## story 86
*confirmation.no
    - utter_confirmation.no


``
2

What do you mean by is not performing good?

3

I am facing a similar issue. Rasa isn’t able to predict the action for the intent present in single line story and invokes the TwoStageFallbackPolicy. Policies in my config.yml is as follows:

policies:

  • name: MemoizationPolicy max_history: 3

  • name: KerasPolicy featurizer:

    • name: MaxHistoryTrackerFeaturizer

      max_history: 3

      state_featurizer:

      • name: BinarySingleStateFeaturizer

  • name: MappingPolicy

  • name: FormPolicy

  • name: TwoStageFallbackPolicy

    nlu_threshold: 0.65

    core_threshold: 0.35

    fallback_nlu_action_name: “action_default_fallback”

    fallback_core_action_name: “action_default_ask_affirmation”

    deny_suggestion_intent_name: “out_of_scope”

My log looks like this:

 rasa.core.policies.memoization  - There is no memorised next action
 rasa.core.policies.mapping_policy  - There is no mapped action for the predicted intent, 'ask_flush_slots'.
 rasa.core.policies.form_policy  - There is no active form
 rasa.core.policies.ensemble  - Predicted next action using policy_4_TwoStageFallbackPolicy
 rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
 rasa.core.processor  - Action 'action_listen' ended with events '[]'
 rasa.core.agent  - Deleted lock for conversation 'T0NyStArk' (unused)

What may be causing such an issue? Any idea anyone? Should I fiddle with augmentation_factor during training?

4

could be low nlu confidence

5

@Ghostvv Can you explain a bit more on this?? Is the nlu confidence threshold mentioned somewhere in domain.yml ?

6

the confidence of your nlu intent classification is below nlu_threshold: 0.65 that you set in the config

7

@Ghostvv Hi… I am facing similar issue. All my stories have single intent. The NLU and Core thresholds are 0.3 each. What can i do here? I have even tried augmentation factor from 20 to 100 but to no avail.