Epochs iteration value and Fallback classifier threshold

Rasa Open Source
1

Is there any specific way to define the “epochs” value of “Pipeline” and “Policies” in the config.yml?

And same goes to “Fallback classifier” threshold, how to define the threshold value?

2

@Zeusturbo Well, as mentioned in the doc : Policies you can mentioned the epochs as per you training data requirement.

For Epochs: One epoch is equals to one forward pass and one backward pass of all the training examples. Sometimes the model needs more epochs to properly learn. Sometimes more epochs don’t influence the performance. The lower the number of epochs the faster the model is trained. Here is how the config would look like:

config.yml

policies:
- name: TEDPolicy
  epochs: 200

pipelines: 
- name: TEDPolicy
     max_history: 5
     epochs: 100
     constrain_similarities: true

For Fallback Classifier: Classifies a message with the intent nlu_fallback if the NLU intent classification scores are ambiguous. The confidence is set to be the same as the fallback threshold . Ref: Components

config.yml

pipeline:
# other components
- name: FallbackClassifier
  threshold: 0.7

Tips: Do refer this link, how to implement default fallback message : Fallback and Human Handoff

I hope this will give you better understanding about the same and solve you issue.

3

But if i reduce the epochs it will affect the accuracy of the model

4

@Zeusturbo Well, it depends on data basically.

Well, the correct answer is the number of epochs is not that significant. more important is the validation and training error .

5

Thanks for the reply.

My issue here is if i use a chain of actions for a scenario, while i initate the final action it always leads to the “Fallback” intent.

ex:

i have an intent to search some “Bar” in a particular locality.

the first action will display the list of bars in the “carousel”(I’m using webchat UI). each carousel have button for “More details” if i click it. it should initiate the next action. But it will go to “fallback” .

i real not getting why

image
image722×487 7.98 KB

i also tried with “Checkpoint” its not working either.

6

@Zeusturbo how many training examples in bar intent? and share config.yml

7

@nik202

here is the config file.

image
image751×773 19 KB

i have “16” nlu train examples.

And need to mention. while i used the old version with markdown for nlu and stories. its works perfectly, once i changed to “yml” it started to behave strage. most of the second intents(more details) falls under "fallback"intent

8

image
image1618×951 39.3 KB
@nik202

here is the code snipped for the action

“Bar_more” payload have the intent to initiate the next action.

9

@Zeusturbo both bar_more and bar_options have 16+ examples, and on which intent you are getting fallback?

10

“bar more” have no training intent cos im pass the value from the api as a “input”. i just specified "bar more " in domain under intents.

Fallback appearing while i initiate “Bar_more”

domain structure below

image

image

11

@Zeusturbo mean user: utter for bar_more option and bot: Fetch the data from API and return as response right? and bar_more doesn’t have any training example is that the case?

12

yes, thats right. Bot fetch input is from the API. i storing those as entities values and pass to the “Bar_more” slots . So there wont be any NLu training example for “Bar_more” intent. Its is jst an another payload to display the fetched data.

13

@Zeusturbo So, in that case there is no linking of Epochs as bar_more doesn’t have training data to train, does it make sense? So, the issue is something different :thinking::thinking:

14

I was suspected the Threshold and epochs cos the second intent might have less confidence so it will initiate the Fallback instead of the actual intent. I’m not sure

Also the story structure seems fine, i dont realy get where the error occurs… :confused:

15

@Zeusturbo can you update this

policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: RulePolicy
    nlu_threshold: 0.4
    core_threshold: 0.4
    fallback_action_name: "action_default_fallback"

Just delete the FallbackClassifier line 18,19 and 20 and add utter_default: " Sorry did not get it you" in domain and delete all the previous trained model.

Example:


utter_default:
    text: I'm sorry, I didn't quite get you? Please try again..
16

@nik202 sure… i will do it right away… :+1: