End-to-end Training [Experimental]

Hi there, If one builds a bot entirely based on pre-existing live chat messages, would it make sense to:

  1. “Clean” these conversations (remove redundancies, non useful paths etc)
  2. use e2e training only
  3. introduce intent later only if it saves significant training time?

Thank you!

I have 400 pure e2e stories from historic conversational data. I am getting OOM error while training core ( rasa train --augmentation 0 ). I am using 1 GPU with 12 GB RAM. I am using this config.yml.

pipeline:
- name: WhitespaceTokenizer
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: CountVectorsFeaturizer
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4
- name: DIETClassifier
  epochs: 100
- name: EntitySynonymMapper
- name: ResponseSelector
  epochs: 100
policies:
- name: TEDPolicy
  epochs: 10
  max_history: 5
- name: RulePolicy

I have tried changing batch_size but nothing helped.

PS. Its working file for smaller data set. It worked fine for 50 e2e stories.

2 Likes

@vikrant67 Were you able to resolve this challenge with getting your OOM error? I’m interested in trying a similar experiment, and would love to learn from your experience so far.

Tom

was trying out this feature, but got an error during the training stage of rasa core

  File "/media/dingusagar/rasa_2_8/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 960, in <listcomp>
[domain.intents.index(intent) for intent in tracker_intents] 
ValueError: 'my order is late' is not in list

My stories look like this :

  • story: Order late steps:
    • user: “my order is late”
    • action: utter_sorry_to_hear

I am using rasa 2.8, the config is the default one in rasa 2.8. from the error i feel like the user utterance is treated like an intent and its complaining that such intent is not present. Could someone help me understand what am i doing wrong.