LanguageModelFeaturizer in pipeline dont work!

Rasa Open Source
1

when i add LanguageModelFeaturizer to pipeline, i get the error:

image
image1595×540 81.9 KB

_____________________________________________

my config:

language: vi

pipeline:
#- name: VietnameseTokenizer
  - name: "tone_prediction_component.TonePredictionComponent"
  - name: LanguageModelFeaturizer
    # model_name: "bert"
    # model_weights: "bert-base-multilingual-cased"
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
  - name: FallbackClassifier
    threshold: 0.3
    ambiguity_threshold: 0.1

#  ambiguity_threshold: 0.1
##
# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/


policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: RulePolicy
2

As the error says, you need to have a tokenizer before LanguageModelFeaturizer

3

it work, thank you very much!

1 Like