How to use own model in pipeline?

Is it possible to use my own model in pipeline? which is in “config.yml” for example, I want to create and teach my own Classifier or Tokenizer and integrate it to rasa?

image

if yes, how can I do that? and where is the locations of the default models like:

- name: WhitespaceTokenizer

- name: RegexFeaturizer

- name: LexicalSyntacticFeaturizer

Hey @Galym, welcome to the forum! :slight_smile:

I recommend to first read more about custom components here, I’m sure you’ll find many answers there.

As for placing your component in the “right” place in the pipeline: It really depends on what your component does, what features (produced by other components) it consumes and what features (potentially used by other components) it adds. E.g. if the component is a classifier that uses features creates by some featurisers, it will come in the pipeline after the featurisers.

Let me know if you’ve got any more specific questions :slight_smile:

1 Like

Thank you very much SamS, after your clarification and provided link, a lot of things a clear now! I recently begin to study RASA and quite impressed what it can do.

After finishing my own model I want to integrate it to RASA. If I will have obstacles with it, I will write it back here :slight_smile:

1 Like

Hello @SamS, Could you please help me with my custom component problem?

After playing around and reading rasa docs I begin to integrate my model into the rasa pipeline but unfortunately, it was unsuccessful despite several attempts and I decided first found some related tutorial and do it to understand the pipeline more deeply.

I decided to do this one , it seems exactly what I need to understand “in” and “out” of each models but unfortunately, I cant implement even this one :frowning: I changed here there and I’m still struggling with this tutorial. I understand overall concepts and theory but trying to solve technical details in each step it seems I made them even more complicated for me.

To make it easy to understand I installed a third brand new Rasa and followed that instruction and get my first error. How should I solve this problem correctly?

So the config.yml the same with turorial

The files: nlu.yml, stories.yml and domain.yml I lived from default Rasa coz in tutorial they bit old.

I’m not sure is it needed but I did this path to: image

with results image

The printer.py file the same as in tutorial

So the bottom line is in the previous two Rasa environment I tried to solve all problems in different files like registry.py, components.py, etc. What are the correct steps to make this tutorial work in this new rasa 2.2/2.3?

If I would understand and did this tutorial I hope I can implement my own model. I would be super grateful if you could help me!

Hey @Galym, welcome to the forum! My guess is that – among other things – the reason why things don’t work for you is that the tutorial itself is outdated. I’m sure @koaning (the author of the tutorial) will be able to help here :slight_smile:

Ahh yeah that tutorial is super outdated. My bad! Adding a notice right now.

What might help is that an up-to-date version of the printer component can be found here as part of our nlu-rasa-examples repository. It’s our contrib-like library for custom components. It’s not officially supported but you can find many custom tricks, especially if you’re interested in Non-English NLP.

You can also find custom word-embeddings here, should it be of interest to see how to implement one of those.

1 Like

Thanks a lot @SamS :slight_smile:

Hello @koaning thank you very much for providing the right place to look, I am definitely interested in all of the components and I recently began my journey to NLP.

After installing rich and try “rasa train” I get the below error. It seems that I need to change in a couple of places?

Look what I get:

For some reason it’s don’t like line 45 below

In addition, I found and learning/testing another component sentiment. Would you please help me to change this simple sentiment.py (1.5 KB) sentiment.py file with any sequence of pipeline to new Rasa 2.2/2.3? If you able to do this you will make not only my day but whole my spring month :slight_smile: Thanks in advance!

@Galym in the future, feel free to post the actual text into the channel instead of the screenshots. That way we can more easily copy/paste the code and help you debug. If you want to share code you can use the markdown syntax to give the code proper highlighting too. You can read more about that syntax here.

That said, I’m not 100% sure what is going wrong because I’m also not 100% sure what you’re running. Could you run rasa --version and share the results? Also, could you share your config.yml-file?

1 Like

Thanks @koaning for pointing out and your quick response. Let my try put actual text :slight_smile:

ERROR:

Traceback (most recent call last):
  File "/home/galym/Nrasa/v/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/__main__.py", line 116, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py", line 58, in <lambda>
    train_parser.set_defaults(func=lambda args: train(args, can_exit=True))
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py", line 102, in train
    finetuning_epoch_fraction=args.epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 109, in train
    loop,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/utils/common.py", line 308, in run_in_loop
    result = loop.run_until_complete(f)
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 174, in train_async
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 353, in _train_async_internal
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 415, in _do_training
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 610, in _train_core_with_validated_data
    model_to_finetune=model_to_finetune,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/train.py", line 70, in train
    agent.train(training_data, **additional_arguments)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/agent.py", line 726, in train
    training_trackers, self.domain, interpreter=self.interpreter, **kwargs
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ensemble.py", line 201, in train
    trackers_to_train, domain, interpreter=interpreter, **kwargs
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 500, in train
    **kwargs,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/policy.py", line 183, in featurize_for_training
    training_trackers, domain, interpreter, bilou_tagging
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 209, in featurize_trackers
    tracker_state_features = self._featurize_states(trackers_as_states, interpreter)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 76, in _featurize_states
    for tracker_states in trackers_as_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 76, in <listcomp>
    for tracker_states in trackers_as_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 74, in <listcomp>
    for state in tracker_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py", line 284, in encode_state
    self._extract_state_features(sub_state, interpreter, sparse=True)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py", line 246, in _extract_state_features
    parsed_message = interpreter.featurize_message(message)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/interpreter.py", line 158, in featurize_message
    result = self.interpreter.featurize_message(message)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/nlu/model.py", line 470, in featurize_message
    component.process(message, **self.context)
  File "/home/galym/Nrasa/printer.py", line 88, in process
    print_message(message)
  File "/home/galym/Nrasa/printer.py", line 45, in print_message
    features["intent"] = {k: v for k, v in features["intent"].items() if "id" != k}
AttributeError: 'str' object has no attribute 'items'

rasa --version

(v) [galym@mx]$ rasa --version
2021-02-25 23:34:24.511716: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-25 23:34:24.511786: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Rasa Version     : 2.3.2
Rasa SDK Version : 2.3.1
Rasa X Version   : None
Python Version   : 3.7.3
Operating System : Linux-4.19.0-12-amd64-x86_64-with-MX-19.3-patito_feo
Python Path      : /home/galym/Nrasa/v/bin/python3

config.yml file:

language: en

pipeline:

  • name: WhitespaceTokenizer
  • name: printer.Printer alias: after tokenizer
  • name: CountVectorsFeaturizer
  • name: printer.Printer alias: after 1st cv
  • name: CountVectorsFeaturizer analyzer: char_wb min_ngram: 1 max_ngram: 4
  • name: printer.Printer alias: after 2nd cv
  • name: LexicalSyntacticFeaturizer
  • name: printer.Printer alias: after lexical syntactic featurizer
  • name: DIETClassifier epochs: 20
  • name: printer.Printer alias: after diet classifier

policies:

  • name: MemoizationPolicy
  • name: TEDPolicy
  • name: MappingPolicy

printer.py from your provided link

let me know if you need anything else

I am beginning to wonder if there is a bug on the rasa-nlu-examples repo.

It might be that this:

    features["intent"] = {k: v for k, v in features["intent"].items() if "id" != k}

needs to be turned into:

    features["intent"] = {k: v for k, v in features["intent_ranking"].items() if "id" != k}

I’ve added this to the issue list to check out. If you could try this locally, I’d be much obliged to any feedback.

Link to issue: Fix printer intents · Issue #104 · RasaHQ/rasa-nlu-examples · GitHub

I changed the line

features["intent"] = {k: v for k, v in features["intent"].items() if "id" != k}
to
features["intent"] = {k: v for k, v in features["intent_ranking"].items() if "id" != k}
but no result, another error

Traceback (most recent call last): File “/home/galym/Nrasa/v/bin/rasa”, line 8, in sys.exit(main()) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/main.py”, line 116, in main cmdline_arguments.func(cmdline_arguments) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py”, line 58, in train_parser.set_defaults(func=lambda args: train(args, can_exit=True)) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py”, line 102, in train finetuning_epoch_fraction=args.epoch_fraction, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py”, line 109, in train loop, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/utils/common.py”, line 308, in run_in_loop result = loop.run_until_complete(f) File “uvloop/loop.pyx”, line 1456, in uvloop.loop.Loop.run_until_complete File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py”, line 174, in train_async finetuning_epoch_fraction=finetuning_epoch_fraction, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py”, line 353, in _train_async_internal finetuning_epoch_fraction=finetuning_epoch_fraction, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py”, line 415, in _do_training finetuning_epoch_fraction=finetuning_epoch_fraction, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py”, line 610, in _train_core_with_validated_data model_to_finetune=model_to_finetune, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/train.py”, line 70, in train agent.train(training_data, **additional_arguments) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/agent.py”, line 726, in train training_trackers, self.domain, interpreter=self.interpreter, **kwargs File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ensemble.py”, line 201, in train trackers_to_train, domain, interpreter=interpreter, **kwargs File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py”, line 500, in train **kwargs, File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/policy.py”, line 183, in featurize_for_training training_trackers, domain, interpreter, bilou_tagging File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py”, line 209, in featurize_trackers tracker_state_features = self._featurize_states(trackers_as_states, interpreter) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py”, line 76, in _featurize_states for tracker_states in trackers_as_states File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py”, line 76, in for tracker_states in trackers_as_states File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py”, line 74, in for state in tracker_states File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py”, line 284, in encode_state self._extract_state_features(sub_state, interpreter, sparse=True) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py”, line 246, in _extract_state_features parsed_message = interpreter.featurize_message(message) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/interpreter.py”, line 158, in featurize_message result = self.interpreter.featurize_message(message) File “/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/nlu/model.py”, line 470, in featurize_message component.process(message, **self.context) File “/home/galym/Nrasa/printer.py”, line 88, in process print_message(message) File “/home/galym/Nrasa/printer.py”, line 45, in print_message features[“intent”] = {k: v for k, v in features[“intent_ranking”].items() if “id” != k} KeyError: ‘intent_ranking’

I just ran it locally and it seems to work fine. No need to switch intent to intent_ranking.

Just to check, could you show me the conversation that occurs just before you see the error?

1 Like

sure here it is, I did everything again from scratch:

(v) [galym@mx]$ 
(v) [galym@mx]$ 
(v) [galym@mx]$ rasa train
2021-02-26 21:49:59.295813: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-02-26 21:49:59.295852: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-02-26 21:50:00.606995: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-02-26 21:50:00.607025: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2021-02-26 21:50:00.607042: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (mx): /proc/driver/nvidia/version does not exist
2021-02-26 21:50:01 INFO     rasa.model  - Data (core-config) for Core model section changed.
2021-02-26 21:50:01 INFO     rasa.model  - Data (nlu-config) for NLU model section changed.
2021-02-26 21:50:01 INFO     rasa.model  - Data (core-config) for Core model section changed.
2021-02-26 21:50:01 INFO     rasa.model  - Data (nlu-config) for NLU model section changed.
Training NLU model...
/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/utils/train_utils.py:437: UserWarning: constrain_similarities is set to `False`. It is recommended to set it to `True` when using cross-entropy loss. It will be set to `True` by default, Rasa Open Source 3.0.0 onwards.
  category=UserWarning,
/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/utils/train_utils.py:410: UserWarning: model_confidence is set to `softmax`. It is recommended to set it to `cosine`. It will be set to `cosine` by default, Rasa Open Source 3.0.0 onwards.
  category=UserWarning,
2021-02-26 21:50:01 INFO     rasa.shared.nlu.training_data.training_data  - Training data stats:
2021-02-26 21:50:01 INFO     rasa.shared.nlu.training_data.training_data  - Number of intent examples: 69 (7 distinct intents)

2021-02-26 21:50:01 INFO     rasa.shared.nlu.training_data.training_data  -   Found intents: 'affirm', 'greet', 'deny', 'goodbye', 'mood_great', 'mood_unhappy', 'bot_challenge'
2021-02-26 21:50:01 INFO     rasa.shared.nlu.training_data.training_data  - Number of response examples: 0 (0 distinct responses)
2021-02-26 21:50:01 INFO     rasa.shared.nlu.training_data.training_data  - Number of entity examples: 0 (0 distinct entities)
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component WhitespaceTokenizer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component Printer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component CountVectorsFeaturizer
2021-02-26 21:50:01 INFO     rasa.nlu.featurizers.sparse_featurizer.count_vectors_featurizer  - 80 vocabulary slots consumed out of 1080 slots configured for text attribute.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component Printer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component CountVectorsFeaturizer
2021-02-26 21:50:01 INFO     rasa.nlu.featurizers.sparse_featurizer.count_vectors_featurizer  - 697 vocabulary slots consumed out of 1697 slots configured for text attribute.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component Printer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component LexicalSyntacticFeaturizer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component Printer
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:01 INFO     rasa.nlu.model  - Starting to train component DIETClassifier
2021-02-26 21:50:01.788201: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2691275000 Hz
2021-02-26 21:50:01.788915: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x61b31a0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-02-26 21:50:01.788932: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Epochs: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00,  6.16it/s, t_loss=4.652, i_acc=0.739]
2021-02-26 21:50:12 INFO     rasa.utils.tensorflow.models  - Finished training.
2021-02-26 21:50:12 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:12 INFO     rasa.nlu.model  - Starting to train component Printer
2021-02-26 21:50:12 INFO     rasa.nlu.model  - Finished training component.
2021-02-26 21:50:13 INFO     rasa.nlu.model  - Successfully saved model into '/tmp/tmp1qver3m5/nlu'
NLU model training completed.
Training Core model...
/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/mapping_policy.py:52: FutureWarning: 'MappingPolicy' is deprecated and will be removed in the future. It is recommended to use the 'RulePolicy' instead. (will be removed in 3.0.0)
  docs=DOCS_URL_MIGRATION_GUIDE,
Processed story blocks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2034.10it/s, # trackers=1]
Processed story blocks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 1036.06it/s, # trackers=3]
Processed story blocks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 265.17it/s, # trackers=12]
Processed story blocks: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 65.56it/s, # trackers=39]
Processed rules: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2854.24it/s, # trackers=1]
/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/shared/utils/io.py:93: UserWarning: Found rule-based training data but no policy supporting rule-based data. Please add `RulePolicy` or another rule-supporting policy to the `policies` section in `config.yml`.
  More info at https://rasa.com/docs/rasa/rules
Processed trackers: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 912.93it/s, # actions=12]
Processed actions: 12it [00:00, 10303.31it/s, # examples=12]
Processed trackers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 120/120 [00:00<00:00, 234.65it/s, # actions=441]
╔═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║                                                                                                 after tokenizer                                                                                                 ║
╚═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
Traceback (most recent call last):
  File "/home/galym/Nrasa/v/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/__main__.py", line 116, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py", line 58, in <lambda>
    train_parser.set_defaults(func=lambda args: train(args, can_exit=True))
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/cli/train.py", line 102, in train
    finetuning_epoch_fraction=args.epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 109, in train
    loop,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/utils/common.py", line 308, in run_in_loop
    result = loop.run_until_complete(f)
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 174, in train_async
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 353, in _train_async_internal
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 415, in _do_training
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/train.py", line 610, in _train_core_with_validated_data
    model_to_finetune=model_to_finetune,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/train.py", line 70, in train
    agent.train(training_data, **additional_arguments)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/agent.py", line 726, in train
    training_trackers, self.domain, interpreter=self.interpreter, **kwargs
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ensemble.py", line 201, in train
    trackers_to_train, domain, interpreter=interpreter, **kwargs
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 500, in train
    **kwargs,
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/policies/policy.py", line 183, in featurize_for_training
    training_trackers, domain, interpreter, bilou_tagging
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 209, in featurize_trackers
    tracker_state_features = self._featurize_states(trackers_as_states, interpreter)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 76, in _featurize_states
    for tracker_states in trackers_as_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 76, in <listcomp>
    for tracker_states in trackers_as_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 74, in <listcomp>
    for state in tracker_states
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py", line 284, in encode_state
    self._extract_state_features(sub_state, interpreter, sparse=True)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/featurizers/single_state_featurizer.py", line 246, in _extract_state_features
    parsed_message = interpreter.featurize_message(message)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/core/interpreter.py", line 158, in featurize_message
    result = self.interpreter.featurize_message(message)
  File "/home/galym/Nrasa/v/lib/python3.7/site-packages/rasa/nlu/model.py", line 470, in featurize_message
    component.process(message, **self.context)
  File "/home/galym/Nrasa/printer.py", line 88, in process
    print_message(message)
  File "/home/galym/Nrasa/printer.py", line 45, in print_message
    features["intent"] = {k: v for k, v in features["intent"].items() if "id" != k}
AttributeError: 'str' object has no attribute 'items'

Should I change the pipeline sequence?

So it’s happening during training? Interesting. :thinking: Let me try that on my machine.

Could you try training just the nlu part? rasa train nlu

You can check the shell after via rasa shell nlu. It seems both ways of training run fine on my machine but it might help us debug what is happening on yours. At the moment, I’m not able to reproduce this.

1 Like

Thanks a lot now it’s fine :slight_smile:

Thank you very much @koaning for helping with printer.py. It’s unbelivable and works :slight_smile: Would you please help me with this second file, it’s super easy for you, that is it :slight_smile: I will digest it after that slowly to put my own model in DL sentiment analysis file.

If I put this nltk sentiment.py file after tokenizer for example?

language: en

pipeline:

  • name: WhitespaceTokenizer
  • name: printer.Printer alias: after tokenizer
  • sentiment.SentimentAnalyzer--------------------------------
  • name: CountVectorsFeaturizer
  • name: printer.Printer alias: after 1st cv
  • name: CountVectorsFeaturizer analyzer: char_wb min_ngram: 1 max_ngram: 4
  • name: printer.Printer alias: after 2nd cv
  • name: LexicalSyntacticFeaturizer
  • name: printer.Printer alias: after lexical syntactic featurizer
  • name: DIETClassifier epochs: 20
  • name: printer.Printer alias: after diet classifier

policies:

  • name: MemoizationPolicy
  • name: TEDPolicy
  • name: MappingPolicy

this file

from rasa.nlu.components import Component
from rasa.nlu import utils
from rasa.nlu.model import Metadata

import nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
import os

class SentimentAnalyzer(Component):
    """A pre-trained sentiment component"""

    name = "sentiment"
    provides = ["entities"]
    requires = []
    defaults = {}
    language_list = ["en"]

    def __init__(self, component_config=None):
        super(SentimentAnalyzer, self).__init__(component_config)

    def train(self, training_data, cfg, **kwargs):
        """Not needed, because the the model is pretrained"""
        pass



    def convert_to_rasa(self, value, confidence):
        """Convert model output into the Rasa NLU compatible output format."""

        entity = {"value": value,
                  "confidence": confidence,
                  "entity": "sentiment",
                  "extractor": "sentiment_extractor"}

        return entity

    def process(self, message, **kwargs):
        """Retrieve the text message, pass it to the classifier
            and append the prediction results to the message class."""

        sid = SentimentIntensityAnalyzer()
        res = sid.polarity_scores(message.text)
        key, value = max(res.items(), key=lambda x: x[1])

        entity = self.convert_to_rasa(key, value)

        message.set("entities", [entity], add_to_output=True)

    def persist(self, file_name, model_dir):
        """Pass because a pre-trained model is already persisted"""

        pass

Thanks a lot @koaning anyway, if you can’t help with this one but it will be super great!

Just so I understand, you’re trying to attach sentiment as an entity? The sentiment of a sentence isn’t really an entity .