RASA X doesn't work if there are synonyms in the training data

[Deprecated] Rasa X Community Edition
1

I tried latest versions of Rasa and Rasa X, and i got this error when running RasaX

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such column: True
[SQL: CREATE INDEX message_log_in_training_data_idx ON message_log (in_training_data) WHERE in_training_data is True]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Then after using particular versions of these dependencies I was able to start RASA X in browser (when there are no synonyms in nlu,yml)

rasa: 2.8.2 pip install rasa-x==0.39.3 --extra-index-url https://pypi.rasa.com/simple SQLAlchemy: 1.3.22 sanic-jwt: 1.6.0

And Rasa X wont works if I have Synonyms in my training Data

2

@athulvingt please experiment whilst using rasa open source, if its working on rasa open source then we need to investigate the issue and if its not working even on rasa open source then still we need to understand the root cause of error. rasa shell --debug Thanks.

3

it works fine on rasa open source. I always train the model in Rasa open source and before, as of now I am using Rasa X to share the assistant with testers. Everytime after training a model, I will comment the synonyms section before starting rasaX and then share it to the testers.

4

@athulvingt can you delete the older models and train the model again. @athulvingt can you add manually the same synonyms in Rasa X? and try.

5

I think the problem is with my training data, this is how my synonyms looked like

- synonym: check in
  examples: |
    - checkin
    - check in

I typed the value in examples as well, once I removed value from examples everything works fine.

thank you @nik202 for your time and support

6

@athulvingt share your nlu file or you using synonyms different files or mentioned in nlu.yml file only?

@athulvingt can you please share config file?

@athulvingt please check the training examples for the same.

Ref: Training Data Format | NLU Training Data and this Github Repo: https://github.com/RasaHQ/rasa-demo/blob/main/data/nlu/nlu.yml#L2189

Hope this will help you. But, its strange that synonyms not recognised.

7 
version: "2.0"
language: en_core_web_md
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: RegexEntityExtractor
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    retrieval_intent: faq
  - name: ResponseSelector
    epochs: 100
    retrieval_intent: chitchat
  - name: FallbackClassifier
    threshold: 0.7
  - name: DucklingEntityExtractor
    url: http://duckling.rasa.com:8000
    dimensions:
    - amount-of-money
    - time
    - number
    - phone-number
    - duration
  - name: SpacyNLP
    model: "en_core_web_md"
    case_sensitive: false
  - name: "SpacyEntityExtractor"
    # Note: It is not possible to use the SpacyTokenizer + SpacyFeaturizer in
    #       combination with the WhitespaceTokenizer, and as a result the
    #       PERSON extraction by Spacy is not very robust.
    #       Because of this, the nlu training data is annotated as well, and the
    #       DIETClassifier will also extract PERSON entities .
    dimensions: ["PERSON"]
policies:
- name: AugmentedMemoizationPolicy
- name: TEDPolicy
  epochs: 40
- name: RulePolicy
  core_fallback_threshold: 0.4
  core_fallback_action_name: "action_default_fallback"
  enable_fallback_prediction: True

my nlu.yml file is bit long, it might make this page messy. What i meant is that the below synonym doesn’t works because “check in” is the value and example as well, if I remove “check in” from synonym it works.

  • synonym: check in examples: |
    • checkin
    • chekin in

I tired adding synonyms in Rasa X, there also if I add an example that is same as value, it wont be shown after I save it, thats how i figured it.

8

I went through the link you shared and i saw a synonym like this

- synonym: core
  examples: |
    - CORE
    - Core
    - core
    - dialogue management
    - Dialogue management
    - Dialogue Management
    - DIALOGUE MANAGEMENT

In my system RasaX wont work unless i remove the third example

9

@athulvingt third example, what you mean.

10

Here synonym values is “core”, and the example as as follows

  1. CORE
  2. Core
  3. core # this is what I meant by third example(.it is same as the synonym value)
  4. dialogue management
  5. Dialogue management
  6. Dialogue Management
  7. DIALOGUE MANAGEMENT
11

@athulvingt I guess, it will not take the same synonyms name as mentioned in example, and I guess it’s also not synonyms.

12

Hi @athulvingt are you able to solve this issue Athul?

13

Yeah I solved it

14

@athulvingt Can I request to please share the solution, when ever you got time and close this thread for others. Thanks.

16

@athulvingt What do you mean by duplicates means you using only single examples like core, dialogue management ; doesn’t matter if its case sensitive right?

17

all synonyms are mapped into a single word, in the above example, Core, Core, core, dialogue management, etc are mapped into “core”, If you look carefully the third word ‘core’ is same as the word it has to be mapped, that was making the problem in my case

Remove the third entry from the examples , It should work

18

@athulvingt I guess that what I suggested to you? :thinking: RASA X doesn't work if there are synonyms in the training data - #11 by nik202

@athulvingt Thanks for the share !

19

yeah , right

20

@athulvingt Please close this thread as solution for other :wink:

21

I am unable to tag a comment solution and to close this thread