Hi guys I am working on building a digital assistant using rasa and I am facing this problem
- While training the response selectors have a very high loss eg 5.6 but the r_acc (accuracy) is 100 . why is this happening ?
Well the intents are classified properly but I want to know if this is a common thing or will it end up creating problems in the future.
Here is my config.yml file
language: en
pipeline:
# No configuration for the NLU pipeline was provided. The following default pipeline was used to train your model.
# If you'd like to customize it, uncomment and adjust the pipeline.
# See https://rasa.com/docs/rasa/tuning-your-model for more information.
- name: WhitespaceTokenizer
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: "CountVectorsFeaturizer"
# Analyzer to use, either 'word', 'char', or 'char_wb'
"analyzer": "word"
# Set the lower and upper boundaries for the n-grams
"min_ngram": 1
"max_ngram": 1
# Set the out-of-vocabulary token
"OOV_token": "_oov_"
# Whether to use a shared vocab
"use_shared_vocab": False
- name: LanguageModelFeaturizer
# Name of the language model to use
model_name: "bert"
# Pre-Trained weights to be loaded
model_weights: "rasa/LaBSE"
cache_dir: null
- name: RegexEntityExtractor
case_sensitive: False
use_lookup_tables: True
- name: "DucklingEntityExtractor"
# url of the running duckling server
url: "http://localhost:8000"
# dimensions to extract
dimensions: ["time", "amount-of-money", "distance","email","phone-number"]
timeout : 3
- name: DIETClassifier
epochs: 150
constrain_similarities: true
# model_confidence: linear_norm
- name: EntitySynonymMapper
- name: ResponseSelector
epochs: 100
retrieval_intent: faq
- name: ResponseSelector
epochs: 100
retrieval_intent: chitchat
- name: ResponseSelector
epochs: 100
retrieval_intent: inform
- name: FallbackClassifier
threshold: 0.55
ambiguity_threshold: 0.05
# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
- name: RulePolicy
- name: MemoizationPolicy
max_history: 3
- name: TEDPolicy
max_history: 5
epochs: 300
constrain_similarities: true
And here is the nlu.yml file containing the response selector utterances I had to move it here instead of domain.yml as it becz of the user warnings
responses:
utter_faq/contact_inquiry:
- text: I am calling you because of the request you have made for insurance quote are you interested ?
- text: The reason I am calling Is because of the request you had submitted online are you interested in looking for a better policy ?
- text: We have received a request for you for a auto insurance quote are you still looking ?
- text: I am calling you in response to the request you had submitted online for a auto quote are you still looking ?
utter_faq/personal_questions:
- text: My name is Reha I am a independent financial agent calling you to get a better quote and saving money are you interested ?
- text: I am Reha hi I work as a independent financial agent calling you to get a better quote and saving money are you interested ?
- text: My name is Reha I am a independent financial agent I work for <company_name> are you interested in looking for a quote ?
- text: Reha,I am a independent financial agent working for <company_name> I can help you get a better quote are you interested ?
utter_faq/email:
- text: yes I will mail you the details should
- text: sure will mail you the all the policy details
- text: ya will mail you the details
- text: ya will email you the detials.
utter_faq/website:
- text: our website is <website link>
utter_faq/ask_question:
- text: Yes please go ahead
- text: yes sure you can
- text: yes go ahead
- text: dont hesitate just ask anything about the insurance
- text: yes sure please go ahead
- text: sure go on
utter_faq/policy_questions:
- text: <respond to policy query>
utter_faq/price_questions:
- text: <utter_price_response>
- text: yes the <utter_price_response>
- text: yes its is the <utter_price_response>
- text: <utter_price_response>
utter_chitchat/ask_howdoing:
- text: I am fine thanks for asking
- text: I am going great thank for asking
- text: fine just doing my work thanks for asking
- text: great thanks for asking
- text: ok thanks for asking
utter_chitchat/ask_isbot:
- text: I am a digital assistant powered by ai
- text: Yes I'm a digital person powered by ai
- text: Yes I am REHA a digital bot powered by Artificial intelligence
- text: The names Reha a digital person to help you find the best quotes.
utter_chitchat/ask_ishuman:
- text: No I am not a human My name is Reha I am a Artificial intelligence lifeform
- text: Not a human a digital assistant created to help you find the best insurance
- text: I am a digital Assistant not a human
utter_inform/customer_basic_information:
- text: <customer_name> to get you the best policy I would like to ask you some details if that ok with you ?
utter_inform/customer_auto_information:
- text: <customer_name> to get you the best policy I would like to ask you some details if that ok with you ?
utter_inform/current_insurer:
- text: <customer_name> to get you the best policy I would like to ask you some details if that ok with you ?
Rasa version
- rasa 2.6.0
- rasa-sdk 2.6.0
- rasa-x 0.0.1
- rasalit 0.1.2
I am also including my domain.yml and nlu.yml file domain.yml (13.4 KB)
nlu.yml (31.8 KB)