How to Set confidence level threshold value for rasa core and nlu?

Hello All, I am running 2 docker containers for core and nlu. I am trying to set confidence level threshold value for both core and nlu using the below steps … \n

  1. created a config.yml file
    policies:
  • name: “FallbackPolicy”
    nlu_threshold: 0.9
    core_threshold: 0.9
    fallback_action_name: ‘utter_default’

config.yml (129 Bytes)

  1. Utter_default is an action created in domain.yml file actions:
  • utter_default
  • utter_greet

templates:
utter_greet:
- text: “hello ,how can I help you ?”
- text: "hey "
utter_default:
- “I did not understand.”

  1. in docker file am trying to train and run core using the below commands\

FROM python:3.6
WORKDIR /app

run pip install rasa_core

EXPOSE 5005

COPY stories.md /app COPY domain.yml /app COPY endpoints.yml /app COPY actions.py /app COPY config.yml /app RUN python -m rasa_core.train -d domain.yml -s stories.md -o models/dialogue

CMD python -m rasa_core.run -d models/dialogue -u sample/nlu --endpoints endpoints.yml --enable_api -c rest --cors ‘*’

After building and running core and nlu container , am trying to hit the below core endpoint endpoint = http://localhost:5005/conversations/1234/respond
body = {
“q” : “hi”
}
response = [
{ “recipient_id”: “1234”, “text”: “I did not understand.” } ]

But my expectation is ,it should perform utter_greet action as am getting the confidence level of the same message is more than 0.9 from nlu.

Can someone please help me on this .

My requirement is to perform utter_default action only when the confidence level is below 90%.

What’s the content of your endpoints file? And how are you running the docker containers?

1 Like

nlu:
url: “http://172.19.0.2:5000
endpoints.yml (39 Bytes)

I am running 2 docker containers (core and nlu ) separately and then am giving the nlu ip in endpoints.yml file. Its working fine if I don’t give the config file . But my requirement is to give a confidence threshold value. Let me know if you need all files of core and nlu ,so that you can simulate the same in your local.

Do you expose the NLU port? And have you seen the guide here: Building Rasa with Docker ?

1 Like

I have done it by following the docs only. Its working perfectly without giving the config.yml file . Now I have mentioned the core and nlu confidence threshold value in the config file.

After trained core with the config file its always performing the default action. When am checking the same message (ex- hi) by directly hitting NLU , its returning 0.96 confidence level, which is more than the threshold value I have given . My expectation is it should perform the greet action instead of default action.

What are your Core policies?