Restrict NLU parsing to a subset of intents

Rasa Open Source
1

Hello @everyone,
I’m here from project Rhasspy, an open source pluggable offline assistant. I’m not a member of the project, but I’m working on a contribution to it and bumped into an issue with Rasa.

Rhasspy uses a pipeline of services to implement the assistant. One step in that pipeline is NLU, which can be configured for using Rasa NLU API.

However, while designing a framework to write “skills” for this assistant, we bumped into the fact that the trained model is just one. Consider this nlu.md:

  ## intent: Skill1:ShutdownServer
  - please shut down the server
  - would you shut down the server
  - go on and shut down the server

  ## intent: Skill1:RestartServer
  - please restart the server
  - would you reboot the server
  - go on and restart the server

  ## intent: Skill1:CancelCommand
  - no
  - never mind
  - forget it

  ## intent: Skill2:CancelCommand
  - nope
  - no
  - never mind you
  - forget that

(I know those intent phrases are shorts in length and number, that’s just an example to prove my point)
When parsing a text with the NLU, I’d like to ask to interpreter to consider only the first two intents or to consider only intents for a specific skill, e.g. Skill2.
The first case is useful when receiving the first question (I can’t say “No” without even asking something before that). The second case is useful for intermediate responses, because e.g. Skill2 is in charge of the current dialogue so only intents from Skill2 should be interpreted.

I was thinking about another field in the /model/parse request body where I could specify a list of intent names to consider when parsing the text:

{
  "text": "please shut down the server",
  "intent_filter": [  // consider only these intents while interpreting
    "Skill1:ShutdownServer",
    "Skill1:RestartServer"
 ]

Am I making any sense here? Is there some other approach I could use? One thing I’d like to stick with is using only the NLU part of Rasa, if that’s possible.

Thanks!!

2

Hi @daniele_athome,

Am I making any sense here?

Your question makes a lot of sense :+1:

The NLU part of Rasa Open Source is stateless. Rasa Open Source uses stories to re-act differently to questions depending on the context. Using stories you can easily handle the described cases, e.g.

Story: User starts with no

  • no
    • utter_that_doesnt_make_sense

I suggest to check out the tutorial and especially the parts related to writing stories: https://rasa.com/docs/rasa/user-guide/rasa-tutorial/#write-your-first-stories

3

Thanks for your answer!

I was afraid of that. I read what you suggested and I ask you know: does the model/parse API endpoint take stories into account? What I mean is, if I define stories correctly, can I achieve what I was talking about? That is, being able to distinguish “starter intents” from “continuation intents” just by asking that API.

What I’m afraid is that Rasa allows only for certain actions being executed based on stories. HTTP API can’t really interact with that (because the API are stateless, much like the NLU itself as you said). So I either do everything (questions-responses) with Rasa or use only the stateless NLU.

So how can the core determine which intents to detect? If the core uses the NLU and the NLU is stateless, how does it know about stories and domains? Let me explain what I mean with an example:

I recently trained the NLU for two intents with identical sentences. When asked to parse via HTTP API, it returned confidence for the first intent 0.8, and confidence for the second intent 0.3. How is that possible? Shouldn’t the confidence be the same (or similar) since they are trained on the exact same data?

I’ll do some experiments in the meantime.

4

. I read what you suggested and I ask you know: does the model/parse API endpoint take stories into account?

No, it doesn’t.

What I’m afraid is that Rasa allows only for certain actions being executed based on stories. HTTP API can’t really interact with that (because the API are stateless, much like the NLU itself as you said).

You can also interact with stories via HTTP. E.g. you could send message to the REST Channel.

I recently trained the NLU for two intents with identical sentences. When asked to parse via HTTP API, it returned confidence for the first intent 0.8, and confidence for the second intent 0.3. How is that possible? Shouldn’t the confidence be the same (or similar) since they are trained on the exact same data?

NLU uses machine learning components. The components start of with arbitrary initial configurations and are trained (and hence the “configuration” of these components gets better) using your NLU training data. I assume that due to the lack of training data, the model just has some natural bias towards one of the intents.

5

Thanks for your answer!

I’m afraid the REST channel is limited to simple bot response messages. Maybe I could do something with a custom connector, but I need to properly study the API to see if it provides enough data (that is, I need NLU-related data such as intents and slots; the bot response message is not important for me).

I’m not an expert on machine learning (actually I know nothing eheh) so that concept wasn’t obvious for me, thanks for explaining!