Rasa NLU classification in intent subsets

Rasa Open Source
1

Hello!

My question is related to Restrict NLU parsing to a subset of intents.

Rasa NLU is stateless but I wonder if that makes impossible to select a subset of the intents used in training to classify users input. I’m not an expert on machine learning so I fall short on how Rasa models are trained and used for queries.

I think it would be a great feature to include a context tag to intent training data, something like:

## intent: general:greet
- Hello
- Hi there

## intent: general:service_inquery
- What can you do for me?
- What services you provide?

## intent: flightbooking:book
- I want to book a flight
- Can you help me book a flight?

## intent: flightbooking:refund
- Can you help me refund a ticket
- I need to refund a flight

And add a parameter to the API so we can select contexts for evaluation:

{
  "text": "I need to book a flight",
  "context": "flightbooking"
}

I’m considering using RASA NLU for a an existing project that uses a NLU module based on string matching and regular expressions. I already refactored that module in something that allows me to plugin and compare NLU modules and I would love to try Rasa and assess the results. I plan to build a docker container and use the Rasa NLU Api, to do intent matching. Response generation is already implemented within the project.

What you recommend for this?

Is it possible to use different trained model within a single Rasa installation so I could have a process that would build a different model for each context?

Thanks for your help!

2

This isn’t something that Rasa offers out of the box but I’m curious to hear more about your use case. You seem to want to pass a context along as input.

One question about that; is this context known upfront or does it need to be inferred from the conversation? Technically we have systems, like TED, that infer context from the conversation implicitly to predict the next action. It sounds like you want to have the context be predicted explicitly in order to use it to predict the intent.

1 Like
3

Hi Vincent,

The context would be known upfront, and the training examples would be optionally annotated with a context. This would provide a conditioning option to perform NLU upon a contextualized set of intents.

For bigger domains we might have intent ambiguity when users say something in response of a bot request, because the real meaning of a sentence may vary according to context. If we capture the context on the first interactions we might improve the intent classification by context-conditioning the next interactions. The human brain is excellent on reasoning because it does a lot of context filtering.

My use case is a bit special as I’m integrating Rasa NLU on an existing chatbot framework, and the only way I can integrate the NLU modules is by tapping into the intent classification phase. This framework provides an high level domain language for creating “conversations” around known objects the application provides. The conversation revolves around the objects and the information those objects store in a database. Since this “objects” are known and well described, the training examples are automatically generated around what is defined the “domain”, which includes all the objects.

An example of a typical conversation anotaded with context information:

[CTX: null]
U: "Hey you! I want a list of scientific papers about on the Biology domain".
B: "Sure thing! I have around 379 papers about that, want me to send you in an email?"

[CTX:paper]
U: "No thanks, but can you find me those with the keywords abc and xyz?"
B: "Yep, there are 9, let me put here the titles. etc etc"

[CTX:paper]
U: "Cool. Show me also the abstracts please?"
B: "Here they are! Etc etc"

U: "Ok now, can you list all conferences for the Biology domain?"
B: There are plenty! Want me to send you in an email?

[CTX:conference]
U: "Nah, but if you could find me those with previous keywords would be sweet!"
B: Sure thing boss!

As you can see there is a lot of context moving around. Some of the context the framework will store and use in if need in later user requests. The keywords abc and xyz are an example of that.

There is also a lot going on on context navigation here both paper or conference “objects” have quasi similar intents for "find me all with keyworks ". Example phrases are generated on a top level context, where the name is needed on the user request, so the chatbot will know what object the query corresponds. Then “object” context bound examples are generated for contextualized queries:

U: Can you read me the abstracts? U: And what about the authors?

I believe NLU inference with context conditioning can be powerful.

Now, for easy solving this for me, would it work if I prefix the training sets with the context information and use that when inferring?

Something like:

// Examples for paper

PAPER PAPER PAPER find me all papers
PAPER PAPER PAPER find me all scientific papers
etc
PAPER PAPER PAPER who are the authors

// Example of conferences
CONFERENCE CONFERENCE CONFERENCE find me all conferences
etc etc

Then when U: asks “who are the authors?” the framework knowing the context of paper could prefix the phrase with CONFERENCE CONFERENCE CONFERENCE before sending in the NLU API.

You think this would work?

Thanks

4

@koaning :wave:

I am wondering what is your take on this.

Thank you!

5

What you’re proposing might work, but it’s not something that Rasa was designed for. The only way to know for sure if it works is by running it. If the context is really known upfront I suppose you could inject it as a user message. Theoretically, it could work. But it does feel like a giant hack because it’ll mess up the tokenizer. Suddenly there will be tokens in a user message that shouldn’t be interpreted as tokens, rather as tags.