Multiple intent detection with slot tagging for each intent

Rasa Open Source
1

I want to use Rasa NLU to accomplish the task of processing utterances with multiple intents and slot tagging the entities related to each intent appropriately.

An example scenario would be -

“turn off the bedroom light and wake me up at 6am”

where the first intent is “turn_off” with an entity, [device] = bedroom light and the second intent is set_alarm with an entity [time] = “6am”

I just need to use it for NLU, and pass the detected information to an IOT service.

I am able to get the intent detection working, but am not able to wrap my head around how to identify the relevant entities/fill slots.

Any ideas would be much appreciated.

Thanks Shubham

2

@shubhamnatraj could you clarify what you’re not sure about specifically? The entities should be extracted automatically

3

@akelad thanks for the response. I was able to extract the entities since then.

The thing I am facing issues with is misclassification of entities and mapping which entity is associated with which intent. I have some similar entities by which I mean there are many entities which are just text strings.

Is there a mechanism by which entity extraction can be correlated with the intent, the entities identified are influenced by which intent is identified (since I know which entities belong with which intent)

Also, I have a couple of small queries regarding training data -

I understand that it takes around 20-25 instances of an entity for the ner_crf extractor to be able to identify an entity. If I have two intents A and B, in the training of which I have provided adequate examples of their required entities. When I train the joint intent for A+B (same entities), do I need a fresh 25 utterances for the entities again or will it be able to do with fewer? Based on the fact that the entities have been given enough examples earlier and we just need enough to train the joint intent properly.

Also, is training multi intents for the same recurring intent possible, like intentA+intentA ? I tried it but it seems to work sometimes and not work sometimes.

Thanks Shubham

4

Here is an example of the misclassification I am talking about.

I have an intent “reminder”

## intent:reminder
- remind me to [submit assignment](reminder_content) at [9 am](reminder_time)
- remind me to [check my mail](reminder_content) at [9 o clock in the morning](reminder_time)
- remind me to [go to school](reminder_content) at [10 o clock in the evening](reminder_time)
- remind me to [have lunch](reminder_content) at [12 o clock](reminder_time)
- remind me to [drink water](reminder_content) at [1900 hrs](reminder_time)

And another intent to set some device values

intent:DeviceSetValueResult

- change the [tv channel](DeviceSetValueResult_CapabilityName:CP_CHANNEL) to [golf channel](DeviceSetValueResult_TvChannelName)
- change the [station](DeviceSetValueResult_CapabilityName:CP_CHANNEL) to [tlc](DeviceSetValueResult_TvChannelName)
- change the [program](DeviceSetValueResult_CapabilityName:CP_CHANNEL) to [amc](DeviceSetValueResult_TvChannelName)
- change the [channel number](DeviceSetValueResult_CapabilityName:CP_CHANNEL) to [fox news](DeviceSetValueResult_TvChannelName)

Here are a few results from when I test the NLU using rasa shell nlu

change the tv channel to fox news
{
  "intent": {
    "name": "DeviceSetValueResult",
    "confidence": 0.9845221042633057
  },
  "entities": [
    {
      "start": 11,
      "end": 21,
      "value": "CP_CHANNEL",
      "entity": "DeviceSetValueResult_CapabilityName",
      "confidence": 0.9042012692848238,
      "extractor": "CRFEntityExtractor",
      "processors": [
        "EntitySynonymMapper"
      ]
    },
    {
      "start": 25,
      "end": 33,
      "value": "fox news",
      "entity": "DeviceSetValueResult_TvChannelName",
      "confidence": 0.8185708036487913,
      "extractor": "CRFEntityExtractor"
    }
  ],
  "intent_ranking": [
    {
      "name": "DeviceSetValueResult",
      "confidence": 0.9845221042633057
    },
    {
      "name": "play_music+turn_on",
      "confidence": 0.00534422229975462
    },
    {
      "name": "goodbye",
      "confidence": 0.002920436905696988
    },
    {
      "name": "play_music",
      "confidence": 0.002064654603600502
    },
    {
      "name": "turn_on",
      "confidence": 0.0013101432705298066
    },
    {
      "name": "mood_great",
      "confidence": 0.0013033980503678322
    },
    {
      "name": "greet",
      "confidence": 0.0010761971352621913
    },
    {
      "name": "deny",
      "confidence": 0.0008673689444549382
    },
    {
      "name": "affirm",
      "confidence": 0.00037158536724746227
    },
    {
      "name": "play_music+reminder",
      "confidence": 0.0002198833244619891
    }
  ],
  "text": "change the tv channel to fox news"
}
Next message:
change the channel number to hbo
{
  "intent": {
    "name": "DeviceSetValueResult",
    "confidence": 0.9459835290908813
  },
  "entities": [
    {
      "start": 11,
      "end": 25,
      "value": "CP_CHANNEL",
      "entity": "DeviceSetValueResult_CapabilityName",
      "confidence": 0.8206810267221036,
      "extractor": "CRFEntityExtractor",
      "processors": [
        "EntitySynonymMapper"
      ]
    },
    {
      "start": 29,
      "end": 32,
      "value": "hbo",
      "entity": "reminder_content",
      "confidence": 0.0,
      "extractor": "CRFEntityExtractor"
    }
  ],
  "intent_ranking": [
    {
      "name": "DeviceSetValueResult",
      "confidence": 0.9459835290908813
    },
    {
      "name": "play_music+turn_on",
      "confidence": 0.015744846314191818
    },
    {
      "name": "goodbye",
      "confidence": 0.009633923880755901
    },
    {
      "name": "play_music",
      "confidence": 0.007139007095247507
    },
    {
      "name": "turn_on",
      "confidence": 0.005111034959554672
    },
    {
      "name": "mood_great",
      "confidence": 0.0051033636555075645
    },
    {
      "name": "greet",
      "confidence": 0.0044085304252803326
    },
    {
      "name": "deny",
      "confidence": 0.0036966227926313877
    },
    {
      "name": "affirm",
      "confidence": 0.0019085295498371124
    },
    {
      "name": "play_music+reminder",
      "confidence": 0.0012706633424386382
    }
  ],
  "text": "change the channel number to hbo"
}
Next message:
change the channel number to hbo plus
{
  "intent": {
    "name": "DeviceSetValueResult",
    "confidence": 0.9323456287384033
  },
  "entities": [
    {
      "start": 11,
      "end": 25,
      "value": "CP_CHANNEL",
      "entity": "DeviceSetValueResult_CapabilityName",
      "confidence": 0.824950568997213,
      "extractor": "CRFEntityExtractor",
      "processors": [
        "EntitySynonymMapper"
      ]
    },
    {
      "start": 29,
      "end": 37,
      "value": "hbo plus",
      "entity": "reminder_content",
      "confidence": 0.45617189246776346,
      "extractor": "CRFEntityExtractor"
    }
  ],
  "intent_ranking": [
    {
      "name": "DeviceSetValueResult",
      "confidence": 0.9323456287384033
    },
    {
      "name": "play_music+turn_on",
      "confidence": 0.019504211843013763
    },
    {
      "name": "goodbye",
      "confidence": 0.01205013133585453
    },
    {
      "name": "play_music",
      "confidence": 0.008962946012616158
    },
    {
      "name": "turn_on",
      "confidence": 0.006410704459995031
    },
    {
      "name": "mood_great",
      "confidence": 0.006364417262375355
    },
    {
      "name": "greet",
      "confidence": 0.0053586713038384914
    },
    {
      "name": "deny",
      "confidence": 0.004654101096093655
    },
    {
      "name": "affirm",
      "confidence": 0.0025136014446616173
    },
    {
      "name": "play_music+reminder",
      "confidence": 0.0018355821957811713
    }
  ],
  "text": "change the channel number to hbo plus"
}

As you can see, the first utterance is correctly parsed, but in the second and third utterances have misclassifications.

How can I improve upon this?

5

Hm, I don’t think this is so much a confusion of entities that belong to different intents, but the fact that the entities themselves can be confusing. The reason for that is probably a lack of training data as of now. E.g. “hbo” has probably never been seen in your training data and therefore the model won’t know what to do with it.

If you have a list of known channels, I would suggest providing that as a lookup table.

Apart from that, not sure which Rasa version you’re on, but as of 1.8.0, we have a new pipeline for entity extraction and intent classification, which could be worth trying out: Choosing a Pipeline

6

I did try a lookup table, ran into an issue with it.

Created another thread for it here. Could you please have a look.