How to Create a Complex Lookup Table with Two Dimensions, where multiple strings can map to the same value of an entity

Rasa Open Source
1

I understand that a lookup table contains one column, which defines the multiple values an entity can take.

I want to write a custom component/piece of code to simulate a kind of two-dimentional lookup.

Here is an example. I have an entity, “TVChannel” which can take multiple values.

{ name: 'Consumer News & Business Channel', list: ['CNBC', 'Consumer News & Business Channel', 'Consumer News and Business Channel'] },

  { name: 'Consumer News & Business Channel World HDTV', list: ['CNBCWO', 'CNBC World', 'Consumer News and Business Channel World', 'Consumer News & Business Channel World'] },

{ name: 'Hi-Def On Demand', list: ['HDOD', 'Hi Def On Demand', 'On Demand HD'] }
{ name: 'HGTV On Demand', list: ['HGOD', 'HGTV On Demand', 'Home and Garden TV On Demand'] }

In here, the “name” field is the value the entity should take, but I have a list of input string/values which should all map to the same entity value.

I understand that Entity Synonyms are useful for this purpose, but it doesn’t seem very feasible for me, as I have a huge amount of data, the channel list itself has >400 channels, each with a few possible iterations of their names.

I also have a significant number of other entities which also require similar two dimentional recognition.

The Entity Synonym Approach doesn’t seem scalable to me for such a large amount of data.

Is there a way I can write a custom component/piece of code that can perform this 2D Lookup for me? Could anyone point me in the right direction as to where it would fit in and if it even possible?

Or maybe is there a way to modify the existing entity synonym mechanism such that this can be done?

Thanks Shubham

2

Hi @shubhamnatraj,

interesting problem! Could you use a custom action to map your detected entities? What’s nice about this, is that you also don’t have to bake the mapping into the model, but can manage it from the outside (e.g. supply an updated list to the action).

3

Hi @Tobias_Wochinger,

Thanks for the response! I am currently just using NLU. Not using any part of Core, and in my broader use case, there’s no reason to use core.

Is there something I could do by adding a custom NLU component somewhere? Since I have multiple such entities, maybe provide the mappings for all of them in a component?

4

Yes, a custom NLU component (maybe extending the existing ones for Synonyms) sounds like a good idea!