Hello Everyone,
We (myself & @naveensiwas ) have Entity overlapping issue and tried to solve it by “use_entities” functionality in the domain file for a particular intent but it is not working .
Is there any way in RASA where we can define list of entities related to a particular intent so that it doesn’t extract any other entity apart from this list, for that intent ?
We are using RASA 2.8.14, Can anyone please help/suggest us on this issue.
Thanks in Advance.
Upgrade to 3.x and try Mapping Conditions. You can specify not_intent and intent options in the slot mappings.
entities:
- entity_name
slots:
slot_name:
type: any
mappings:
- type: from_entity
entity: entity_name
role: role_name
group: group name
intent: intent_name
not_intent: excluded_intent
Hi @stephens
Thank you so much for your quick response.
Based on your suggestion this is what I understood, please correct me If I am wrong.
Usually I get my entities extracted by RASA using tracker.latest_message[‘entities’] in custom Action, where it will give me all the entities present in user question. If I follow the approach suggested by you then I have to define one slot for each entity by specifying not_intent and intent options in the slot mappings.
If we go as per your suggestion, we will still get the entity overlapping warning because first entities will be extracted and then slots will be filled.
Actually we have two types of questions, let me explain both of them with the help of examples.
static question (which does not contain any entity, it’s same as FAQ question) and this type of training data belongs to intent faq.
dynamic question (which contains some entities) and this type of training data belongs to different intent show_details.
Scenario #1 : When user ask a static question “What is 1141 Code”, then RASA extract “1141 Code” as entity, it should not suppose to extract it as entity, because in this intent training data we have not specify any entity.
Scenario #2 : When user as a dynamic question “What is the value of 1141 code for site Aberdare”, then RASA extract “1141 Code” and “Aberdare” as entity and we form a SQL query based on entity values in custom action to get the response.
Scenario #3 : When two different entities has same value, then rasa shows “overlapping entities” warning.
UserWarning: Parsing of message: 'what are the Node type available in Aberdare' lead to overlapping entities: Node Type of type cdpr_filter_name extracted by RegexEntityExtractor overlaps with Node Type of type cdpr_site_column_name extracted by RegexEntityExtractor. This can lead to unintended filling of slots. Please refer to the documentation section on entity extractors and entities getting extracted multiple times:https://rasa.com/docs/rasa/components#entity-extractors
What we are expecting now is :-
During scenario #1 rasa should not extract any entity, is there any way to deal with this situation ?
During scenario #3 rasa should consider only a particular entity OR list of entities associated with that intent. Is there any way in RASA where we can define list of entities related to a particular intent so that it doesn’t extract any other entity apart from this list for that Intent.
Can you please help us to solve these scenarios, it’s little urgent for us because of our deliverables.
Thanks in Advanced.
You can try ignoring entities for certain intents. This should allow you to not extract any entity in scenario #1. For the intent from scenario #3 you can specify use_entities to only extract entities you wish to extract.
Hi @Balowen,
We have tried this use_entities approach in different-different ways, after doing changes every time in domain.yml file, model has been re-trained and then entity extraction has been tested using rasa shell nlu and none of them has any impact on entity extraction. It’s same as earlier only, no change in output.
We have tried all of approached given below, but none of them worked for us.
use_entities: []
use_entities: false
use_entities: False
use_entities: None
If I look at the docs, I see that RASA supports use_entities: []
Can someone please help/suggest us, what’s wrong we are doing ?
Thanks in Advanced
Good question, I am also very interested in this.
Hi All,
I have created a custom component to handle this entity overlapping issue.
Component Configuration : This module must be placed just after your EntityExtractor as given below. We also need to mention the Intent and their respective Entities in a list form, so it will extract only those entities for that particular Intent.
For Example
intent_1 only entity_1 and entity_2 will be extracted rest everything will be discard.intent_2 no entity will be extracted and everything will be discard, because empty list has been defined for intent_2
language: "en"
pipeline:
- ................
- other components
- ................
- name: RegexEntityExtractor
use_lookup_tables: True
use_regexes: True
- name: "rasa_custom_module.filter_entities.FilterEntities"
entities:
intent_1: ["entity_1", "entity_2"]
intent_2: []
- name: EntitySynonymMapper
FilterEntities Need to create a rasa_custom_module folder in the root directory of your project and place the code given below in .py file, named this file as filter_entities.py
import typing
import logging
from typing import Any, Optional, Text, Dict, List, Type
from rasa.nlu.components import Component
from rasa.shared.nlu.training_data.message import Message
logger = logging.getLogger(__name__)
if typing.TYPE_CHECKING:
from rasa.nlu.model import Metadata
class FilterEntities(Component):
"""A new component to resolve Entity Overlaps issue based on configuration"""
# Which components are required by this component.
# Listed components should appear before the component itself in the pipeline.
@classmethod
def required_components(cls) -> List[Type[Component]]:
"""Specify which components need to be present in the pipeline."""
return []
# Defines the default configuration parameters of a component
# these values can be overwritten in the pipeline configuration
# of the model. The component should choose sensible defaults
# and should be able to create reasonable results with the defaults.
name = "FilterEntities"
defaults = {
"entities": {},
"extractor_names": [
"RegexEntityExtractor"
],
}
# Defines what language(s) this component can handle.
# This attribute is designed for instance method: `can_handle_language`.
# Default value is None which means it can handle all languages.
# This is an important feature for backwards compatibility of components.
supported_language_list = ["en"]
# Defines what language(s) this component can NOT handle.
# This attribute is designed for instance method: `can_handle_language`.
# Default value is None which means it can handle all languages.
# This is an important feature for backwards compatibility of components.
not_supported_language_list = None
def __init__(self, component_config: Optional[Dict[Text, Any]] = None) -> None:
super(FilterEntities, self).__init__(component_config)
def process(self, message: Message, **kwargs: Any) -> None:
"""Process an incoming message.
Get the Intent of user message and filter the entities based on 'Intent and Entities-List' provided in component config
"""
# Check for Intent and Entities in user message
if message.get('intent') and isinstance(message.get('intent'), dict) and message.get('entities') and isinstance(message.get('entities'), list):
# Info message
logger.info("Filtering the Entities based on 'Intent and Entities-List'")
intent = message.get("intent")
message_entities = message.get("entities")
indices_to_remove = []
entities_to_filter = filter(
lambda e: e["extractor"] in self.component_config["extractor_names"],
message_entities,
)
for index, entity in enumerate(entities_to_filter):
if (
intent["name"] in self.component_config["entities"].keys()
and entity["entity"]
not in self.component_config["entities"][intent["name"]]
):
indices_to_remove.append(index)
for i in sorted(indices_to_remove, reverse=True):
del message.get("entities")[i]
@classmethod
def load(
cls,
meta: Dict[Text, Any],
model_dir: Text,
model_metadata: Optional["Metadata"] = None,
cached_component: Optional["FilterEntities"] = None,
**kwargs: Any,
) -> "FilterEntities":
"""Load FilterEntities from file."""
if cached_component:
return cached_component
else:
return cls(meta)