How to handle punctuation and symbol in rasa?

Rasa version - 1.3.7

pipeline: “supervised_embeddings”

Hi All,

I have trained the bot with no punctuation in intent like.

## intent: ask_holiday_in_a_year

    How many holidays do we have in a year?

If i ask below question to bot

  1. How many holidays do we have in a year? - ( NLU is able to recognize it correctly).

  2. How, many ()? Holidays!!,do!@#we have$%^ in a %^& year. - (NLU is able to recognize it correctly.)

  3. How many ###################### holidays do we have in a year? .(NLU is not able to recognize it correctly.)

  4. How many ####### holidays %%^&*$$% do we have in a year? .(NLU is not able to recognize it correctly.)

For cases 1 and 2, it worked but for case 3 and 4, it didn’t work? Is there any way(adding some settings in the pipeline)i can handle these symbols and punctuation and give the expected result?

You could delete the symbols at the beginning of the pipeline with a custom nlu component. In the third parameter of the maketrans function, add the symbols you want to delete.

from rasa.nlu.components import Component
import typing
from typing import Any, Optional, Text, Dict

if typing.TYPE_CHECKING:
    from rasa.nlu.model import Metadata


    class DeleteSymbols(Component):

        provides = ["text"]
        #requires = []
        defaults = {}
        language_list = None

        def __init__(self, component_config=None):
            super(DeleteSymbols, self).__init__(component_config)

        def train(self, training_data, cfg, **kwargs):
            pass

        def process(self, message, **kwargs):
            mt =  message.text
            message.text = mt.translate(mt.maketrans('', '', '$%&(){}^'))

        def persist(self, file_name: Text, model_dir: Text) -> Optional[Dict[Text, Any]]:
            pass

        @classmethod
        def load(
            cls,
            meta: Dict[Text, Any],
            model_dir: Optional[Text] = None,
            model_metadata: Optional["Metadata"] = None,
            cached_component: Optional["Component"] = None,
            **kwargs: Any
        ) -> "Component":
            """Load this component from file."""

            if cached_component:
                return cached_component
            else:
                return cls(meta)
1 Like

@Gehova thanks a lot for providing the code and it worked.