Remove punctuation like ?,!, in message

I have two questions.

  1. Are those operators like ? removed for parsing NLU?
  2. You see that the operators are separate tokens. How can I remove then from the user message. So, I would like to have that information? and information is the same. Former are two tokens and I have a custom handling for one token messages. Can I do that in the message preprocessor? And how?

Thanks in advance!

1 Like

Hello guys,

I have also the same question. How we can remove punctuation from training set?

same question. how to remove/tokenize self-defined separator?