# Rasa not picking special characters in an entity

**URL:** <https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169>\
**Category:** Rasa Open Source\
**Created:** [January 30, 2020, 1:21pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169 "2020-01-30T13:21:47Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![shubham1140](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@shubham1140](https://forum.rasa.com/u/shubham1140)\
**Post date:** [January 30, 2020, 1:21pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/1 "2020-01-30T13:21:47Z")

</div>

hi team,

Rasa nlu is unable to pic special characters in an entity such as ( alfred,rodger , europe/london ). can u help me with this@akelad  
Thanks

---

<div class="post-metadata">

**Author:** ![Tanja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/tanja/32/4696_2.png) [@Tanja](https://forum.rasa.com/u/Tanja)\
**Post date:** [February 3, 2020, 9:26am UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/2 "2020-02-03T09:26:11Z")

</div>

Hi @shubham1140!

> ( alfred,rodger , europe/london )

What special characters are you referring to? Can you point me to them?

---

<div class="post-metadata">

**Author:** ![nadachaabani1](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/nadachaabani1/32/5919_2.png) [@nadachaabani1](https://forum.rasa.com/u/nadachaabani1)\
**Post date:** [February 3, 2020, 10:17am UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/3 "2020-02-03T10:17:52Z")

</div>

i have the same problem with special characters such as `.` in `.net` and `+` in `c++`

```auto
- Knowledge in [C++](competency), [Python](competency), [Linux](competency) and [GIT](competency)
- i am good with [c#](competency) , [.net](competency) and [react](competency)

```

```auto
 f"Misaligned entity annotation for '{collected_text}' "
c:\users\mega\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\extractors\crf_entity_extractor.py:533: UserWarning: Misaligned entity annotation for 'C' in sentence 'Knowledge 
in C++, Python, Linux and GIT, administration tools' with intent 'inform'. Make sure the start and end values of the annotated training examples end at token boundaries (e.g. don't include trailing whitespaces or punctuation).

```

```auto
  f"Misaligned entity annotation for '{collected_text}' "
c:\users\mega\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\extractors\crf_entity_extractor.py:533: UserWarning: Misaligned entity annotation for 'net' in sentence 'i am good with c# , .net and react' with intent 'inform'. Make sure the start and end values of the annotated training examples end at token boundaries (e.g. don't include trailing whitespaces or punctuation).

```

---

<div class="post-metadata">

**Author:** ![ganeshv](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ganeshv/32/4644_2.png) [@ganeshv](https://forum.rasa.com/u/ganeshv)\
**Post date:** [February 3, 2020, 10:58am UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/4 "2020-02-03T10:58:40Z")

</div>

Hello @nadachaabani1, @shubham - there’s a potential solution for you here - [Having trouble formatting training examples that contains a '-' or other punctuation signs](http://forum.rasa.com/t/having-trouble-formatting-training-examples-that-contains-a-or-other-punctuation-signs/23957/4).

tl;dr - there’s a regex that in typical cases ignores special characters as delimiters in strings. So cases where I would like the string `75-100` to be extracted into two entities `75` and `100`, would fail. A solution would be to modify the regex to your specific need.

---

<div class="post-metadata">

**Author:** ![shubham1140](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@shubham1140](https://forum.rasa.com/u/shubham1140)\
**Post date:** [February 3, 2020, 12:57pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/5 "2020-02-03T12:57:28Z")

</div>

> [@ganeshv](#):
>
> xtract

the issue with me is that i need to consider (/, ‘,’) as a part of single entity not as a separators. for example (city/state is my single entity ) and the problem is rasa nlu is not picking this entity due to (/) present in this entity. @ganeshv , @akelad

---

<div class="post-metadata">

**Author:** ![shubham1140](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@shubham1140](https://forum.rasa.com/u/shubham1140)\
**Post date:** [February 3, 2020, 12:59pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/6 "2020-02-03T12:59:35Z")

</div>

@Tanja special characters like ( ‘,’ , ‘/’ ) etc in an entity values are not picked by rasa nlu . other example is if i write (doesn’t) , this is also not picked although trained on this

---

<div class="post-metadata">

**Author:** ![Tanja](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/tanja/32/4696_2.png) [@Tanja](https://forum.rasa.com/u/Tanja)\
**Post date:** [February 3, 2020, 6:42pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/7 "2020-02-03T18:42:19Z")

</div>

As already pointed out by @ganeshv, we have a regex in place that splits words on those characters into separate tokens. So if you are using the `WhitespaceTokenizer` this will happen. If you want to keep the words, you can first of all try a different tokenizer or update the regex by writing a custom tokenizer (you can use the `WhitespaceTokenizer` as an example and just update the regex over there).

---

<div class="post-metadata">

**Author:** ![bayesianwannabe](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@bayesianwannabe](https://forum.rasa.com/u/bayesianwannabe)\
**Post date:** [March 20, 2020, 10:18pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/8 "2020-03-20T22:18:16Z")

</div>

Hey shubham, how are you?

I am having the same issue… have you solved it? I wonder which tokenizer you are using… I am working with SpacyTokenizer but, even if the vanilla lib keeps stuff like 05/02 together, it seems that a different treatment is adopted on this component.

---

<div class="post-metadata">

**Author:** ![samscudder](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/samscudder/32/4466_2.png) [@samscudder](https://forum.rasa.com/u/samscudder)\
**Post date:** [April 1, 2020, 1:48pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/9 "2020-04-01T13:48:08Z")

</div>

We use spacytokenizer and I’m not having any problems with “-” or “.” in examples.

---

<div class="post-metadata">

**Author:** ![sibbsnb](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/sibbsnb/32/2285_2.png) [@sibbsnb](https://forum.rasa.com/u/sibbsnb)\
**Post date:** [May 12, 2020, 10:52pm UTC](https://forum.rasa.com/t/rasa-not-picking-special-characters-in-an-entity/24169/10 "2020-05-12T22:52:43Z")

</div>

yeah spacy solved my issue here to detect 1 charactor symbol
