# Utterance Duplication

**URL:** <https://forum.rasa.com/t/utterance-duplication/14729>\
**Category:** Rasa Open Source\
**Created:** [August 23, 2019, 2:52pm UTC](https://forum.rasa.com/t/utterance-duplication/14729 "2019-08-23T14:52:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![serdar](https://avatars.discourse-cdn.com/v4/letter/s/d26b3c/32.png) [@serdar](https://forum.rasa.com/u/serdar)\
**Post date:** [August 23, 2019, 2:52pm UTC](https://forum.rasa.com/t/utterance-duplication/14729/1 "2019-08-23T14:52:05Z")

</div>

Hello,

Is there anyway to check utterance duplication in training set? Especially working on large training sets and similar utterances to each other, it might be an issue.

Thanks

---

<div class="post-metadata">

**Author:** ![dakshvar22](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/dakshvar22/32/3993_2.png) [@dakshvar22](https://forum.rasa.com/u/dakshvar22)\
**Post date:** [August 27, 2019, 12:23pm UTC](https://forum.rasa.com/t/utterance-duplication/14729/2 "2019-08-27T12:23:14Z")

</div>

@serdar Hi, do you mean checking for duplicate utterance with different intents in training data? If they are assigned to the same intent, duplicate examples would be filtered out.

---

<div class="post-metadata">

**Author:** ![serdar](https://avatars.discourse-cdn.com/v4/letter/s/d26b3c/32.png) [@serdar](https://forum.rasa.com/u/serdar)\
**Post date:** [September 8, 2019, 9:59pm UTC](https://forum.rasa.com/t/utterance-duplication/14729/3 "2019-09-08T21:59:15Z")

</div>

Hi @dakshvar22, I mean the utterance might fall under the different intents. Is there anyway to check it? Thanks

---

<div class="post-metadata">

**Author:** ![dakshvar22](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/dakshvar22/32/3993_2.png) [@dakshvar22](https://forum.rasa.com/u/dakshvar22)\
**Post date:** [September 10, 2019, 9:59am UTC](https://forum.rasa.com/t/utterance-duplication/14729/4 "2019-09-10T09:59:04Z")

</div>

No, that wouldn’t be flagged. I think it’s a good idea to include it. Would you be up to open a feature request on our [GH repo](https://github.com/RasaHQ/rasa)?

---

<div class="post-metadata">

**Author:** ![serdar](https://avatars.discourse-cdn.com/v4/letter/s/d26b3c/32.png) [@serdar](https://forum.rasa.com/u/serdar)\
**Post date:** [October 4, 2019, 4:26pm UTC](https://forum.rasa.com/t/utterance-duplication/14729/5 "2019-10-04T16:26:08Z")

</div>

Hi @dakshvar22, I open a feature request on Git as you suggest. Thanks