Hello Community, I am using French model and i have remarked that there is a kind of confusion between prediction in sentence with and without stop_words. I dont understand the cause of this confusion. Below more explication. I have added stop_words for “CountVectorsFeaturizer” stop_words are: [‘est’,'ce,‘que’,‘tu’] I have then made two intents tries: - est-ce que tu peux m'aider --> gives intent_cant_help - tu peux m'aider --> gives intent_bot_notice - aider --> gives intent_bot_notice I cannot understand this difference as we normally remove all stop words in the pipeline before prediction. For mor details. below is my configuration language: fr pipeline: - name: WhitespaceTokenizer token_pattern: (?u)\b\w+\b - name: CRFEntityExtractor - name: EntitySynonymMapper - name: CountVectorsFeaturizer analyzer: "word" stop_words: ['je', 'veux', 'souhaite', 'savoir', 'voudrais', 'il', 'elle', 'aimerai', 'aimerais', 'devrais', 'pourrais', 'vais', 'aime','alors','au','aucuns','aussi','autre','avant','avec','avoir','bon','car','ce','cela','ces','ceux','chaque','ci','comme','comment','dans','des','du','dedans','dehors','depuis','devrait','doit','donc','dos','début','elles','en','encore','essai','est','et','eu','fait','faites','fois','font','hors','ici','ils','juste','la','le','les','leurs','là','ma','maintenant','mais','mes','mien','moins','mon','même','ni','notre','nous','ou','où','par','parce','pas','peut','peu','plupart','pour','pourquoi','quand','que','quel','quelle','quels','quelles','qui','sa','sans','ses','seulement','si','sien','sont','son','sous','soyez','sur','ta','tandis','tellement','tels','tes','ton','tous','tout','trop','très','tu','voient','vont','votre','vous','vu','ça','étaient','été','être','a', 'à', 'pouvez', 'suis', '!', '?', '.', ':','au', 'aux', 'avec', 'ce', 'ces', 'dans', 'de', 'des', 'du', 'elle', 'en', 'et', 'eux', 'il', 'ils', 'je', 'la', 'le', 'les', 'leur', 'lui', 'ma', 'mais', 'me', 'même', 'mes', 'moi', 'mon', 'ne', 'nos', 'notre', 'nous', 'on', 'ou', 'par', 'pas', 'pour', 'qu', 'que', 'qui', 'sa', 'se', 'ses', 'son', 'sur', 'ta', 'te', 'tes', 'toi', 'ton', 'tu', 'un', 'une', 'vos', 'votre', 'vous', 'c', 'd', 'j', 'l', 'à', 'm', 'n', 's', 't', 'y', 'été', 'étée', 'étées', 'étés', 'étant', 'étante', 'étants', 'étantes', 'suis', 'es', 'est', 'sommes', 'êtes', 'sont', 'serai', 'seras', 'sera', 'serons', 'serez', 'seront', 'serais', 'serait', 'serions', 'seriez', 'seraient', 'étais', 'était', 'étions', 'étiez', 'étaient', 'fus', 'fut', 'fûmes', 'fûtes', 'furent', 'sois', 'soit', 'soyons', 'soyez', 'soient', 'fusse', 'fusses', 'fût', 'fussions', 'fussiez', 'fussent', 'ayant', 'ayante', 'ayantes', 'ayants', 'eu', 'eue', 'eues', 'eus', 'ai', 'as', 'avons', 'avez', 'ont', 'aurai', 'auras', 'aura', 'aurons', 'aurez', 'auront', 'aurais', 'aurait', 'aurions', 'auriez', 'auraient', 'avais', 'avait', 'avions', 'aviez', 'avaient', 'eut', 'eûmes', 'eûtes', 'eurent', 'aie', 'aies', 'ait', 'ayons', 'ayez', 'aient'…

[image] Asmacats: RANDOM_SEED: 7777777 Please try setting random_seed instead of RANDOM_SEED Also consider setting “est-ce” as a stopword.

Confusion between intent with and without stop_words

Asmacats (Asma Trabelsi) March 1, 2022, 10:53am 3

hello @ChrisRahme changes have not impact prediction. Any other solutions please

Topic		Replies	Views
Prediction confusion Rasa Open Source	6	691	February 25, 2022
Stop words not working properly (?) Rasa Open Source	3	840	February 24, 2022
Question about rasa ponctuation Rasa Open Source	0	172	February 10, 2022
Rasa stopwords issue Rasa Open Source	3	1473	November 11, 2019
Rasa detect random meaningless sequence of character as intent with high confidence Rasa Open Source	2	1047	December 10, 2019