# NLU and class impallance **URL:** \ **Category:** Rasa Open Source\ **Created:** [March 7, 2021, 11:36am UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972 "2021-03-07T11:36:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![petasis](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@petasis](https://forum.rasa.com/u/petasis)\ **Post date:** [March 7, 2021, 11:36am UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972/1 "2021-03-07T11:36:14Z")
Hi all, My NLU file is large. Not too many of intents, but intents with thousands of examples. The problem is that intents with few examples are no more detected. the intent “stop” contains the example “stop”, and the intent “deny” contains the example “no”. However, typing “stop” or “no” get a confidence score in intent classification as low as 0.2 (and nlu\_fallback gets activated). How to fix this problem? I think a partial solution is to make some examples like stop or no map directly to an intent. Something like KeywordIntentClassifier, but with the ability to specify which intents to load (and not load everything). The other alternative is to up-sample somehow the examples of the minority classes. ---
**Author:** ![mloubser](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/mloubser/32/13021_2.png) [@mloubser](https://forum.rasa.com/u/mloubser)\ **Post date:** [March 10, 2021, 4:08pm UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972/2 "2021-03-10T16:08:24Z")
The first solution to class imbalance is always to correct it by collecting more data for underrepresented intents. If an intent is there just to be triggered by specific words, you can rather use buttons with hard-coded intents e.g. `payload: /deny`. You can correct some imbalance with sampling techniques or hyperparameter tuning, but an imbalance that large indicates something is wrong with the structure of the data itself. How are you collecting examples for your intents? Do they come from real conversations, or are they synthetic? ---
**Author:** ![petasis](https://avatars.discourse-cdn.com/v4/letter/p/e47c2d/32.png) [@petasis](https://forum.rasa.com/u/petasis)\ **Post date:** [March 12, 2021, 8:20am UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972/3 "2021-03-12T08:20:24Z")
@mloubser The source of imbalance is the FAQ handling. If you have 3000 topics, with 3-6 questions each, what can you do? How many alternatives regarding yes/no/stop can you gather? The questions to the FAQ are synthetic (I wrote them all), and they are handled by the response selector (with a success rate around 80%). ---
**Author:** ![ivanmkc](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/ivanmkc/32/14007_2.png) [@ivanmkc](https://forum.rasa.com/u/ivanmkc)\ **Post date:** [June 1, 2021, 5:59am UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972/4 "2021-06-01T05:59:34Z")
I also have this problem due to an intent having multiple possible values for an entity, all of which are combined with the different ways to phrase the intent, leading to a combinatorial explosion. See [Confusion in Using Entity Synonyms - #2 by ivanmkc](http://forum.rasa.com/t/confusion-in-using-entity-synonyms/3329/2) Eager to hear from Rasa what the answer is. ---
**Author:** ![koaning](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/koaning/32/6626_2.png) [@koaning](https://forum.rasa.com/u/koaning)\ **Post date:** [June 8, 2021, 12:08pm UTC](https://forum.rasa.com/t/nlu-and-class-impallance/40972/5 "2021-06-08T12:08:49Z")
Is there a hierarchy in your FAQ questions? Typically we have the example that “chitchat” is a different set of responses than “FAQ”. In your case it might also be possible to split up the FAQ questions into subgroups. Might that help? In general; getting a score to properly represent certainty is a huge unsolved problem in ML. There’s an algorithm whiteboard video [here](https://www.youtube.com/watch?v=ev1tNXPo3tE&list=PL75e0qA87dlG-za8eLI6t0_Pbxafk-cxb&index=41&ab_channel=RasaRasa) that highlights recent work done from our research team on the topic. We recently introduced some hyperparameters for DIET that might help. There’s also a PyData talk [here](https://youtu.be/Z8MEFI7ZJlA?t=246) that highlights how estimated probabilities are not a great proxy for certainty, which might help explain why it’s a hard problem to get right.