# DIETClassifier does not generalize? **URL:** \ **Category:** Rasa Open Source\ **Created:** [June 16, 2020, 3:56pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279 "2020-06-16T15:56:22Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 16, 2020, 3:56pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/1 "2020-06-16T15:56:22Z")
Hi all, I am wondering whether some of you had maybe kind of similar problem with the DIETClassifier. Currently I am testing how good the generalization works with the standard scenario of flying from A to B. AND now I got stuck - there is no generalization happening at all. It looks like the classifier is doing a word mapping as the entity extraction works fine as long I am using cities that I used in the nlu training data. Any ideas? Andre ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 16, 2020, 5:47pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/2 "2020-06-16T17:47:22Z")
Hi @AndreD, > [@AndreD](#): > > standard scenario of flying from A to B From your example, Did you mean the `entities` cannot able to correctly identify the location and destination when you are giving input to the bot. ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 16, 2020, 7:03pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/3 "2020-06-16T19:03:57Z")
Hi Murali, exactly, but I get good entity matches (\>60%) when I take location names that are mentioned in the training data. When I take similar utterances and change only the location name - let’s say Berlin -\> Pune I get no matches at all… BR Andre ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 16, 2020, 7:26pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/4 "2020-06-16T19:26:42Z")
@AndreD, Have you tried [Entities & Roles](https://rasa.com/docs/rasa/nlu/entity-extraction/#entities-roles-groups). Declaring like this in stories.md: ``` - I am from [UK]{"entity":"Country", "role":"currentlocation"} and want to go to [USA]{"entity":"Country", "role":"destination"} - I am from [USA]{"entity":"Country", "role":"currentlocation"} and want to go to [UK]{"entity":"Country", "role":"destination"} ``` and you can get these values from `custom actions` and `utter` from there. ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 16, 2020, 8:05pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/5 "2020-06-16T20:05:41Z")
Do you think it makes any difference when I use roles in addition? ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 17, 2020, 6:07am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/6 "2020-06-17T06:07:59Z")
Yes, `entities` will now have an role. Although `entity` is a Country in my case, But it has two options either it can be a `currentlocation` or `destination`. I used `FormAction` here. So It will do automatic slot filling for me. ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/a/a409ec421c1a9542203ff40f3d31761f402b533b.png). If you want to try different approach. You can write a` if/else` condition and predict user is going to utter current location first and destination second. [Example](http://forum.rasa.com/t/using-a-single-entity-for-different-form-slots/29000/4) I done before. By doing this you will lose NER capabilities. ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 7:17am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/7 "2020-06-17T07:17:25Z")
Murali, thank you so much helping me out here 🙂 But let’s please start from the original question. So before I can dig any deeper into the customer actions I need to understand the intent/entity extraction problem… So coming back to you comments above. Why did you mention the stories.md? I mean, I wouldn’t put these utterances into the stories file, but into the nlu.md, would I? ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 17, 2020, 7:24am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/8 "2020-06-17T07:24:31Z")
Apologizes, I declared in nlu.md, But mentioned stories.md Yes It will be in nlu.md, Working late hours on my projects it got me a bit. 😅 ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/0/035b8fb3d7abdf21aa277b1dd5d79c582bd7bc9d.png) ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 10:15am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/9 "2020-06-17T10:15:37Z")
no problem. How many examples do you have annotated for the country entity? Looking at the image I see only the two you mentioned before. I am asking because I am wondering whether this is enough training data for the model to generalize … ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 10:16am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/10 "2020-06-17T10:16:26Z")
can you also show me your pipeline ? This is how mine is looking currently … ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/5/5fbacfe97ebd94bf3a927e5369eb3fe7875cdd2d.png) ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 11:40am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/11 "2020-06-17T11:40:39Z")
Now I have taken your “Country” Example one-by-one. I also took the en language model. I do not see any generalization happening either 🤔 ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 17, 2020, 5:25pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/12 "2020-06-17T17:25:54Z")
This is my config file. ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/a/a29328cebbaf26c3694775c6e724dade285af0fe.png) How’s your `actions.py ` looks like, I have only two examples in my `nlu.md` ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 6:48pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/13 "2020-06-17T18:48:24Z")
I cannot see any major differences which could cause my observations… Using your Country example delivers the following in my runtime: When I utter the sentence “I am in UK and want to go to the USA” the extractor delivers two country entities like expected When I utter the sentence “I am in India and want to go to Australia” no entities are extracted at all, which is not as I expected it - do you agree? Maybe I totally missunderstood the concept but … ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/1/18c7786dfe75dbde00dc4246d81e4f554dd345b4.png) ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 17, 2020, 6:50pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/14 "2020-06-17T18:50:46Z")
I have no code so far in my actions.py as I first wanted to understand why the extraction isn’t working … ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 17, 2020, 7:19pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/15 "2020-06-17T19:19:31Z")
Got it. Now you need to write some code in the` actions.py` to extract those `entities` and store it in a `slot` by means match those entities roles to the given slot and then we have to utter from there or do the steps you want to do. `You can try this example code: This will match the entities to their roles` ``` from typing import Any, Text, Dict, List from rasa_sdk import Action, Tracker from rasa_sdk.executor import CollectingDispatcher from rasa_sdk.forms import FormAction from rasa_sdk.events import SlotSet class CountryForm(FormAction): def name(self) -> Text: return "CountryForm" @staticmethod def required_slots(tracker: Tracker) -> List[Text]: """A list of required slots that the form has to fill""" return ["currentlocation", "destination"] def slot_mappings(self) -> Dict[Text, Any]: return { "currentlocation": self.from_entity(entity="Country", role= "currentlocation"), "destination": self.from_entity(entity="Country", role="destination") } def submit(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]) -> List[Dict]: current = tracker.get_slot('currentlocation') des = tracker.get_slot('destination') print("Current",current) print("des",des) dispatcher.utter_message("I am from {} and I want to go to {}".format(current, des)) return [] ``` stories.md ``` ## country test * greet - CountryForm - form{"name": "CountryForm"} - form{"name": null} ``` nlu.md ![image](https://europe1.discourse-cdn.com/flex013/uploads/rasa/original/2X/b/b61206e043f2f99466d4a6b7bf674e3573ab441c.png) config.yml ` - name: FormPolicy` do not forget to declare formpolicy on the config.yml ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 18, 2020, 6:32am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/16 "2020-06-18T06:32:55Z")
Hi Murali, thanks a lot, I will definetely do so…I will come back whether it works. It is anyway part of my learning process using the RASA stack 🙂 But this will not help regarding the NLU topic as such, as I only can store something that I have been able to extract before. Like mentioned before in the ‘Country’ example: _…When I utter the sentence “I am in UK and want to go to the USA” the extractor delivers two country entities like expected_ _When I utter the sentence “I am in India and want to go to Australia” no entities are extracted at all, which is not as I expected it - do you agree? Maybe I totally missunderstood the concept but …_ ---
**Author:** ![MuraliChandran14](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/muralichandran14/32/9120_2.png) [@MuraliChandran14](https://forum.rasa.com/u/MuraliChandran14)\ **Post date:** [June 18, 2020, 7:36am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/17 "2020-06-18T07:36:23Z")
> [@AndreD](#): > > “I am in India and want to go to Australia” If you have not declared that in your nlu.md, then you cannot extract those entities. ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 18, 2020, 11:44am UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/18 "2020-06-18T11:44:50Z")
So basicly this was my main question… and I really have doubts believing it to be honest 😉 Why should I try to use a machine learning mechanism for something that a simple mappingtable can solve even better? Reading the following in the documentation _“…in order to properly train your model with entities that have roles/groups, make sure to include enough training data examples for every combination of entity and role/group label. Also make sure to have some variations in your training data, so that the model is able to generalize. For example, you should not only have example like `fly FROM x TO y` , but also include examples like `fly TO y FROM x` …”_ was for my understanding an indication that the model is foreseen to generalize from several examples … ---
**Author:** ![sainimohit23](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/sainimohit23/32/9483_2.png) [@sainimohit23](https://forum.rasa.com/u/sainimohit23)\ **Post date:** [June 18, 2020, 12:13pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/19 "2020-06-18T12:13:39Z")
Try this pipeline. Works pretty well for me- ```auto pipeline: - name: ConveRTTokenizer - name: ConveRTFeaturizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CRFEntityExtractor # added new - name: CountVectorsFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier epochs: 50 - name: EntitySynonymMapper - name: ResponseSelector epochs: 100 ``` ---
**Author:** ![AndreD](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@AndreD](https://forum.rasa.com/u/AndreD)\ **Post date:** [June 18, 2020, 3:38pm UTC](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279/20 "2020-06-18T15:38:30Z")
Thanks for your contribution to help me. I tried it, but I am running into a runtime error. It seems that the dependencies are not correct as I got this during training of nlu: undefined symbol: \_ZN10tensorflow8OpKernel11TraceStringEPNS\_15OpKernelContextEb It may help to show me your full package list [Next page](https://forum.rasa.com/t/dietclassifier-does-not-generalize/30279.md?page=2)