# Manipulating message in parse function of model.py

**URL:** <https://forum.rasa.com/t/manipulating-message-in-parse-function-of-model-py/2368>\
**Category:** Rasa Open Source\
**Created:** [November 1, 2018, 4:11pm UTC](https://forum.rasa.com/t/manipulating-message-in-parse-function-of-model-py/2368 "2018-11-01T16:11:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![datistiquo](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/datistiquo/32/124_2.png) [@datistiquo](https://forum.rasa.com/u/datistiquo)\
**Post date:** [November 1, 2018, 4:11pm UTC](https://forum.rasa.com/t/manipulating-message-in-parse-function-of-model-py/2368/1 "2018-11-01T16:11:52Z")

</div>

I want to ask if this is right how I am manipulate the incoming message from core before parsing with NLU. I do this within the parse function of model.py where I just use a simple stemmer. Now I get different intent for

`problem with data`

`problems with data`

Although both are get stemmed to same way! the word `problems` is stemmed same like `problem`.

I wonder why? Same sentences gets parsed afterwards in the components. Are there some caveats?

After parsing I cehck if intent is `None` from tensorflow and want to overwrite it. Although the tracker are seem to get correctly filled, I see some difference when uttering a None word vs entering `\BLA` ?

```auto
    def parse(self, text, time=None, only_output_properties=True):
        # type: (Text) -> Dict[Text, Any]
        """Parse the input text, classify it and return pipeline result.

        The pipeline result usually contains intent and entities."""
        stemmer= Stemmer()

        if not text:
            # Not all components are able to handle empty strings. So we need
            # to prevent that... This default return will not contain all
            # output attributes of all components, but in the end, no one
            # should pass an empty string in the first place.
            output = self.default_output_attributes()
            output["text"] = ""
            return output
        tokens = nltk.word_tokenize(text)
        tokens = [stemmer.stem(t) for t in tokens]
        text=" ".join(tokens)
        #logger.info("Parsed Message: **************'{}'".format(text))
        #print(text)
        message = Message(text, self.default_output_attributes(), time=time)

        for component in self.pipeline:
            component.process(message, **self.context)
        
       
        if message.data['intent']['name']==None:
           # print("YES")
            m=text.split()
            if len(m)==1:                
                message.data['intent']['name']='BLA'
                message.data['intent']['confidence']=0.99
      # print(message.data['intent'])

        output = self.default_output_attributes()
        output.update(message.as_dict(
                only_output_properties=only_output_properties))
        return output

```

Any idea @akelad

🙂

---

<div class="post-metadata">

**Author:** ![datistiquo](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/datistiquo/32/124_2.png) [@datistiquo](https://forum.rasa.com/u/datistiquo)\
**Post date:** [November 1, 2018, 6:40pm UTC](https://forum.rasa.com/t/manipulating-message-in-parse-function-of-model-py/2368/2 "2018-11-01T18:40:51Z")

</div>

Here is the log showing a real difference in tracker states. I input a text which is None intent and compare it versus `/Bla` directly to which I want to set a None word to like above.

```auto
vbn
DEBUG:rasa_core.tracker_store:Creating a new tracker for id 'default'.
DEBUG:rasa_core.tracker_store:Recreating tracker for id 'default'
vbn
DEBUG:rasa_core.processor:Received user message 'vbn' with intent '{'name': 'BLA', 'confidence': 0.99}' and entities '[]'
parsedata
{'intent': {'name': 'BLA', 'confidence': 0.99}, 'entities': [], 'intent_ranking': [], 'text': 'vbn'}
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 2 events
DEBUG:rasa_core.processor:Current slot values:
...
NEXT ACTION
UserUttered(text: vbn, intent: {'name': 'BLA', 'confidence': 0.99}, entities: [])

CURRENT STAE
{'sender_id': 'default', 'slots': {...}, 'latest_message': {'intent': {'name': 'BLA', 'confidence': 0.99}, 'entities': [], 'intent_ranking': [], 'text': 'vbn'}, 'latest_event_time': 1541096797.4163308, 'paused': False, 'events': None}

trackers_as_states
[[None, {}, {'prev_action_listen': 1.0}]]

POLICY NAME
KerasPolicy
0.24645854532718658
POLICY NAME
FallbackPolicy
0.1
DEBUG:rasa_core.policies.ensemble:Predicted next action using policy_0_KerasPolicy
DEBUG:rasa_core.policies.ensemble:Predicted next action 'action_BLA' with prob 0.65.
ACTION NAME
action_BLA

```

compared to uttering with `/BLA`

```auto
/BLA
DEBUG:rasa_core.tracker_store:Recreating tracker for id 'default'
DEBUG:rasa_core.processor:Received user message '/BLA' with intent '{'name': 'BLA', 'confidence': 1.0}' and entities '[]'
parsedata
{'text': '/BLA', 'intent': {'name': 'BLA', 'confidence': 1.0}, 'intent_ranking': [{'name': 'BLA', 'confidence': 1.0}], 'entities': []}
DEBUG:rasa_core.processor:Logged UserUtterance - tracker now has 14 events
DEBUG:rasa_core.processor:Current slot values:
...
NEXT ACTION
UserUttered(text: /BLA, intent: {'name': 'BLA', 'confidence': 1.0}, entities: [])
CURRENT STAE
{'sender_id': 'default', 'slots': {...}, 'latest_message': {'text': '/BLA', 'intent': {'name': 'BLA', 'confidence': 1.0}, 'intent_ranking': [{'name': 'BLA', 'confidence': 1.0}], 'entities': []}, 'latest_event_time': 1541097292.5325198, 'paused': False, 'events': None}

trackers_as_states
[[None, {}, {'intent_BLA: 1.0, 'prev_action_listen': 1.0}]]

POLICY NAME
KerasPolicy
0.9567252397537231
POLICY NAME
FallbackPolicy
0.1
DEBUG:rasa_core.policies.ensemble:Predicted next action using policy_0_KerasPolicy
DEBUG:rasa_core.policies.ensemble:Predicted next action 'action_BLA' with prob 1.00.
ACTION NAME
action_BLA

```

So you see a difference in the tracker state:

```auto
trackers_as_states
[[None, {}, {'prev_action_listen': 1.0}]]

```

VS

```auto
trackers_as_states
[[None, {}, {'intent_BLA: 1.0, 'prev_action_listen': 1.0}]]

```

Manual overwriting the intent in model.py seems to miss something. The intent for a word with `BLA` intent coming from overwriting don’t yield the the bla intent inside the tracker as seen from the last log.

Therefore my Keras Policy does not work so well.

---

<div class="post-metadata">

**Author:** ![datistiquo](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.rasa.com/datistiquo/32/124_2.png) [@datistiquo](https://forum.rasa.com/u/datistiquo)\
**Post date:** [November 5, 2018, 1:01pm UTC](https://forum.rasa.com/t/manipulating-message-in-parse-function-of-model-py/2368/3 "2018-11-05T13:01:36Z")

</div>

@akelad. I do it now with a custom component and the 2nd issue is solved. But I had to add a intent ranking not just name and confidence? So, adding a intent ranking in model.py would be suffiecient too.

Could you imagine what is the problem with my first issue at the beginning using the stemmer?

Thanks!