I want to ask if this is right how I am manipulate the incoming message from core before parsing with NLU. I do this within the parse function of model.py where I just use a simple stemmer. Now I get different intent for
problem with data
problems with data
Although both are get stemmed to same way! the word problems is stemmed same like problem.
I wonder why? Same sentences gets parsed afterwards in the components. Are there some caveats?
After parsing I cehck if intent is None from tensorflow and want to overwrite it. Although the tracker are seem to get correctly filled, I see some difference when uttering a None word vs entering \BLA ?
def parse(self, text, time=None, only_output_properties=True):
# type: (Text) -> Dict[Text, Any]
"""Parse the input text, classify it and return pipeline result.
The pipeline result usually contains intent and entities."""
stemmer= Stemmer()
if not text:
# Not all components are able to handle empty strings. So we need
# to prevent that... This default return will not contain all
# output attributes of all components, but in the end, no one
# should pass an empty string in the first place.
output = self.default_output_attributes()
output["text"] = ""
return output
tokens = nltk.word_tokenize(text)
tokens = [stemmer.stem(t) for t in tokens]
text=" ".join(tokens)
#logger.info("Parsed Message:************** '{}'".format(text))
#print(text)
message = Message(text, self.default_output_attributes(), time=time)
for component in self.pipeline:
component.process(message, **self.context)
if message.data['intent']['name']==None:
# print("YES")
m=text.split()
if len(m)==1:
message.data['intent']['name']='BLA'
message.data['intent']['confidence']=0.99
# print(message.data['intent'])
output = self.default_output_attributes()
output.update(message.as_dict(
only_output_properties=only_output_properties))
return output
Any idea @akelad
