From the tutorial here, I can see output format for entities: https://blog.rasa.com/enhancing-rasa-nlu-with-custom-components
I want to fetch tokens from SpacyTokenizer (the previous component in nlu pipeline), operate on them and return to the SpacyFeaturizer(the next component in nlu pipeline).
This is what my process() function looks like:
def process(self, message, **kwargs):
"""Retrieve the tokens of the new message, pass it to the classifier
and append prediction results to the message class."""
tokens = [t.text for t in message.get("tokens")]
corrected_tokens = self.preprocessing(tokens)
tokens = self.convert_to_rasa(corrected_tokens)
message.set("tokens", [token], add_to_output=True)
What should be in my convert_to_rasa function?
Thanks!