Implementing TFIDF as a custom component? [necro]

Rasa Open Source
4

what do you mean “not really as a full-fledged approach”? It works very well for Q&A type interactions.

You can pass the train method, that doesn’t have to be implemented. E.g. this custom spell checker component I built as an example a while ago doesn’t use the train method:

from autocorrect import spell

class RasaSpellChecker(Component):

    defaults = {}
    requires = ["tokens"]
    provides = ["tokens"]
    name = "rasa_spell_checker"

    def __init__(self, component_config=None):
        super(RasaSpellChecker, self).__init__(component_config)

    def train(self, training_data, cfg, **kwargs):
        pass

    def process(self, message, **kwargs):
        entity_list = message.get("entities")
        donot_replace = []
        if entity_list:
            message.set("entities", [])
            for e in entity_list:
                print(e)
                if e["entity"] == "name":
                    donot_replace.append(e["value"])

        tokens = [t.text for t in message.get("tokens")]
        correct_tokens = [spell(t) if t not in donot_replace else t for t in tokens]

        for i, t in enumerate(message.get("tokens")):
            t.text = correct_tokens[i]

In this case it sets the tokens of the message, in your case you would set the “features” instead, like in the spacy featurizer for example: rasa/spacy_featurizer.py at master · RasaHQ/rasa · GitHub