How to run a function before training the model?

Rasa Open Source
1

Hello, I’m working on a project which need to update stories files and nlu files regularly through Mongodb. I have already make a function to convert from Mongodb document to yml files (Namely stories.yml and nlu.yml). How can I run this function every time I run rasa train? Should I make a custom pipeline and run this function or are there any other ways? Thanks for reading.

2

@namph-sgn Hi, got it means as the data added in the Mongo DB Database, you will create the data into nlu.yml (that make senses to me) but, I am thinking how you will create the stories for the same? or you already created the default set of stories for your use case. Further, I guess you have written the code in actions.py ( do let me know if I am right), that means you also mentioned the tracker store for mongo db, so this means you firstly need to run the action server whilst using rasa run actions and then you need to train the rasa whilst using rasa train, then only it will train the model based on nlu and based on corresponding stories. Please do let me clear this first and then will see the solution. Good Luck!

1 Like
3

Thank for your quick response. To answer your questions:

  1. We are currently using a drag and drop front-end to create stories. Right now stories mainly contains utterance and form questions. (Somewhat like ManyChat with entity detection and intent classification) Every stories will be collected and changed in MongoDB.
  2. We decided to managed all stories and nlu on Mongodb because stories will be created and changed on a weekly basis. We design it this way to make it easy to create/change/remove stories. The only time we need to converse to yml is when we need to train the model.
  3. I haven’t decided where to put the conversion code in. Because I only need to update the yml files when training only so I’m thinking of putting it in a custom importer just before creating Story Graph from the yml file (Correct me if my method is inefficient here).
4

@namph-sgn Can you share some screenshot for me? or some scenario ? Me bit confused how you dealing with nlu and stories honestly using drag and drop and 3 point not clear to me :frowning:

1 Like
5

You can edit the virtual environment’s activation file to execute a custom command.

Since you already created the function and it works well, put it inside a file called myfile.py, which is inside a folder called myfolder in the root of your chatbot.

Add the following to the end of your venv/Scripts/action.bat (on Windows):

doskey train=python myfolder/myfile.py ^& rasa train $*

Then, when you want to execute that, just write train in the console.

Or, to do it the “official” way, use a Training Data Importer.

1 Like
6

@ChrisRahme The frontend and backend are communicating through API. Will there be any possible problems about running the script with bat files?
Currently trying the Training Data Importers, seems less hacky that way.

1 Like
7

You can try running your script from another .bat file before changing the virtual environment and see if it works. But it should, unless there are relative paths in your scripts, which you need to transform into absolute paths (you could do it manually, but dynamically inside the code is better).

Anyway, indeed, Importers are preferable :slight_smile:

1 Like
8

Because our customer are not good at code. We made a drag and drop system for them Example: https://botstar.com/. There will be frequent changes to the stories like this:

  • Intent: greet
  • Action: greet_back

Changed to:

  • Intent: greet
  • Action: utter_sale
  • Action: greet_back

And because the changes is so frequent, create/delete/change them on a mongodb database is easier than done on a yml file. My idea is to change them all on the database first then convert them to the yml file for training. My problem is I don’t know where to put the code converting mongodb to yml file: in Training Data Importers, in Components or any other place.

1 Like
9

It’s an awesome project!

10

@ChrisRahme Thank you very much. Still trying out RASA but I’m happy that the community is so helpful and active.

1 Like
11

Nice, I’m able to achieve the intended result by copying the original RASA Training Data Import in and just run the function before running the get_stories function. Considered this topic closed.
On the other notes. I’m using RASA 2.8.1. In the current version 2.8.7 all functions in importers switch from async to normal function so this code might not work for future reader. 

async def get_stories(
        self,
        template_variables: Optional[Dict] = None,
        use_e2e: bool = False,
        exclusion_percentage: Optional[int] = None,
    ) -> StoryGraph:
        """update_file_from_db()"""
        return await utils.story_graph_from_paths(
            self._story_files,
            await self.get_domain(),
            template_variables,
            use_e2e,
            exclusion_percentage,
        )
1 Like