Rasa X - Annotate New data not auto-populating from New Conversations

Hi Everyone,

Started using Rasa/Rasa-X a few months ago. So far, my bot is working fine. I used the Mongotracker to store the conversations in MongoDB and then used the migrate script to import those conversations to Rasa-X. This part is working fine and the conversations are being imported correctly to the Conversations page. But the intents from the conversations are not getting auto-populated to the “Annotate new data” under NLU training. I tried several Rasa-X versions by upgrading/downgrading but somehow, this doesnt seem to work. I can annotate the intents from the conversations page, but that is very exhausting. Is there some extra step required to import from Conversations to Annotate new data? Because as per the documentation, my understanding is that it should be auto-populated. Thanks!!

That’s due to the migration script. It doesn’t populate the table required for the logs to show up.I updated the migration script to do so. Can you please check?

Hi Tobias,

Thank you for your prompt response. It is still not getting populated. The conversations page is getting loaded fine.

It also gives the below error for all trackers while the migration script is running.

No model found for project ‘default’. Traceback (most recent call last): File “C:\Users\vvasani\AppData\Local\Programs\Python\Python36\lib\site-packages\rasax\community\services\logs_service.py”, line 220, in save_nlu_logs_from_event event.parse_data, event_id=event_id File “C:\Users\vvasani\AppData\Local\Programs\Python\Python36\lib\site-packages\rasax\community\services\logs_service.py”, line 160, in create_log_from_parse_data log = self._create_log(parse_data, event_id, created_from_model) File “C:\Users\vvasani\AppData\Local\Programs\Python\Python36\lib\site-packages\rasax\community\services\logs_service.py”, line 193, in _create_log raise ValueError(f"No model found for project ‘{project}’.") ValueError: No model found for project ‘default’. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [01:33<00:00, 11.70s/it] Finished migrating trackers (0 were skipped since they were already migrated).

Hi @Tobias_Wochinger,

Please let me know if you have any updates/tips? Am i missing something here?

@vvasani Thanks, that was an error in the script. I fixed it, can you please try again?

Hi @Tobias_Wochinger,

Thanks for your prompt response! I tried to run the updated migration script. It resulted in the below error:

Welcome to Rasa X

This script will migrate your old tracker store to the new SQL based Rasa X tracker store. Let’s start!

? Please provide the path to your endpoints configuration which specifies the credentials for your old tracker store: endpoints.yml ? Do you want to migrate to the local version of Rasa X? Yes Start migrating 9 trackers. 0%| | 0/9 [00:00<?, ?it/s] Traceback (most recent call last): File “migrate.py”, line 242, in args.max_trackers, File “migrate.py”, line 143, in _migrate_tracker_store_to_rasa_x _replay_tracker_events(tracker, event_service, logs_service, analytics_service) File “migrate.py”, line 206, in _replay_tracker_events stringified_event, sender_id=event.conversation_id TypeError: () got an unexpected keyword argument ‘sender_id’

Thanks!

@vvasani Is that the full stack trace? Which Rasa X version are you running?

Hi @Tobias_Wochinger,

I am simply running the migration script independently and trying to get the existing conversations from MongoDB to Rasa-X. I have not yet integrated my solution and am pushing my conversations from Jupyter to Mongo tracker store. I tried it one more time today and am still getting the same error.

I have Rasa-X (0.22.2) and Rasa (1.4.3).

Thanks!

Could you please try with Rasa X 0.23? I believe that some function parameters changed throughout the versions.

Hi @Tobias_Wochinger,

I updated to Rasa X 0.23.2, however still getting the same error when I run the migrate script:

C:\Users\vvasani\Documents\Timesheets>python migrate.py Welcome to Rasa X :rocket:

This script will migrate your old tracker store to the new SQL based Rasa X tracker store. Let’s start!

? Please provide the path to your endpoints configuration which specifies the credentials for your old tracker store: endpoints.yml ? Do you want to migrate to the local version of Rasa X? Yes ? Found existing trackers in your Rasa X tracker store. Do you still want to migrate the new trackers? Yes Start migrating 9 trackers. 0%| | 0/9 [00:00<?, ?it/s] Traceback (most recent call last): File “migrate.py”, line 242, in args.max_trackers, File “migrate.py”, line 143, in _migrate_tracker_store_to_rasa_x _replay_tracker_events(tracker, event_service, logs_service, analytics_service) File “migrate.py”, line 206, in _replay_tracker_events stringified_event, sender_id=event.conversation_id TypeError: () got an unexpected keyword argument ‘sender_id’

This is weird :exploding_head: Do you have the latest script from Migrates a Rasa tracker store (Redis, Mongo, SQL) to Rasa X. · GitHub ?

And can you please do a pip list to make sure it’s actually 0.23.2?

Hi @Tobias_Wochinger,

Yes, everytime I test it, I create a new file by copying the code from Github.
pip list gives me the below. rasa 1.5.1 rasa-sdk 1.5.1 rasa-x 0.23.2

I have Python 3.6.4 - am wondering if this has something to do with that?

Pushed a fix :+1: Can you please try again?

Hi @Tobias_Wochinger,

This is awesome! Thank you so much. The Annotate new data page is being populated in the RasaX UI now.

However, am getting this error on the shell: UserWarning: Could not find endpoint file at path ‘endpoints.yml’: File ‘/app/endpoints.yml’ does not exist… f"Could not find endpoint file at path ‘{config.endpoints_path}’:\n{e}."

Never mind… I added it as argument while starting Rasa x and it worked. Thanks a ton!

Hi @vvasani,

sorry, this warning is a bit misleading. You can just ignore it for now!