Urgent: Empty nlu.md file in Rasa X. Unable to upload new nlu file as well

[Deprecated] Rasa X Community Edition
1

Hi,

I deployed Rasa X on GCP using the following tutorial: The Rasa Masterclass Handbook: Episode 9 and installed Rasa x version 0.27.5.

I was successfully able to complete all the steps and integrate Rasa X with my Rasa bot which I uploaded on github. I have followed the correct file hierarchy while uploading the bot and have uploaded all files like nlu.md, stories.md, domain.yml etc. However, I was unable to upload the model I trained on my local machine due to its size, but I believe that shouldn’t be an issue since I can train a new model on Rasa X.

The problem I’m facing is that when I open Rasa X and sync with the git server, all the stories, the configuration and domain data is there, however when I go to the Training Data section, there are 0 examples. Even when I downloaded the training data, it’s an empty nlu.md file. So Rasa X is able to get all the files from my github except the nlu.md file. The training data I’ve added on github is in JSON format and is around 236 KB in size.

I even tried to use the ‘Upload a new NLU file’ button followed by the ‘Upload and Replace’ button but am getting the following error: ‘Upload Failed, training data upload failed’. I’ve tried deploying previous versions of Rasa X as well and have encountered the same error with them.

I’m unable to understand the problem, would really appreciate if someone could help me with this. Thank you.

2

Hi Nishtha,

If you NLU data is in json format, the file should be named nlu.json not nlu.md.

Can you confirm that the nlu file is under the data directory.

You should also look for Rasa X errors via docker-compose logs rasa-x and when you upload the NLU data in the UI, first open the browser debug window network tab and see look for a more detailed error from the POST.

Greg

3

Hey @stephens,

Thank you for your reply. I changed the file into a JSON file and updated it on the github repo as well (under the data directory). Even after Rasa X completed syncing with the git server, I’m seeing an empty training space and I’m still unable to upload it using the button.

I checked the logs and this is what I think the problem is:

rasa-x_1           | [SQL: INSERT INTO nlu_training_data_entity (example_id, entity, value, original_value, start, "end", extractor, entity_synonym_id) VALUES (%(ex
ample_id)s, %(entity)s, %(value)s, %(original_value)s, %(start)s, %(end)s, %(extractor)s, %(entity_synonym_id)s) RETURNING nlu_training_data_entity.id]
rasa-x_1           | [parameters: {'example_id': 14072, 'entity': 'dispatch_request_data', 'value': '{"service_type":"Break Fix","sub_service_type":"parts_and_labou
r","service_tag":"1133MBX","env":"ge2","primary_contact":{"first_name":"","last_name": ... (284 characters truncated) ... ","DSPType":"","ClassType":"","DSP":"","pa
rts":{"number":"integer","quantity_to_order":"integer","quantity_to_return":"integer","partusednew":"New"}}', 'original_value': '{"service_type":"Break Fix","sub_se
rvice_type":"parts_and_labour","service_tag":"1133MBX","env":"ge2","primary_contact":{"first_name":"","last_name": ... (284 characters truncated) ... ","DSPType":""
,"ClassType":"","DSP":"","parts":{"number":"integer","quantity_to_order":"integer","quantity_to_return":"integer","partusednew":"New"}}', 'start': 37, 'end': 619, '
extractor': None, 'entity_synonym_id': None}]
rasa-x_1           | (Background on this error at: http://sqlalche.me/e/9h9h)
rasa-x_1           | Exception occurred while handling uri: 'http://35.240.255.248/api/projects/default/data'
rasa-x_1           | Traceback (most recent call last):
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
rasa-x_1           |     cursor, statement, parameters, context
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
rasa-x_1           |     cursor.execute(statement, parameters)
rasa-x_1           | psycopg2.errors.StringDataRightTruncation: value too long for type character varying(255)
rasa-x_1           | 
rasa-x_1           | 
rasa-x_1           | The above exception was the direct cause of the following exception:
rasa-x_1           | 
rasa-x_1           | Traceback (most recent call last):
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sanic/app.py", line 976, in handle_request
rasa-x_1           |     response = await response
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/api/decorators.py", line 205, in decorated_function
rasa-x_1           |     return await await_and_return_response(args, kwargs, request)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/api/decorators.py", line 135, in await_and_return_response
rasa-x_1           |     response = await response
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/api/decorators.py", line 42, in decorated_function
rasa-x_1           |     return await f(request, user=user, *args, **kwargs)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/api/blueprints/nlu.py", line 386, in training_data_bulk
rasa-x_1           |     data_service.replace_data(project_id, rjs, RASA, user["username"])
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/data_service.py", line 1565, in replace_data
rasa-x_1           |     project_id, data, data_format, username, filename, dump_data
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/data_service.py", line 1726, in save_bulk_data
rasa-x_1           |     self._add_intents_and_entities_to_domain(examples, project_id)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/data_service.py", line 400, in _add_intents_and_entities_to_domain
rasa-x_1           |     self._add_items_to_domain(entities, intents, project_id)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/data_service.py", line 424, in _add_items_to_domain
rasa-x_1           |     origin="NLU training data",
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/domain_service.py", line 512, in add_items_to_domain
rasa-x_1           |     domain = self._get_or_create_domain(project_id)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/domain_service.py", line 190, in _get_or_create_domain
rasa-x_1           |     domain = self._get_domain(project_id)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/rasax/community/services/domain_service.py", line 146, in _get_domain
rasa-x_1           |     return self.query(Domain).filter(Domain.project_id == project_id).first()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3298, in first
rasa-x_1           |     ret = list(self[0:1])
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3076, in __getitem__
rasa-x_1           |     return list(res)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3402, in __iter__
rasa-x_1           |     self.session._autoflush()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1602, in _autoflush
rasa-x_1           |     util.raise_(e, with_traceback=sys.exc_info()[2])
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
rasa-x_1           |     raise exception
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1591, in _autoflush
rasa-x_1           |     self.flush()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2496, in flush
rasa-x_1           |     self._flush(objects)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2637, in _flush
rasa-x_1           |     transaction.rollback(_capture_exception=True)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
rasa-x_1           |     exc_value, with_traceback=exc_tb,
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
rasa-x_1           |     raise exception
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2597, in _flush
rasa-x_1           |     flush_context.execute()
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rasa-x_1           |     rec.execute(self)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
rasa-x_1           |     uow,
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
rasa-x_1           |     insert,
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1136, in _emit_insert_statements
rasa-x_1           |     statement, params
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 984, in execute
rasa-x_1           |     return meth(self, multiparams, params)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
rasa-x_1           |     return connection._execute_clauseelement(self, multiparams, params)
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
rasa-x_1           |     e, statement, parameters, cursor, context
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
rasa-x_1           |     sqlalchemy_exception, with_traceback=exc_info[2], from_=e
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
rasa-x_1           |     raise exception
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
rasa-x_1           |     cursor, statement, parameters, context
rasa-x_1           |   File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
rasa-x_1           |     cursor.execute(statement, parameters)
rasa-x_1           | sqlalchemy.exc.DataError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurrin
g prematurely)
rasa-x_1           | (psycopg2.errors.StringDataRightTruncation) value too long for type character varying(255)`

Does the ‘(psycopg2.errors.StringDataRightTruncation) value too long for type character varying(255)’ error mean that my file is too large or that the individual examples are too large? And how can I fix this error? One way would be to manually add all the examples on Rasa X using the UI but that would be too time consuming for me.

Thanks!

4

Hi Nishtha,

It does look like there is an issue with your json. You might try running it through a jsonlint tool like this.

I would also convert the file to markdown with the rasa data convert nlu command described here.

Greg

5

Hi there,

I have nearly the same isue since yesterday. The only difference I see in the server side logs is an error that says some duplicate constraint is violated …

sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint “message_log_pkey” rasa-x_1 | DETAIL: Key (id)=(50) already exists.

Any ideas ?

BR Andre