Hello, I am training a chemical experimental QA chatbot, which can chat and answer questions related to chemical experiments. I am currently using a custom action to read the data from mongodb, so that nlu is trained as an intent and can run successfully, but I After increasing the amount of data, I found that rasa nlu cannot train the intent, is there any other way to make it run successfully, thank you [image] (training failed error message)

@Rosscoffman [image] This is part of my data, question and a corresponding answer, most of them are like this, I organize the question set into a csv file and upload it to mongodb compass [image] There are 777243 records in total

@nik202 HI,Sorry to trouble you After the data set becomes larger, rasa nlu cannot train the intent normally. Is there any way to improve it?

Hi, are you adding new classes of intents?

I only set the question in the database as an intent (question) [image]

@dan246 ok no worries. When you say after the data set becomes larger what you actually doing that it increase the size of data set and what you mean by dataset? Is that you mean the Rasa NLU intents with the set of questions you are adding or anything else? In rasa there is no database ( so what to know what you meant by this) Thanks.

@nik202 I use custom action to let rasa read the QA data in mongodb compass, convert the questions in it into intents and train nlu, but after I increase the number of questions, the nlu training error occurs

@dan246 it should not show the error, but you need to check the resources required for running the Rasa. Check Rasa documentation. Secondly, check are you able to convert the data into nlu as set of questions. Third, try delete the previous trained model and re run train and talk to your bot. Fourth, try upload in a small batchs so that you will able to figure out after what amount of data it’s breaking. Otherwise, try write the code to fetch the data from CSV may be MongoDB can have some issue. Do this and let me know.

@nik202 Hi, I tried a small part of the data and it works, but when I put all the data in, it doesn’t work, after it shows this (image), then my computer has a Blue Screen of Death [1]

@dan246 as expected your computer not have resources to meet the current data handling, so increase the resources and see the difference. @dan246 try more small data again.

Nlu intent training with large dataset

Rasa Open Source

Rosscoffman (Ross) October 1, 2022, 5:54am 2

First of all tell me what types of data you have and Give me proper Details.

Rasa x run error

Topic		Replies	Views
Using API Train of Rasa NLU by big data set? Rasa Open Source	1	643	January 28, 2020
How large is a typical dataset for training Rasa NLU? Rasa Open Source	3	2145	September 18, 2018
Rasa nlu train with a large dataset is stuck Rasa Open Source	20	2294	April 8, 2020
Rasa NLU training fails beyond certain Intents (without any error message) Rasa Open Source	2	858	September 30, 2019
How does rasa train on large data Rasa Open Source	2	388	September 13, 2022

Nlu intent training with large dataset

Related topics