Rasax train model exception

[Deprecated] Rasa X Community Edition
1

rasa :2.7 rasax:0.41

when i train model in rasax ,it was failed , my nludata has 5400 examples i have train about 1 hour it waring me failed

the logs like that:

2022/01/26 17:29:52 [error] 39#39: *1107 upstream timed out (110: Connection timed out) while 
reading response header from upstream, client: 27.17.10.210, server: , request: "POST 
/api/projects/default/models/jobs HTTP/1.1", upstream: 
"http://172.18.0.6:5002/api/projects/default/models/jobs", host: "47.94.242.77", referrer: 
"http://47.94.242.77/data/training-data"

why warning me timed out

i found train start rasax is work ok, the rasa logs don’t have any error

2

@yangli can you please start with small nlu data (50 or 100) and train and check you are getting the response or not?

3

Is there a limit for rasa train example，when i have 4000 example it train OK， but when i increase to 5200 example it train failed

4

@yangli no, did you able to train with small batches of nlu data?

5

It’s just a timeout from Nginx because training takes a long time, so it assumes the request failed.

Try setting proxy_read_timeout to a higher value in your Nginx config.

7

yes ,i train about 4000 examples it train success.

Does the training data affect the completion of the training?

8

yes，that’s had reconfiguration 5 hours ， but i found train about 4000 examples it take 1 hour 40 minute train success, examples up to the about 5000 it train failed and i found rasax logs didn’t had any error

9

@yangli are you training the NLU examples using Rasa X ? Or Local Machine and uploading?

@yangli can you train the NLU example 4000 examples using Rasa Open Source 2.7 please note the time.

Note: Delete all the previous trained models and then only train and note the time.

FYI: I have 2000 Examples for one of my projects and it trains in max 10min - 15mins using Rasa X (Server Side)and Rasa Open Source (Local machine)

@yangli please even share your system configuration.

Well, training much depends on NLU training examples, but an issue can be anything, I can’t comment on this.

10

Even I want to know this, are you fetching the training data while using API?