Hi,
I have a laptop with 16 GB of Ram and i’m trying to train a model with a file that contain 307,175 rows but the laptop can not with this…
it’s possible to train a model a incremental way for example 10,000 by 10,000 rows?
Thanks
You might want to look into the batch size section for the Embedding policy here to see if you can tweak it to be lower. That I think is the main memory-related parameter.
What is the behavior that you’re seeing?
Hi Kevin, thanks for the answer
when i try to train the model a get this error
ERORR 2019-07-26 09:21:37 INFO rasa.nlu.model - Starting to train component WhitespaceTokenizer 2019-07-26 09:21:42 INFO rasa.nlu.model - Finished training component. 2019-07-26 09:21:42 INFO rasa.nlu.model - Starting to train component RegexFeaturizer 2019-07-26 09:21:42 INFO rasa.nlu.model - Finished training component. 2019-07-26 09:21:42 INFO rasa.nlu.model - Starting to train component CRFEntityExtractor 2019-07-26 09:21:42 INFO rasa.nlu.model - Finished training component. 2019-07-26 09:21:42 INFO rasa.nlu.model - Starting to train component EntitySynonymMapper 2019-07-26 09:21:42 INFO rasa.nlu.model - Finished training component. 2019-07-26 09:21:42 INFO rasa.nlu.model - Starting to train component CountVectorsFeaturizer Traceback (most recent call last): File “C:\Users\rjose\AppData\Local\Programs\Python\Python37\Scripts\rasa-script.py”, line 11, in load_entry_point(‘rasa’, ‘console_scripts’, ‘rasa’)() File “c:\users\rjose\documents\rasahq\rasa\rasa_main_.py”, line 76, in main cmdline_arguments.func(cmdline_arguments) File “c:\users\rjose\documents\rasahq\rasa\rasa\cli\train.py”, line 77, in train kwargs=extract_additional_arguments(args), File “c:\users\rjose\documents\rasahq\rasa\rasa\train.py”, line 42, in train kwargs=kwargs, File “c:\users\rjose\appdata\local\programs\python\python37\lib\asyncio\base_events.py”, line 568, in run_until_complete return future.result() File “c:\users\rjose\documents\rasahq\rasa\rasa\train.py”, line 100, in train_async kwargs, File “c:\users\rjose\documents\rasahq\rasa\rasa\train.py”, line 203, in _train_async_internal kwargs=kwargs, File “c:\users\rjose\documents\rasahq\rasa\rasa\train.py”, line 256, in _do_training fixed_model_name=fixed_model_name, File “c:\users\rjose\documents\rasahq\rasa\rasa\train.py”, line 463, in _train_nlu_with_validated_data config, nlu_data_directory, _train_path, fixed_model_name=“nlu” File “c:\users\rjose\documents\rasahq\rasa\rasa\nlu\train.py”, line 89, in train interpreter = trainer.train(training_data, **kwargs) File “c:\users\rjose\documents\rasahq\rasa\rasa\nlu\model.py”, line 192, in train updates = component.train(working_data, self.config, **context) File “c:\users\rjose\documents\rasahq\rasa\rasa\nlu\featurizers\count_vectors_featurizer.py”, line 242, in train X = self.vectorizer.fit_transform(lem_exs).toarray() File “c:\users\rjose\appdata\local\programs\python\python37\lib\site-packages\scipy\sparse\compressed.py”, line 962, in toarray out = self._process_toarray_args(order, out) File “c:\users\rjose\appdata\local\programs\python\python37\lib\site-packages\scipy\sparse\base.py”, line 1187, in _process_toarray_args return np.zeros(self.shape, dtype=self.dtype, order=order) MemoryError
Do you find any solution on this @RobertJose