Load times for NLU Model Interpreter

Rasa Open Source
1

I have been doing some performance tests for model loading of DIETClassifiers without pre-trained word vectors. Looks like there is a lot of smart caching / loading going on. Does anybody know why loading the first model is so much slower than the latter ones? Also is there a way to speed up the load time for the very first model?

@koaning would be awesome if you could point me in the right direction.

from pathlib import Path
import time
from rasa.nlu.model import Interpreter

load_times = []
for model_path in Path("./models_10").glob("*"):
    start = time.time()
    nlu_interpreter = Interpreter.load(model_path)
    load_times.append(time.time() - start)
print(load_times)
>>>>>>>>>
[4.209419012069702, 1.2521369457244873, 1.2520649433135986, 1.256606101989746, 1.4406280517578125, 1.234318733215332, 1.238109827041626, 1.2923908233642578, 1.4882800579071045]
2

I think it’s Tensorflow.

When you run rasa from the command line there’s also a unconfortable waiting time. That’s because tensorflow needs to load. You can check locally, the tensorflow payload on disk is typically about 700MB in your virtualenv.

1 Like