Disable_eager_execution with rasa not working

Rasa Open Source
1

I have created a pipeline, in which I am using Rasa as python library, and predict the response/intent if intent is out_of_scope then i process the request using deep learning model with elmo embeddings.

So to load the TF graph for that i need to disable eager execution , So i used

import tensorflow.compat.v1 as tf

tf.compat.v1.disable_eager_execution()

but by disabling this my rasa model stopped working, now when i try to use

queryResponseList = await agent.handle_text(query)

it shows

Traceback (most recent call last):
  File "C:\MyRasaProj\env37\lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 371, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "C:\MyRasaProj\env37\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 59, in __call__   
    return await self.app(scope, receive, send)
  File "C:\MyRasaProj\env37\lib\site-packages\uvicorn\middleware\debug.py", line 96, in __call__
    raise exc from None
  File "C:\MyRasaProj\env37\lib\site-packages\uvicorn\middleware\debug.py", line 93, in __call__
    await self.app(scope, receive, inner_send)
  File "C:\MyRasaProj\env37\lib\site-packages\fastapi\applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\middleware\errors.py", line 181, in __call__       
    raise exc from None
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\middleware\errors.py", line 159, in __call__       
    await self.app(scope, receive, _send)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\exceptions.py", line 82, in __call__
    raise exc from None
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "C:\MyRasaProj\env37\lib\site-packages\starlette\routing.py", line 52, in app
    response = await func(request)
  File "C:\MyRasaProj\env37\lib\site-packages\fastapi\routing.py", line 215, in app
    dependant=dependant, values=values, is_coroutine=is_coroutine
  File "C:\MyRasaProj\env37\lib\site-packages\fastapi\routing.py", line 149, in run_endpoint_function      
    return await dependant.call(**values)
  File "C:\MyRasaProj\api\chat_service.py", line 61, in process
    botResponse = await RasaBot.process(query, clientId)
  File "C:\MyRasaProj\bots\rasa_bot.py", line 66, in process
    queryResponseList = await agent.handle_text(query)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\agent.py", line 623, in handle_text
    return await self.handle_message(msg, message_preprocessor)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\agent.py", line 536, in handle_message
    return await processor.handle_message(message)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\processor.py", line 91, in handle_message
    tracker = await self.log_message(message, should_save_tracker=False)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\processor.py", line 322, in log_message
    await self._handle_message_with_tracker(message, tracker)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\processor.py", line 587, in _handle_message_with_tracker
    parse_data = await self.parse_message(message, tracker)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\processor.py", line 566, in parse_message
    text, message.message_id, tracker, metadata=message.metadata
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\core\interpreter.py", line 145, in parse
    result = self.interpreter.parse(text)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\nlu\model.py", line 453, in parse
    component.process(message, **self.context)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\nlu\classifiers\diet_classifier.py", line 963, in process
    out = self._predict(message)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\nlu\classifiers\diet_classifier.py", line 878, in _predict
    return self.model.rasa_predict(model_data)
  File "C:\MyRasaProj\env37\lib\site-packages\rasa\utils\tensorflow\models.py", line 270, in rasa_predict  
    return tf_utils.to_numpy_or_python_type(self._tf_predict_step(batch_in))
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\def_function.py", line 814, in _call 
    results = self._stateful_fn(*args, **kwds)
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\function.py", line 2829, in __call__ 
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\function.py", line 1938, in _call_flat
    flat_outputs = forward_function.call(ctx, args_with_tangents)
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\eager\function.py", line 579, in call      
    executor_type=executor_type)
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\ops\functional_ops.py", line 1175, in partitioned_call
    args = [ops.convert_to_tensor(x) for x in args]
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\ops\functional_ops.py", line 1175, in <listcomp>
    args = [ops.convert_to_tensor(x) for x in args]
  File "C:\MyRasaProj\env37\lib\site-packages\tensorflow\python\framework\ops.py", line 1465, in convert_to_tensor
    raise RuntimeError("Attempting to capture an EagerTensor without "
RuntimeError: Attempting to capture an EagerTensor without building a function.

this is is how i am loading model

agent = Agent.load(
                model,
                action_endpoint=EndpointConfig(ACTION_ENDPOINT)
            )

is it possible to make this working?

I am using rasa==2.4.1

2

Hey @ermarkar, I’ll answer here on the forum as it’s more convenient. Do you mind deleting the duplicate StackOverflow question? :slightly_smiling_face:

First things first: What version of Rasa are you using? Second, could you share a more complete code example so that I can see what parts of Rasa you’re importing and how you use them? I assume that you first do some training (while using Rasa as a library), then load the trained model and then do the await agent.handle_text(query) call as mentioned above?

3

i have updated the question for the code the version

4

@ermarkar sorry for such delay, I’m now coming back to this. When I tried to load and use a model trained from the command line, using something like this:

import rasa
from rasa.exceptions import ModelNotFound
from rasa.core.agent import Agent
import tensorflow.compat.v1 as tf

if __name__ == '__main__':
    model = "/Users/sams/new/models/20210825-152401.tar.gz"

    unpacked_model = rasa.model.get_model(model)

    # tf.compat.v1.disable_eager_execution()
    _agent = Agent.load(unpacked_model)
    query = "Hi there!"
    print(rasa.utils.common.run_in_loop(_agent.handle_text(query)))

everything worked. After uncommening the disable_eager_execution command, I got an error:

ValueError: Calling `Model.fit` in graph mode is not supported when the `Model` instance was constructed with eager mode enabled. Please construct your `Model` instance in graph mode or call `Model.fit` with eager mode enabled.

But you’re getting a different error. Are you also training your model with the eager mode disabled?

Maybe I’ll ask differently: In TensorFlow 2.0, eager mode is turned on by default. But you can still build a model and load it (like we do, if you look into the TED or RasaTransformerModel classes for examples). Do you really need to disable the eager mode? Alternatively, maybe you can disable the eager mode only once you’ve loaded the Rasa model? I wonder if this would work, given that the Rasa model’s TF graph and your model’s TF graph should be separate…

5

Thanks @SamS, I will check this too, but in my case for elmo model I created different service now, so everything got solved. Thanks a lot :slight_smile:

1 Like