Performance issue with action end point

We are facing performance issue in RASA. For 85 utterances one by one through automated script it took around ~185 seconds. One utterance response is taking more than around 2 seconds. Our custom actions are accessed by action_endpoint: url: http://localhost:5055/webhook. We could see that webhook url call from rasa/core/action.py to rasa_sdk/endpoint.py is taking more than 1 second, eventhough both RASA server and Action server are running in same server. We tried calling the rasa_sdk/endpoint executor.run() directly from core/action.py and then the execution time got reduced and we are getting response in 1 sec. But we don’t think this is the right approach. Do we have any way to reduce this delay ?

2 Likes

Hi @shyamzacharia,

welcome to the community!

  • With which rasa/ rasa-sdk versions are you running?
  • What does your custom action do? Does it do further API request?
  • How large is your tracker (how many events)?
  • Is the delay always the same or does it increase the longer the conversation goes on?

Hi Tobias, Thanks for your ask. Please find the below inline replies:

With which rasa/ rasa-sdk versions are you running? • rasa version:-1.5.1 • rasa_sdk version:1.5.1

What does your custom action do? Does it do further API request? Custom action will just formulate response with/without some rules. But there are no API calls happening from custom actions. We could see that custom action response is not even taking 1-2 milliseconds.

How large is your tracker (how many events)? • Number of events in the train data, comes between 25-30

Is the delay always the same or does it increase the longer the conversation goes on? • Dont thinks so. The initial conversation is having around 1 seconds delay. But in the final conversation after 7-8 utterance, we are getting around 1 seconds delay. In between start and end we could see a delay of 2 seconds also intermittently. We also tried the tracker store with Redis as well, but that did not improve the throughput at all.

Could you please suggest a workaround to avoid the delay ?

Hi, thanks for your information. A conversation history with 25-30 events is actually not large. Which tracker store are you using? Do you have an endpoint in place? Is there a way that you could check at which point the delay is happening? How many stories / nlu examples do you have? And did you change the max_history parameter?

@shyamzacharia Did You find a solution to this issue?? I am facing the same with Rasa 1.4.3 version. @Tobias_Wochinger I dont think there is any issue with any of those things you suggested.

My normal pre defined actions like “utter_greet” etc are getting replied quickly. But its taking approx 2-3 seconds when a custom action is called.

It has nothing to do with the content of the Actions.py .For me even a small custom action like this-

class ActionCheckContract(Action):
def name(self):
    return "contract_action"

def run(self, dispatcher, tracker, domain):
    intent = tracker.latest_message["intent"].get("name")
    if intent == "inform_contract_owner":
        dispatcher.utter_template("utter_change_mind", tracker)
        return []

**Takes the equal amount of time to response in compared to this Action.py (5% of Actions,py)

class ActionCheckContract(Action):
def name(self):
    return "contract_action"

def run(self, dispatcher, tracker, domain):
    client = pymongo.MongoClient("localhost", 27017)
    db = client.in_dev
    collection = db.Contract
    entity = tracker.get_latest_entity_values("contract_name")
    entity_DB = next((j for j in collection.find({"_id": ObjectId(entity)})), None)
    print(entity_DB)
    intent = tracker.latest_message["intent"].get("name")
    intents_check_dict = {"inform_contract_owner": 'contractOwner', "inform_contract_number": "contractNumber",
                          "inform_contract_risk_rating": "contractRisk",
                          "inform_contract_annual_spend": "annualSpend",
                          "inform_business_risk_rating": "riskRating", "inform_contract_value": "value",
                          "inform_contract_renewal_period": "renewalPeriodId",
                          "inform_contract_liquidation_damages": "liquidationDamage",
                          "inform_contract_minimum_spend": "minimumSpend",
                          "inform_contract_addressable_spend": "addressableSpend",
                          "inform_contract_policy_number": "policyNumber"}

    def intent_lookup(intent):
        return intents_check_dict.get(intent)

    c = intent_lookup(intent)
    if c is not None:
        print(intents_check_dict)
        dispatcher.utter_message(str(entity_DB[c]))
        return []

It takes the same time ,even if I alter the interface file to change a function and get my entity through text search API locally. So could u please suggest a solution to this? @akelad @Juste @alexweidauer @souvikg10 @erohmensing Please help me out on this.

@shyamzacharia Did you find a solution to this?

@pranay_raj what does your setup look? Where are you running the action server and the rasa server? Are they on the same machine?

@akelad Thank you so much for replying back. My deployement date is approaching :stuck_out_tongue: . I could really use some help of yours. My setup as in?what exactly you mean by setup?

My config.yml-

pipeline:         
name: "WhitespaceTokenizer"*
name: "CountVectorsFeaturizer"*
name: "EmbeddingIntentClassifier"*

policies:
      - name: MemoizationPolicy
      - name: KerasPolicy
      - name: MappingPolicy

I am not using any entity extractor/classifier because Rasa’s Entity extraction for named entity recognition was not efficient enough for my use case, so i went for text search approach.

And both the servers are run separately locally as of now from cmd. The predefined functions gives reply within a second.But custom actions take time from 2-3 seconds.

I have tried executing my project with different Rasa versions.- 1.0.0.a1 , 1.3.1, 1.4 , 1.5 , 1.6.1(latest) Out of which, in version 1.4 and 1.5 I was getting replies from custom action even after 4-5 seconds. In the latest version also it takes approx 2 seconds or more to show result.

So this is all just running on the command line? That’s strange that it takes so long. Can you show me the logs of both rasa and the action server? Specifically i’m interested in the time stamps of the action being predicted and the action server executing the action

This is Rasa shell debug log-

@akelad

Yeah! So, it takes approximately 2 seconds between- (27-29 seconds)

20-01-20 19:23:27 DEBUG rasa.core.actions.action - Calling action endpoint to run action ‘supplier_action’.

2020-01-20 19:23:29 DEBUG rasa.core.processor steps.

This is Action Server log- The request to Action server is also received at : 29 seconds time stamp.(2 seconds late)

@akelad Ma’am, can you please direct me as to what I should do to reduce the request time.? And why exactly is it happening?

@pranay_raj we’re looking into it on our end, but we’ve never seen this before so it’s entirely possible it’s something to do with your machine. Do you have the option of trying this on a different machine?

@akelad OK, I shall try on different machine and let you know within an hour. TYSM for replying.

@akelad Hey. So I checked it on another system as well, it has the same lagging problem of 2 seconds. What may be the reason behind this? The same Issue persist. From the other system as well, after the call is made from core.actions to predicted action, the request is received after two seconds in action server log

@akelad Can you please tell me the files or classes or functions which might be causing this delay. As i can see the request received itself is delayed. So what may be the exact cause or which call could be lagging?? please reply.

Hm, that’s odd. Few more questions:

  • have you modified the rasa or rasa-sdk code in any way?
  • which versions are you using?
  • what commands are you using to start rasa and the rasa-sdk?

@akelad . Thank You for replying.

  1. I did modify rasa-sdk in my system in the interface file, but i guess thats irrelevant since I didnt change anything on another machine.

Currently I am using 1.6.1 of Rasa 3. rasa shell --debug and rasa run actions are the command i am using And also rasa run -m models --enable-api --cors “*” --debug

are you running rasa run -m models... and rasa shell simultaneously?

@akelad No only when I have to use chatbot on UI i run rasa run… else for normal testing i use rasa shell

As far as I can trace the call to action endpoint is getting delayed. May I know what happens after

What calls are made ?? And Just what may be probably the reason???