Detection and OCR API

We are trying to create a chatbot that includes 2 generators: 1 detection and 2 OCR. We don’t know how to add the APIs for these 2 models in the file this is the function of the file: |import requests import json from io import BytesIO from PIL import Image from typing import Any, Text, Dict, List import base64

from rasa_sdk import Action, Tracker from rasa_sdk.executor import CollectingDispatcher class ActionExtractText(Action): def name(self): return “action_extract_text”

def run(self, dispatcher, tracker, domain):
    # Get the URL of the image from the user input
    image_url = tracker.get_slot("image_url")

    # Send a GET request to the API to retrieve the image
    image_response = requests.get(image_url)

    # Convert the image response to a PIL Image object
    image =

    # Convert the image to base64-encoded bytes
    buffered = BytesIO(), format="JPEG")

    encoded_image = base64.b64encode(buffered.getvalue()).decode("utf-8")

    # Send a POST request to the API to classify the image
    api_url = "OCR-API Link"
    data = json.dumps({"image": encoded_image})
    response =, data=data)

    if response.status_code == 200:
        result = response.json()
        dispatcher.utter_message(f"The text is {result}.")
        dispatcher.utter_message("Sorry, I could not read the text at this time.")

    return []

class ActionStoreImageURL(Action): def name(self) → Text: return “action_store_image_url”

def run(self, dispatcher: CollectingDispatcher, tracker: Tracker, domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
    # Extract the image URL entity from the user's input
    image_url = next(tracker.get_latest_entity_values("image_url"), None)

    # Store the image URL in the "image_url" slot
    if image_url:
        tracker.slots["image_url"] = image_url
        dispatcher.utter_message(text=f"Got it, I'll use this image: {image_url}")
        dispatcher.utter_message(text="Sorry, I didn't catch the image URL. Can you please try again?")

    return []

and this is the response we get:

2023-05-02 19:52:56 INFO     root  - Connecting to channel 'cmdline' which was specified by the '--connector' argument. Any other channels will be ignored. To connect to all given channels, omit the '--connector' argument.
2023-05-02 19:52:56 INFO     root  - Starting Rasa server on
2023-05-02 19:52:58 INFO     rasa.core.processor  - Loading model models\20230502-195122-giant-arrow.tar.gz...
2023-05-02 19:53:51 WARNING  rasa.shared.utils.common  - The UnexpecTED Intent Policy is currently experimental and might change or be removed in the future 🔬 Please share your feedback on it in the forum ( to help us make this feature ready for production.
2023-05-02 19:54:02 INFO     root  - Rasa server is up and running.
Bot loaded. Type a message and press enter (use '/stop' to exit):
Your input ->  Hi
Hey! How are you?
Your input ->  the image url is
Sorry, I could not read the text at this time.
Got it, I'll use this image:

Do you see errors in the action server log? How long does it take for this action to run?

I suspect this could be a long running action and that Rasa will hit the timeout waiting for the response. I think the limit is 20 seconds.

You may have to handle the image processing asynchronously and use the Rasa external events feature to send a response when the process is done.