Hi All, I’m integrating gpt-3.5 llm using langchain in the custom action. rasa server is up and running and our frontend chatbot is talking to the server via http. Anyone knows or can suggests a way to render streaming response from llm? I can see I get streaming response from openai api but can’t think of how it could be transferred as real-time http response to the front end. Any opinion would be appreciated!! Thanks.

rasa.core.channels.rest I found this doc, is this a way to go?? If this is the case that means I need to upgrade my rasa version to 3…

Hi @jpark2111 Yes. If it is not available in Rasa 2.x then you’ll have to upgrade to Version 3.x

Maybe you need a websocket or socketio, I use this to send messages to the frontend.

I am using OpenAI API with rasa for generating responses and I am getting responses as well. Now i want to get streamed responses in my VS Code terminal just like ChatGPT. Is there any way to do it? If it cane be done with: rasa.core.channels.rest , how to use it in my actions.py file. Or are there any other ways ?? I am using rasa version 3.5.0

How to stream real time openai response using rasa?

Rasa Open Source

avatar (avatar) March 19, 2024, 9:17pm 8

anybody able to figure this out?

Topic		Replies	Views
Implementing Streaming Responses in Rasa for a ChatGPT-like Experience Rasa Open Source	7	1500	January 14, 2025
Streaming response for custom actions Rasa CALM	2	428	January 14, 2025
How can I have access to the rasa response inside the HTML file and send it to my own robot? Rasa Open Source	0	380	April 5, 2022
Stream receiving messages with stream=true in PHP Rasa Open Source	1	474	November 10, 2023
Rasa Server + Front End with REST API Rasa Open Source	3	1044	November 12, 2020

How to stream real time openai response using rasa?

Related topics