How to stream real time openai response using rasa?

Rasa Open Source
1

Hi All, I’m integrating gpt-3.5 llm using langchain in the custom action. rasa server is up and running and our frontend chatbot is talking to the server via http. Anyone knows or can suggests a way to render streaming response from llm? I can see I get streaming response from openai api but can’t think of how it could be transferred as real-time http response to the front end. Any opinion would be appreciated!! Thanks.

2

rasa.core.channels.rest I found this doc, is this a way to go?? If this is the case that means I need to upgrade my rasa version to 3…

3

Hi @jpark2111

Yes. If it is not available in Rasa 2.x then you’ll have to upgrade to Version 3.x

4

Maybe you need a websocket or socketio, I use this to send messages to the frontend.

5

I am using OpenAI API with rasa for generating responses and I am getting responses as well. Now i want to get streamed responses in my VS Code terminal just like ChatGPT. Is there any way to do it? If it cane be done with: rasa.core.channels.rest, how to use it in my actions.py file. Or are there any other ways ??

I am using rasa version 3.5.0

7

Any solution to this? Thanks!

8

anybody able to figure this out?