Hi everyone 
Quick disclosure first: I work on a small SDK called Autoplay. We just published a Rasa tutorial, and I’d love feedback from this community on it - but more than that, I’m trying to understand what you think makes a Rasa chatbot actually feel helpful to users, not just technically correct.
Here’s the thing that’s been bothering me for a while, and I wanted to talk about it openly with people who build these bots for a living.
The same conversation, two ways:
A user types into your chat widget: “how do I add a teammate?”
Most chatbots - Rasa, Intercom, Botpress, doesn’t matter - will reply something like:
Go to Settings → Team → Invite member, then enter their email and pick a role.
Which is technically correct, and totally useless if the user is already on the Team page with the invite modal half-filled and just doesn’t know what to put in the role dropdown. They didn’t need a walkthrough. They needed someone to look over their shoulder and say, “you’re nearly there - just pick a role and hit Send.”
That second answer is the one users actually want. But the bot has no way to give it, because it has no idea where the user is or what they just clicked. It only sees the question.
This is the problem I keep coming back to. A great chatbot answer isn’t about better NLU or a smarter LLM. It’s about the bot understanding the situation the question was asked in. CSAT scores I’ve seen in CS deployments live and die on this - not on how clever the response was, but on whether it skipped the steps the user had already done and met them where they actually were.
What I’d love to hear from this community:
-
For those of you running Rasa chatbots in production - does this resonate? Do you see your bot give “technically right but useless” answers because it can’t see the user’s context?
-
If you’ve tried to fix it, what did you do? Pass page URL through to the bot? Stuff metadata into the chat session? Something cleverer? I’m curious what the homegrown approaches look like.
-
Outside of being able to “see” what the user is doing - what’s the single biggest thing that’s improved your bot’s helpfulness over time?
-
And separately: in your experience, do users actually want a bot that’s more aware of what they’re doing, or does it cross a creepy line? I have my own opinion, but I’d love to hear yours.
Where Autoplay fits in (briefly, and only because it’s directly relevant): The reason I’m thinking about this is that we built a small SDK whose entire job is to feed a chatbot real-time context about what the user is doing in your product - which page, which buttons, where they got stuck, why they got stuck, what they were trying to achieve - so the bot can give the “you’re nearly there” answer instead of the walkthrough.
We just shipped a tutorial showing how to wire it into Rasa, end-to-end: developers.autoplay.ai/recipes/rasa. It’s open SDK, self-hosted, your LLM keys never leave your infra. Honestly, the most useful thing for me would be if a few of you skimmed it and told me what’s wrong with it, or what’s missing, or what wouldn’t work in your setup. We don’t have years of Rasa-in-production scars yet, but you do.
But honestly, the product is the smaller half of why I’m posting. The bigger question - how a chatbot becomes genuinely helpful instead of just available - is interesting regardless of what stack you use.
Thanks for reading 
- looking forward to hearing how you think about this.