Confused how elements /custom message objects work

Rasa Open Source
1

So I am using a non python backend and I post to rasa server my user message text. Then Rasa hits my /nlg endpoint and I return the bot response. And then rasa returns that bot response to my original post request – that is how I have seen this work.

I tried adding an element (my custom object extending element) to my bot response in the /nlg endpoint but saw strange behavior. The bot response is supposed to have a text field with the bot reply, as well as an element of my own custom message object. There were errors when I didn’t have a button field on my element. And if my bot response had 2 elements on it, in my original post request the bot responses I received included 2 different bot responses, a text field with the title of my element, and none of the original custom message attributes I sent.

Is there a way I can respond on the /nlg endpoint with my custom object on the bot response, and then get the same response in my original post request so that I can use it for my front end?

Thanks

2

@mickeybarcia Maybe not helpful, but did you check this out? Bot Responses

I think (haven’t tested it) you can just respond with something like this:

{
    "text": "",
    "buttons": [],
    "image": null,
    "elements": [],
    "attachments": []
}

If you need your object in the frontent, you could just JSON serialize it into the “text” attribute and then deserialize it again in your frontend, e.g.

{
    "text": "{"a": 1}"
}
3

I am talking about the /nlg endpoint. I want to be able to send the same type of custom object in a bot response whether it is an action or utter but I am still trying to figure out if a custom input channel can do that

4

Sorry, mixed it up. Edited my response above.

Also: If you want a custom representation, you’d need to subclass the OutputChannel and modify it to your requests. What frontend / output channel are you using?

5

My java backend post messages to /webhook/rest/webhook and also holds the /nlg and /webhook endpoints that rasa hits. I need to respond to /nlg with my custom object, and then receive the custom object at the original post so I can reply to my frontend with that. The documentation is very incomplete on how I would make a channel to do all this. I want to use elements but they require a specific structure and edit my bot response.

6

I’m not sure I understand your problem correctly. Afaik this is the typical setup:

                 NLU
                  |

Frontend - Channel - Rasa Core - Action Server - Backend(s)

                  |
                 NLG

Your backend shouldn’t post anything to core or your action server directly. Instead the flow should be like this:

  1. User says something
  2. Input chanel’s webhook receives message
  3. Input channel handles / parses message
  4. Sends message to Core’s dialogue management system
  5. Core processes the message using the NLU
  6. Predicts next action based on tracker state
  7. a) If next action is a template it sends the template name (along with other data) to your NLG endpoint, which returns a response (like the first message below) b) If next action is a custom action Core sends a POST to your action server. The action server responds (like the second message below) and also creates a message like the one above:

{ “events”: [ { “event”: “slot”, “timestamp”: null, “name”: “concerts”, “value”: [ {“artist”: “Foo Fighters”, “reviews”: 4.5}, {“artist”: “Katy Perry”, “reviews”: 5.0} ] } ], “responses”: [ {“text”: “Foo Fighters, Katy Perry”} ] }

  1. Core then hands a this dict to the OutputChannel which processes the response, e.g. Facebook’s output channel converts it to something like this:

{ ‘messaging_type’: messaging_type, ‘recipient’: { ‘id’: entry[‘sender’][‘id’] }, ‘message’: { ‘text’: “Foo Fighters, Katy Perry” } }

  1. The OutputChannel then typically just POSTs this object to your frontend’s webhook

You can also use the Callback channel to POST this object to a webhook of your choice.

So if your backend wants to respond with a certain object you can just pass it as JSON object. The custom action can then send the JSON object as text (as in 7. second message) or you may need to customize the output channel part. The channel will then return this message back to the frontend (or POST it to a webhook if you’re using a CallbackInput). Then your frontend just needs to deserialize the string into a object and do with it whatever it wants (e.g. display it as a button, text, …).

Does that help?

3 Likes
7

I understand the flow of the new rasa. Sending the json object as a string seems odd but I tried looking at how to have an output channel to send my actual object. I will look into both options. Thanks for all your help.