Example of integrating video with Facebook Messenger front end to Rasa

Rasa Open Source
1

I’ve been using Facebook Messenger (FM) as a front-end to Rasa. I’ve been able to successfully show images in FM using Python custom actions. However, when I follow the instructions for showing a video in FM, I get nothing. Attached screen shot of FM input and no response.

Does anybody have a working example of displaying a video in FM triggered from a Rasa custom action?

Here is the relevant snippet of Python code from the custom action:

  media_type = media_dict[slot_dict["media"]]
  logging.warning("slot_dict['movie'] is "+str(slot_dict['movie']))
  try:
     if slot_dict["movie"] == "Ballroom Blitz":
        img = "https://www.youtube.com/watch?v=gYnRmfgKAbg"
        logging.warning("ready Mick?")
     else:
        poster_file = df_dict['movies'][df_dict['movies']['original_title'].str.contains(slot_dict['movie'])]['poster_path']
        logging.warning("poster_file is "+str(poster_file.iloc[0]))
        img = image_path+str(poster_file.iloc[0])
     logging.warning("img is "+str(img))
     logging.warning("latest_input_channel "+str(tracker.get_latest_input_channel()))
     logging.warning("media_type is "+str(media_type))
     # special incantation required to get a graphic to display - none of the 3 other half-baked recommendations worked
     if tracker.get_latest_input_channel() == 'facebook':
        message = {
           "attachment": {
              "type": media_type,
              "payload": {
                 "url": img
              }
           }
        }
        
        logging.warning("about to utter_custom_json for "+str(tracker.get_latest_input_channel()))
        dispatcher.utter_custom_json(message)
     else:
        dispatcher.utter_message(img)
  except:
     if debug_on:
        raise
     dispatcher.utter_message("could not find media - please try another query")

Here is the output in the log file for the utterance “show me the video for Ballroom Blitz”:

    2019-11-20 05:57:18 WARNING  root  - IN CONDITION BY MEDIA
2019-11-20 05:57:18 WARNING  root  - slot_dict['movie'] is Ballroom Blitz
2019-11-20 05:57:18 WARNING  root  - ready Mick?
2019-11-20 05:57:18 WARNING  root  - img is https://www.youtube.com/watch?v=gYnRmfgKAbg
2019-11-20 05:57:18 WARNING  root  - latest_input_channel facebook
2019-11-20 05:57:18 WARNING  root  - media_type is video
2019-11-20 05:57:18 WARNING  root  - about to utter_custom_json for facebook
2019-11-20 05:57:18 WARNING  root  - COMMENT: end of transmission validated![fm_heat|690x91](upload://evB91Yxs9BppaEcORqnThm8mSAK.jpeg)
2

@ryanmark Can you please format your post? The current format makes it quite hard to read it.

3

thanks for the tip - done

4

Awesome, thanks! Which value has the variable media_type? I think the problem is that the facebook channel currently cannot handle videos - a PR for that would be relatively quick I think. Would you be up for taking that on? I can help you on the way :slight_smile:

5

Thanks - value of media_type is “video”

6

Thanks for your answer. Would be up for a quick PR which adds this functionality ? I can also help you if you want :slight_smile:

7

Yes, I would be happy to do a PR. Can you point me to starter info for defining one? I can take a first crack and share the draft with you. Thanks for the response and the recommendation. I think that being able to show video in the context of FM will be a showstopper for us so we’re motivated to do what we can to help make it happen.

8

@ryanmark Thanks for your help, that would be amazing! Just tag me in github (@wochinge ) :tada:

9

Has this feature been added to the repo now?

10

Just checked, and I don’t think so.

1 Like