Connecting RASA with Google Assistant

Hi,

I am trying to connect RASA with Google Assistant as mentioned in the blog Going beyond ‘Hey Google’: building a Rasa-powered Google Assistant

I deployed my Google assistant skill in the Google Actions. While testing its always saying “Place Finder isn’t responding right now. Try again soon”

from rasa_core.utils import EndpointConfig input_channel = GoogleConnector() nlu_interpreter = RasaNLUInterpreter(model_directory) action_endpoint = EndpointConfig(url=“http://localhost:5055/webhook”) agent = Agent.load(‘models/dialogue’, interpreter=nlu_interpreter,action_endpoint=action_endpoint) agent.handle_channels([input_channel], 5004, serve_forever=True)

i getting this error:

ERROR:flask.app:Exception on /webhooks/google_home/webhook [POST] Traceback (most recent call last): File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask\app.py”, line 2292, in wsgi_app response = self.full_dispatch_request() File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask\app.py”, line 1815, in full_dispatch_request rv = self.handle_user_exception(e) File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask\app.py”, line 1718, in handle_user_exception reraise(exc_type, exc_value, tb) File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask_compat.py”, line 35, in reraise raise value File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask\app.py”, line 1813, in full_dispatch_request rv = self.dispatch_request() File “c:\users\lenovo\appdata\local\programs\python\python35\lib\site-packages\flask\app.py”, line 1799, in dispatch_request return self.view_functionsrule.endpoint File “”, line 23, in receive payload = json.loads(request.data) File “c:\users\lenovo\appdata\local\programs\python\python35\lib\json_init_.py”, line 312, in loads s.class.name)) TypeError: the JSON object must be str, not ‘bytes’ 127.0.0.1 - - [2019-06-02 11:46:42] “POST /webhooks/google_home/webhook HTTP/1.1” 500 411 0.028898

Below is the Connector code:

class GoogleConnector(InputChannel):
@classmethod
def name(cls):
    return "google_home"

#def __init__(self):
#    self.out_channel = CustomOutput(url, access_token)

def blueprint(self, on_new_message):   
    google_webhook = Blueprint('google_webhook', __name__)

    @google_webhook.route("/", methods=['GET'])
    def health():
        return jsonify({"status": "ok"})

    @google_webhook.route("/webhook", methods=['POST'])
    def receive():
        payload = json.loads(request.data)		
        sender_id = payload['user']['userId']
        intent = payload['inputs'][0]['intent'] 			
        text = payload['inputs'][0]['rawInputs'][0]['query'] 		
        if intent == 'actions.intent.MAIN':	
            message = "<speak>Hello! <break time=\"1\"/> Welcome to the Rasa-powered Google Assistant skill. You can start by saying hi."			 
        else:
            out = CollectingOutputChannel()			
            on_new_message(UserMessage(text, out, sender_id))
            responses = [m["text"] for m in out.messages]
            message = responses[0]	
        r = json.dumps(
            {
              "conversationToken": "{\"state\":null,\"data\":{}}",
              "expectUserResponse": 'true',
              "expectedInputs": [
                {
                  "inputPrompt": {
                   "initialPrompts": [
                    {
                      "ssml": message
                    }
                  ]
                 },
                "possibleIntents": [
                {
                  "intent": "actions.intent.TEXT"
                }
               ]
              }
             ]
            })
        return r				
      		
    return google_webhook

My actoin.json is

{ “actions”: [ { “description”: “Default Welcome Intent”, “name”: “MAIN”, “fulfillment”: { “conversationName”: “welcome” }, “intent”: { “name”: “actions.intent.MAIN”, “trigger”: { “queryPatterns”:[“talk to Place Finder”] } } }, { “description”: “Rasa Intent”, “name”: “TEXT”, “fulfillment”: { “conversationName”: “rasa_intent” }, “intent”: { “name”: “actions.intent.TEXT”, “trigger”: { “queryPatterns”:[] } } }], “conversations”: { “welcome”: { “name”: “welcome”, “url”: “https://b6938789.ngrok.io/webhooks/google_home/webhook”, “fulfillmentApiVersion”: 2 }, “rasa_intent”: { “name”: “rasa_intent”, “url”: “https://b6938789.ngrok.io/webhooks/google_home/webhook”, “fulfillmentApiVersion”: 2 } } }

Can anyone please help me?

Thanks in advance.

Hi @Juste, can you please help me with this issue?

We are having problems with implementing the connector. We have updated to Rasa 1.0.x and we had to update the code rasa_core.utils changed to rasa.core.utils.

Hi Bhargav. Did you installed flask in your local system ? and Use python version 3.6.4

Hi Bunny,

The problem is with getting the request data using json.loads(request.data)

Now I changed this to user_input = request.get_json()

This is working fine, Thank you very much

Hey Bhargav,

Great to see that you managed to get the connector working. We are currently facing the same challenges as well. Could you also share your run_app.py code with us?

Would appreciate it a lot :slight_smile:

@jason Can you please mention the rasa core and nlu versions you are using?

Also please share your run_app.py so that i can check what is wrong… :blush:

sure! :slight_smile: This is the code that we are currently using. We are using version 1.0

from rasa.core.agent import Agent from rasa.core.interpreter import RasaNLUInterpreter from ga_connector import GoogleConnector from rasa.utils.endpoints import EndpointConfig

action_endpoint = EndpointConfig(url=“http://localhost:5055/webhook”) nlu_interpreter = RasaNLUInterpreter(’./NLUpath’) agent = Agent.load(’./modelpath’, interpreter = nlu_interpreter, action_endpoint=action_endpoint)

input_channel = GoogleConnector() agent.handle_channels([input_channel], 5004)

good to know, we get the following error: sanic.exceptions.NotFound: Requested URL/ not found

Are you providing the correct url in action.json? Below is the action.json i am using. Please provide the correct URL (you will get from ngrok)

{
   "actions": [
      {
        "description": "Default Welcome Intent",
        "name": "MAIN",
        "fulfillment": {
          "conversationName": "welcome"
        },
        "intent": {
          "name": "actions.intent.MAIN",
          "trigger": {
            "queryPatterns":["talk to Place Finder"]
          }
        }
      },
	  {
        "description": "Rasa Intent",
        "name": "TEXT",
        "fulfillment": {
          "conversationName": "rasa_intent"
        },
        "intent": {
          "name": "actions.intent.TEXT",
          "trigger": {
            "queryPatterns":[]
          }
        }
      }],
    "conversations": {
      "welcome": {
        "name": "welcome",
        "url": "http://2038565f.ngrok.io/webhooks/google_webhook/webhook",
        "fulfillmentApiVersion": 2
    },
      "rasa_intent": {
        "name": "rasa_intent",
        "url": "http://2038565f.ngrok.io/webhooks/google_webhook/webhook",
        "fulfillmentApiVersion": 2
    }
  }
}

if this is not working proved me the custom connector code you are using

Going beyond ‘Hey Google’: building a Rasa-powered Google Assistant is an old blog, it will not work with RASA 1.0, I am also having the same issue that @jason having, refer this ( Messages list in CollectingOutputChannel always empty · Issue #3695 · RasaHQ/rasa · GitHub ) for more detail.

Hi Jason

Can you please share the values for NLUpath & modelpath for your run_app.py ?

Thanks, Aditya

Hi Yashwanth,

Do you know which version of RASA would be compatible with the procedure mentioned in the blog entry?

-Aditya

Hi Aditya, I was using rasa core 0.13.8, rasa nlu 0.14.6 and rasa core sdk 0.12.2, and connector is working fine. I think it is compatible with version below 1.0. refer this( Google Assistant + Rasa 1.0 ), juste is working on that update.

Thank you :slight_smile:

Juste updated the tutorial for rasa 1.0 & it works. Go try again, several things have changed.