I just ran into a similar issue running the google assistant demo. I have not had any luck but, I did set the longitute and latitude manaully, and that seemed helped to make the bot respond. If that is any clue. From actions.py line 35-39
get_origin = requests.post(
"https://www.googleapis.com/geolocation/v1/geolocate?key={}".format(key)).json()
print(get_origin)
origin_lat = '37.7927731'
origin_lng = '-122.4054803,16'