Rasa integration with website

Hi, How can use the web address of the Rasa webchat (HTML page) to scrape information from this HTML ? or is there any way to get (collect) information from the rasa conversation?

Hi,
what kind of information do you want to collect? The conversations are stored and you can review them for instance by using Rasa X. If you are looking to extract very specific informations you can use a form, fill the slots and write them into a Google Sheet.
Could you provide some more information on what you are trying to achieve?
Ciao, Vio

Hi @SUMANA,

I’m guessing that you want to search the html after a specific question of the user. You can use a custom action for that. Specify the intent, and then define a custom action to parse the html. I usually use BeautifulSoup to parse html pages.

You can just use rasa open source. You do not need to use rasa x.

Below is some code from on of my recent projects. Note that this code is not from the actions.py file, since I have a separate py file to parse the desired html page.

from bs4 import BeautifulSoup
import requests

def ParseHTML():
	url = f"your_url_here"
	page = requests.get(url)
	status = page.status_code
	valid = 0
	if status == 200:
		soup = BeautifulSoup(page.text, 'html.parser')
		delivered = soup.find_all(class_="name_of_a_desired_class_in_html")
		if delivered:
			valid = 1
		else:
			valid = 0
	else:
		valid = -2
	return valid

Hope it helps.

Thanks!

Hi Violetta, Thanks for your response. My project is, to integrate RASA with an external Python code. In elaboration, in the integration, the RASA command will be used as an input in the python code.

As an example, in the conversation, I asked RASA to change the value of apple to 3. The RASA command is: The value of an apple is 3. And this “value of apple” is the input of the Python code. From the RASA command, it will take the value which is 3 in that case, and run the algorithm (Python code).

In that case, I made a rasa webchat for the conversation and tried to apply a python code to get the specific information for example the value of the apple. However, when I am using the web address of the webchat (HTML page) in the python code as shown below, it’s giving me an error. Code: import requests

url = “file:///C:…index.html”

print(url)

r = requests.get(url)

Error: InvalidSchema: No connection adapters were found for “file:///C:…index.html”

I am new in Rasa and not sure whether I am heading the right way or not.

Hi Kostis,
Thanks for your response and the code. Yes, I want to search the HTML after a specific question of the user. However, still, I am unable to get the information from the HTML page (rasa webchat).

Hi @SUMANA,
I am sorry, I am still not sure, I got you correctly. :woman_shrugging: But couldn’t you set a slot for value of the apple? And then write a custom action, that gets you that slot and writes it into a file from where you can access it? The Sara Demo Bot here collects the slots and writes them into a Google Sheets.
Hope the example might help you!
Ciao

Hi @SUMANA ,

The ‘requests’ library I mentioned in my previous post is an HTTP library. Since you have an html file locally, you can open and read the file.

f = open("file:///C:…index.html", "r")
print(f.read())

This will print the content of your file.

Afterwards, I agree with Violetta. The “value of apple” shold be stored in a slot. Afterwards, you can create a custom action to do anything you want with it, such as store it in Google Sheets, write it to a file, or anything else you want.

Hi,

Thanks @v_kostis for showing me the right track. I have one more query, can I use an external python code as a custom action and the output of the code will be the rasa command (bot uttered)?

Thanks for your suggestions.

Hi @SUMANA, it is possible, but I personally think that you could/ should include your external python code in your custom action. Nevertheless, you can call an external python file if required. The command for python 3.X is presented below:

First, import the .py file in actions.py file

import your_python_file.py

Afterwards, in your custom action, you can call the below command:

exec(open(“your_python_code.py”).read())

You can sane the above command to a variable, and then utter the variable.

Hope this helps :slight_smile:

Hi Kostis, Thank you so much. However, still I can’t solve the problem. Below is my action.py code

from typing import Any, Text, Dict, List

from rasa_sdk import Action, Tracker

from rasa_sdk.executor import CollectingDispatcher

import CV.py

class ActionHelloWorld(Action):

def name(self) -> Text:
     return "action_hello_world"

def run(self, dispatcher: CollectingDispatcher,
         tracker: Tracker,
         domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

    mu_rate = tracker.get_slot("mutation")
    co_rate = tracker.get_slot("crossover")

    print(mu_rate)
    print(co_rate)
    
    result=exec(open(CV.py).read())
    
    message = “The output score is = {}”.format(result)

    dispatcher.utter_message(text=message)

    return [AllSlotsReset ()]

I need the value of the two variables “mu_rate” and “co_rate” to execute the CV.py code wherein the CV.py code I define: m=mu_rate and c=co_rate But it’s not working. Do I need a Followup action in this case.

It will be highly appreciated if you kindly help me.

Hi @SUMANA,

Could you please elaborate on the below?

You mean that CV.py take as arguments “mu_rate” and “co_rate”? Meaning those variables can change the outcome of the CV.py?

Yes, you are right.

Hi All, I need help. My stories (“parameter tuning path”) are not working properly. My codes are below:

nlu.yml: version: “3.1”

nlu:

  • intent: greet

    examples: |

    • hey
    • hello
    • hi
    • hello there
    • good morning
    • good evening
    • moin
    • hey there
    • let’s go
    • hey dude
    • goodmorning
    • goodevening
    • good afternoon
  • intent: goodbye

    examples: |

    • cu
    • good by
    • cee you later
    • good night
    • bye
    • goodbye
    • have a nice day
    • see you around
    • bye bye
    • see you later
  • intent: affirm

    examples: |

    • yes
    • y
    • indeed
    • of course
    • that sounds good
    • correct
  • intent: deny

    examples: |

    • no
    • n
    • never
    • I don’t think so
    • don’t like that
    • no way
    • not really
  • intent: mood_great

    examples: |

    • perfect
    • great
    • amazing
    • feeling like a king
    • wonderful
    • I am feeling very good
    • I am great
    • I am amazing
    • I am going to save the world
    • super stoked
    • extremely good
    • so so perfect
    • so good
    • so perfect
  • intent: mood_unhappy

    examples: |

    • my day was horrible
    • I am sad
    • I don’t feel very well
    • I am disappointed
    • super sad
    • I’m so sad
    • sad
    • very sad
    • unhappy
    • not good
    • not very good
    • extremly sad
    • so sad
  • intent: parameter_tuning_mu

    examples: |

  • intent: crossover_parameter_tuning

    examples: |

  • intent: bot_challenge

    examples: |

    • are you a bot?
    • are you a human?
    • am I talking to a bot?
    • am I talking to a human?

Domain.yml: version: “3.1”

intents:

  • greet
  • goodbye
  • affirm
  • deny
  • mood_great
  • mood_unhappy
  • bot_challenge
  • parameter_tuning_mu
  • crossover_parameter_tuning

entities:

  • parameter
  • value

slots:

parameter:

type: float

mappings:

- type: from_entity

  entity: parameter

value:

type: float

mappings:

- type: from_entity

  entity: value

actions:

  • action_hello_world

responses:

utter_greet:

  • text: “Hey! How are you?”

utter_cheer_up:

utter_did_that_help:

  • text: “Did that help you?”

utter_happy:

  • text: “Great, carry on!”

utter_goodbye:

  • text: “Bye”

utter_ask_mutation_rate:

  • text: “What will be the next parameter?”

utter_ask_crossover_rate:

  • text: “What will be the next value?”

utter_next:

  • text: “ok, what is next?”

utter_iamabot:

  • text: “I am a bot, powered by Rasa.”

session_config:

session_expiration_time: 60

carry_over_slots_to_new_session: true

Stories.yml: version: “3.1”

stories:

  • story: happy path

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_great

    • action: utter_happy

  • story: sad path 1

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_unhappy

    • action: utter_cheer_up

    • action: utter_did_that_help

    • intent: affirm

    • action: utter_happy

  • story: sad path 2

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_unhappy

    • action: utter_cheer_up

    • action: utter_did_that_help

    • intent: deny

    • action: utter_goodbye

  • story: parameter tuning path

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_great

    • action: utter_happy

    • intent: parameter_tuning_mu

    • action: utter_ask_mutation_rate

    • action: utter_next

    • intent: crossover_parameter_tuning

    • action: utter_ask_crossover_rate

    • action: action_hello_world

action.py: from typing import Any, Text, Dict, List

from rasa_sdk.events import SlotSet

from rasa_sdk import Action, Tracker

from rasa_sdk.executor import CollectingDispatcher

class ActionHelloWorld(Action):

def name(self) -> Text:

     return "action_hello_world"

def run(self, dispatcher: CollectingDispatcher,

         tracker: Tracker,

         domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

    mutation = tracker.get_slot("parameter")

    crossover = tracker.get_slot("value")


     

    print(mutation)

    print(crossover)

    dispatcher.utter_message(text="All done")

    return [SlotSet()]

Bot response stopped when I asked something from the intent parameter_tuning_mu.

Bot response: Bot loaded. Type a message and press enter (use ‘/stop’ to exit): Your input → hello
Hey! How are you? Your input → so good
Great, carry on! Your input → set the mutation
Your input →


Your help is highly appreciated.

Hi Kostis, it will be highly appreciated if you kindly help me to solve the problem. The slot value only taking ‘None’. I went through several discussion in Rasa forum. However, can’t solve my problem. I put my queries in Rasa forum but did not get any response yet. I have tried all possibility to fix it but nothing working. I am not sure what I am missing.


Slots in domain file: entities:

  • parameter

slots:

parameter:

type: float

min_value: 0.00

max_value: 1.00

mappings:

- type: from_entity

  entity: parameter

Stories:

  • story: parameter tuning path

    steps:

    • intent: greet

    • action: utter_greet

    • intent: mood_great

    • action: utter_happy

    • intent: parameter_tuning_mu

    • action: utter_ask_mutation_rate

    • intent: new_parameter

    • action: action_ask_parameter

    • action: utter_next


action.py: from typing import Any, Text, Dict, List

from rasa_sdk.events import SlotSet

from rasa_sdk import Action, Tracker

from rasa_sdk.executor import CollectingDispatcher

class ActionAskParameter(Action):

def name(self) -> Text:

     return "action_ask_parameter"

def run(self, dispatcher: CollectingDispatcher,

         tracker: Tracker,

         domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

    parameter = tracker.get_slot("parameter")

    float_parameter = parameter

    print(float_parameter)

    type(float_parameter)

    if  float_parameter is not None:

       

       parameter = ("parameter", parameter)

       
 
   

    return []

022-06-20 11:58:59 DEBUG rasa.core.processor - Policy prediction ended with events ‘[<rasa.shared.core.events.DefinePrevUserUtteredFeaturization object at 0x0000027255ACD128>]’. 2022-06-20 11:58:59 DEBUG rasa.core.processor - Action ‘action_ask_parameter’ ended with events ‘[]’. 2022-06-20 11:58:59 DEBUG rasa.core.processor - Current slot values: parameter: None session_started_metadata: None

Please help me.