{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sify Livewire chatbot Demo\n",
"### Installations --Python 3.6\n",
"1. Rasa NLU\n",
"2. Rasa Core (pip install rasa-core)\n",
"3. SpaCy Language Model"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"rasa_nlu: 0.15.1 rasa_core: 0.14.5\n",
"Loading spaCy language model...\n",
"Hello world!\n"
]
}
],
"source": [
"import logging, io, json, warnings\n",
"logging.basicConfig(level=\"INFO\")\n",
"warnings.filterwarnings('ignore')\n",
"\n",
"import rasa_nlu\n",
"import rasa_core\n",
"import spacy\n",
"\n",
"print(\"rasa_nlu: {} rasa_core: {}\".format(rasa_nlu.__version__, rasa_core.__version__))\n",
"print(\"Loading spaCy language model...\")\n",
"print(spacy.load(\"en\")(\"Hello world!\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Teaching the bot to understand user inputs using Rasa NLU\n",
"### Preparing the NLU Training Data"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing 'nlu_md' (str) to file 'nlu.md'.\n"
]
}
],
"source": [
"nlu_md = \"\"\"\n",
"## intent:greet\n",
"- hey\n",
"- hello there\n",
"- hi\n",
"- hello there\n",
"- good morning\n",
"- good evening\n",
"- hi chatbot\n",
"- hey there\n",
"- let's go\n",
"- hey dude\n",
"- goodmorning\n",
"- goodevening\n",
"- good afternoon\n",
"\n",
"## intent:LIVEWIRE\n",
"- livewire\n",
"- livewire\n",
"- what is livewire\n",
"- define livewire\n",
"- context of livewire\n",
"- could you please give what do you meant livewire\n",
"- content of livewire \n",
"- what do you meant livewire?\n",
"- say something about livewire\n",
"- about livewire\n",
"- brief about livewire \n",
"\n",
"## intent:Course_Launch_Issue\n",
"- Error in the assessment launch\n",
"- error in the assessment launch\n",
"- Issue while taking the assessment\n",
"- Issue with the reference material mapping in Curriculum\n",
"- Gentle reminder -Get Enabled, Get Certified || NOC Services\n",
"- Gentle reminder \n",
"- Issue with the reference material\n",
"- Issue with the Question Display\n",
"- Unable to access the assessment - TCERT-0016-CS01-V1-0-2018\n",
"- Unable to read one of the assessment post editing few questions\n",
"- unable to take one of the assessment post\n",
"- Issue with the Question Display in one of the assessment\n",
"- customer is unable to access enroll courses\n",
"- unable to access enroll\n",
"- unable to read the course\n",
"- i can't access course\n",
"\n",
"## intent:User_Login_Issue\n",
"- User is unable to login into Indostar Site\n",
"- User was getting error while logging in Indostar Site\n",
"- getting error while logging\n",
"- User unable to login into Indostar site\n",
"- User was getting error while logging\n",
"- Unable to login into the Indostar\n",
"- Unable to login\n",
"- User unable to access the application\n",
"- User unable to access\n",
"- i can't login\n",
"- login issue is there\n",
"- login error\n",
"- can not login\n",
"- can't login into indostar\n",
"- while logging i can get error\n",
"- login issue\n",
"- login problem\n",
"- i may facing login problems\n",
"- facing login errors\n",
"- facing login issues\n",
"\n",
"\n",
"## intent:Course_Next_Button\n",
"- User Not able to exit after completing the course \n",
"- Not able to exit\n",
"- User was not getting next button \n",
"- not getting next button \n",
"- User unable to click next button after assessment\n",
"- unable to click next button\n",
"- Next button was not working\n",
"- next button not working \n",
"- User is not moving to next page\n",
"- not moving to next page\n",
"- User is unable to move to next slide\n",
"- unable to move to next slide\n",
"- cant able to exit\n",
"- can't able to click next button\n",
"- cant able to move\n",
"- i can't move next slides\n",
"- can't process after assessment \n",
"\n",
"\n",
"## intent:Course_Creation\n",
"- Unable to edit the course\n",
"- unable to edit\n",
"- able to edit the course\n",
"- edit course\n",
"- cannot able to edit the course\n",
"- can't edit the course\n",
"- Map a course to a curriculum\n",
"- course to a curriculum\n",
"- Assessment movement from Staging to Production - AC Essentials and Troubleshooting\n",
"- Assessment movement \n",
"- Assessment movement from Staging to Production\n",
"- AC Essentials and Troubleshooting\n",
"- i can't edit the course\n",
"- cannot edit the course\n",
"\n",
"\n",
"## intent:Asssessmet_Page_Activity\n",
"- Unable to select the options in the assessment\n",
"- Unable to select the options\n",
"- User was getting error while taking assessment\n",
"- getting error while taking assessment\n",
"- User was unable to mark scenerios like yes/no while taking assessment\n",
"- unable to mark scenerios like yes/no\n",
"- User unable to drag/drop the correct answer\n",
"- unable to drag/drop\n",
"- i can't select the option in the assessment\n",
"- i facing trouble in assessment page\n",
"- can't able to mark scenerios like yes/no\n",
"- problem on error in taking assessment \n",
"\n",
"\n",
"## intent:acces_livewire\n",
"- access livewire\n",
"- how to access livewire\n",
"- how to access sify livewire\n",
"- connect livewire\n",
"- How to connect livewire\n",
"- Do you want to access livewire?\n",
"- how can i do livewire login\n",
"- login livewire\n",
"- livewire enroll \n",
"- how livewire can connect\n",
"- how could i connect livewire\n",
"- i meant want to login livewire\n",
"- livewire login\n",
"- connect access livewire\n",
"- lw\n",
"- want to login livewire\n",
"- live wire login\n",
"- login livewire\n",
"- hey show livewire login link \n",
"- enroll livewire \n",
"\n",
"## intent:app_LiveWire\n",
"- advantage of livewire\n",
"- application of livewire \n",
"- which application livewire can be used \n",
"- livewire applications\n",
"\n",
"\n",
"## intent:goodbye\n",
"- cu\n",
"- good by\n",
"- cee you later\n",
"- good night\n",
"- good afternoon\n",
"- bye\n",
"- goodbye\n",
"- have a nice day\n",
"- see you around\n",
"- bye bye\n",
"- see you later\n",
"\n",
"## intent:mood_affirm\n",
"- yes\n",
"- indeed\n",
"- of course\n",
"- that sounds good\n",
"- correct\n",
"\n",
"## intent:mood_deny\n",
"- no\n",
"- never\n",
"- I don't think so\n",
"- don't like that\n",
"- no way\n",
"- not really\n",
"\n",
"## intent:mood_great\n",
"- perfect\n",
"- not bad\n",
"- very good\n",
"- great\n",
"- i am fine\n",
"- am fine\n",
"- yeah i am fine \n",
"- amazing\n",
"- feeling like a king\n",
"- wonderful\n",
"- I am feeling very good\n",
"- I am great\n",
"- I am amazing\n",
"- I am going to save the world\n",
"- super\n",
"- fine \n",
"- ok \n",
"- good\n",
"- extremely good\n",
"- so so perfect\n",
"- so good\n",
"- so perfect\n",
"\n",
"## intent:botname\n",
"- what is your name ?\n",
"- your name pls\n",
"- your name \n",
"- say your name \n",
"- your sweet name please\n",
"- tell your name\n",
"- hey bot what is your name\n",
"- hey what is you name \n",
"\n",
"## intent:helpuser\n",
"- how did you help me?\n",
"- how can you help me\n",
"- how did you help me\n",
"- you can help me \n",
"- will u help me\n",
"- how could you help me\n",
"- could you please help me\n",
"- pls help me\n",
"\n",
"## intent:botlanguage\n",
"- which language did you speak\n",
"- which language do you speak ?\n",
"- what language you will speak \n",
"- how many language you may know \n",
"- how many language you may know\n",
"- did you know english\n",
"- which language do you speak\n",
"- language you can speak\n",
"- which langauge do you speak\n",
"- do u know english\n",
"\n",
"## intent:botage\n",
"- how old are you ?\n",
"- how old r u \n",
"- how old are you\n",
"- old are you\n",
"- old are u\n",
"- old r u \n",
"- your age \n",
"- what is your age\n",
"- your age please\n",
"- age please\n",
"- age\n",
"- what is your age\n",
"- your age please?\n",
"- how old are you?\n",
"\n",
"## intent:allcourse\n",
"- where do i find all courses?\n",
"- where i find allcourses\n",
"- find all course\n",
"- find courses\n",
"- where i can find all courses\n",
"- how can i find all courses\n",
"\n",
"\n",
"## intent:my_enroll_course\n",
"- where do i find my enrolled courses\n",
"- how can i find my enrolledcourse\n",
"- can i find my enrolled courses\n",
"- take my enrolled courses\n",
"- i want to find my enrolled courses\n",
"\n",
"\n",
"## intent:technical_course\n",
"- where do i find technical courses\n",
"- how can i find technical course\n",
"- find technical courses\n",
"\n",
"\n",
"## intent:comp_training\n",
"- where do i find my completed training \n",
"- find my completed training courses\n",
"- how can i find my completed training courses\n",
"- find my training completed training courses \n",
"\n",
"\n",
"## intent:softskill_course\n",
"- where do i find softskill courses\n",
"- how can i find softskill course\n",
"- how i can find my softskill course\n",
"\n",
"\n",
"## intent:launch_course\n",
"- how do i launch a course\n",
"- how can i launch my course\n",
"- how i can launch a course\n",
"- can i launch a course \n",
"\n",
"\n",
"## intent:enroll_course\n",
"- how do i enroll for a course\n",
"- how can i enroll a course\n",
"- how can course enroll\n",
"- i can enroll course \n",
"- where i can enroll my course\n",
"\n",
"\n",
"\n",
"## intent:start_course\n",
"- how do i start a course\n",
"- how can i start a course\n",
"- how can i start my course\n",
"- i want to start my course\n",
"\n",
"\n",
"## intent:mood_unhappy\n",
"- my day was horrible\n",
"- I am sad\n",
"- i am not fine\n",
"- not fine\n",
"- shamefull\n",
"- not good\n",
"- so bad\n",
"- don't well\n",
"- very very bad\n",
"- I don't feel very well\n",
"- I am disappointed\n",
"- super sad\n",
"- I'm so sad\n",
"- sad\n",
"- very sad\n",
"- feeling bad\n",
"- unhappy\n",
"- bad\n",
"- very bad\n",
"- awful\n",
"- terrible\n",
"- not so good\n",
"- not very good\n",
"- extremly sad\n",
"- so sad\n",
"\n",
"## intent:java_search\n",
"- Looking for course about [java](java_type)\n",
"- Looking for [java](java_type).\n",
"- Any interesting course [java technology](java_type) you can recommend?\n",
"- Would like to read some course about [java](java_type).\n",
"- Any recommendataions for [java](java_type) course to read?\n",
"\n",
"## intent:paper_search\n",
"- Looking for papers about [chatbots](paper_type)\n",
"- Please suggest some interesting papers to read\n",
"- Looking for some new papers about [machine learning](paper_type)\n",
"- Any recommendataions for [statistics](paper_type) papers to read?\n",
"- I have some spare time and would like to read some [mathematics](paper_type) papers.\n",
"- Looking for papers to read\n",
"- Do you have new papers to recommend?\n",
"- Would like to read some papers about [physics](paper_type).\n",
"- Please recommend some interesting papers about [astronomy](paper_type).\n",
"- Looking for [econometrics](paper_type) papers.\n",
"- Looking for papers in [artificial intelligence](paper_type)\n",
"- Please share some papers to read\n",
"- Looking for [Physics](paper_type) to read.\n",
"- Can you recommend me [Mathematics](paper_type)?\n",
"- Any interesting papers [statistics](paper_type) you can recommend?\n",
"\n",
"## intent:inform\n",
"- about [statistics](paper_type)\n",
"- maybe [mathematics](paper_type)\n",
"- about [geography](paper_type)\n",
"- for [machine learning](paper_type)\n",
"- about [differentiation](paper_type)\n",
"- maybe about [calculus](paper_type)\n",
"\n",
"\"\"\"\n",
"\n",
"%store nlu_md > nlu.md"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Defining the NLU Model Configuration"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing 'config' (str) to file 'config.yml'.\n"
]
}
],
"source": [
"config = \"\"\"\n",
"language: \"en\"\n",
"\n",
"pipeline:\n",
"- name: \"WhitespaceTokenizer\"\n",
"- name: \"RegexFeaturizer\"\n",
"- name: \"CRFEntityExtractor\"\n",
"- name: \"EntitySynonymMapper\"\n",
"- name: \"CountVectorsFeaturizer\"\n",
"- name: \"CountVectorsFeaturizer\"\n",
" analyzer: \"char_wb\"\n",
" min_ngram: 1\n",
" max_ngram: 4\n",
"- name: \"EmbeddingIntentClassifier\"\n",
"\"\"\" \n",
"\n",
"%store config > config.yml"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training the NLU Model"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:rasa_nlu.training_data.loading:Training data format of nlu.md is md\n",
"INFO:rasa_nlu.training_data.training_data:Training data stats: \n",
"\t- intent examples: 296 (29 distinct intents)\n",
"\t- Found intents: 'allcourse', 'acces_livewire', 'botname', 'Course_Launch_Issue', 'inform', 'comp_training', 'my_enroll_course', 'app_LiveWire', 'mood_affirm', 'User_Login_Issue', 'softskill_course', 'java_search', 'mood_deny', 'greet', 'LIVEWIRE', 'botlanguage', 'technical_course', 'start_course', 'botage', 'goodbye', 'helpuser', 'Course_Next_Button', 'mood_great', 'enroll_course', 'paper_search', 'Asssessmet_Page_Activity', 'Course_Creation', 'mood_unhappy', 'launch_course'\n",
"\t- entity examples: 22 (2 distinct entities)\n",
"\t- found entities: 'java_type', 'paper_type'\n",
"\n",
"INFO:rasa_nlu.model:Starting to train component WhitespaceTokenizer\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component RegexFeaturizer\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component CRFEntityExtractor\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component EntitySynonymMapper\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component CountVectorsFeaturizer\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component CountVectorsFeaturizer\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Starting to train component EmbeddingIntentClassifier\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
"For more information, please see:\n",
" * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
" * https://github.com/tensorflow/addons\n",
"If you depend on functionality not listed there, please file an issue.\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\rasa_nlu\\classifiers\\embedding_intent_classifier.py:285: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use keras.layers.dense instead.\n",
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Colocations handled automatically by placer.\n",
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\rasa_nlu\\classifiers\\embedding_intent_classifier.py:286: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use keras.layers.dropout instead.\n",
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\tensorflow\\python\\keras\\layers\\core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\tensorflow\\python\\ops\\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.cast instead.\n",
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\tensorflow\\python\\ops\\math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Deprecated in favor of operator or tf.math.divide.\n",
"INFO:rasa_nlu.classifiers.embedding_intent_classifier:Accuracy is updated every 10 epochs\n",
"Epochs: 100%|█████████████████████████████████████████████████| 300/300 [00:09<00:00, 31.21it/s, loss=0.095, acc=0.997]\n",
"INFO:rasa_nlu.classifiers.embedding_intent_classifier:Finished training embedding classifier, loss=0.095, train accuracy=0.997\n",
"INFO:rasa_nlu.model:Finished training component.\n",
"INFO:rasa_nlu.model:Successfully saved model into 'C:\\Users\\015864\\Sify_Chat_bot_livewire1.1 (Custom API action)\\models\\nlu\\default\\current'\n"
]
}
],
"source": [
"from rasa_nlu.training_data import load_data\n",
"from rasa_nlu.config import RasaNLUModelConfig\n",
"from rasa_nlu.model import Trainer\n",
"from rasa_nlu import config\n",
"\n",
"# loading the nlu training samples\n",
"training_data = load_data(\"nlu.md\")\n",
"\n",
"# trainer to educate our pipeline\n",
"trainer = Trainer(config.load(\"config.yml\"))\n",
"\n",
"# train the model!\n",
"interpreter = trainer.train(training_data)\n",
"\n",
"# store it for future use\n",
"model_directory = trainer.persist(\"./models/nlu\", fixed_model_name=\"current\") \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluating the NLU model on a random text"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"intent\": {\n",
" \"name\": \"paper_search\",\n",
" \"confidence\": 0.9407851099967957\n",
" },\n",
" \"entities\": [\n",
" {\n",
" \"start\": 24,\n",
" \"end\": 31,\n",
" \"value\": \"Physics\",\n",
" \"entity\": \"paper_type\",\n",
" \"confidence\": 0.9981841867472812,\n",
" \"extractor\": \"CRFEntityExtractor\"\n",
" }\n",
" ],\n",
" \"intent_ranking\": [\n",
" {\n",
" \"name\": \"paper_search\",\n",
" \"confidence\": 0.9407851099967957\n",
" },\n",
" {\n",
" \"name\": \"java_search\",\n",
" \"confidence\": 0.1371501088142395\n",
" },\n",
" {\n",
" \"name\": \"botlanguage\",\n",
" \"confidence\": 0.12562817335128784\n",
" },\n",
" {\n",
" \"name\": \"LIVEWIRE\",\n",
" \"confidence\": 0.12078073620796204\n",
" },\n",
" {\n",
" \"name\": \"greet\",\n",
" \"confidence\": 0.10813060402870178\n",
" },\n",
" {\n",
" \"name\": \"goodbye\",\n",
" \"confidence\": 0.07771478593349457\n",
" },\n",
" {\n",
" \"name\": \"inform\",\n",
" \"confidence\": 0.07270108163356781\n",
" },\n",
" {\n",
" \"name\": \"mood_unhappy\",\n",
" \"confidence\": 0.0665309801697731\n",
" },\n",
" {\n",
" \"name\": \"softskill_course\",\n",
" \"confidence\": 0.049068138003349304\n",
" },\n",
" {\n",
" \"name\": \"comp_training\",\n",
" \"confidence\": 0.042421095073223114\n",
" }\n",
" ],\n",
" \"text\": \"looking for paper about Physics\"\n",
"}\n"
]
}
],
"source": [
"# A helper function for prettier output\n",
"\n",
"def pprint(o): \n",
" print(json.dumps(o, indent=2))\n",
" \n",
"#pprint(interpreter.parse(\"I am very sad\"))\n",
"#pprint(interpreter.parse(\"what is pal\"))\n",
"pprint(interpreter.parse(\"looking for paper about Physics\"))\n",
"#pprint(interpreter.parse(\"looking for course about java\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Evaluating the NLU model on a test data\n",
"(Here we are using the data at hand i.e nlu.md but it isr recommended to use unseen data)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:tensorflow:From E:\\ANACONDA\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use standard file APIs to check for files with this prefix.\n",
"INFO:tensorflow:Restoring parameters from C:\\Users\\015864\\Sify_Chat_bot_livewire1.1 (Custom API action)\\./models/nlu\\default\\current\\component_6_EmbeddingIntentClassifier.ckpt\n",
"INFO:rasa_nlu.training_data.loading:Training data format of nlu.md is md\n",
"INFO:rasa_nlu.training_data.training_data:Training data stats: \n",
"\t- intent examples: 296 (29 distinct intents)\n",
"\t- Found intents: 'allcourse', 'acces_livewire', 'botname', 'Course_Launch_Issue', 'inform', 'comp_training', 'my_enroll_course', 'app_LiveWire', 'mood_affirm', 'User_Login_Issue', 'softskill_course', 'java_search', 'mood_deny', 'greet', 'LIVEWIRE', 'botlanguage', 'technical_course', 'start_course', 'botage', 'goodbye', 'helpuser', 'Course_Next_Button', 'mood_great', 'enroll_course', 'paper_search', 'Asssessmet_Page_Activity', 'Course_Creation', 'mood_unhappy', 'launch_course'\n",
"\t- entity examples: 22 (2 distinct entities)\n",
"\t- found entities: 'java_type', 'paper_type'\n",
"\n",
"INFO:rasa_nlu.test:Running model for predictions:\n",
"100%|███████████████████████████████████████████████████████████████████████████████| 296/296 [00:00<00:00, 930.38it/s]\n",
"INFO:rasa_nlu.test:Intent evaluation results:\n",
"INFO:rasa_nlu.test:Intent Evaluation: Only considering those 296 examples that have a defined intent out of 296 examples\n",
"INFO:rasa_nlu.test:F1-Score: 0.9966037466037465\n",
"INFO:rasa_nlu.test:Precision: 0.9968629343629343\n",
"INFO:rasa_nlu.test:Accuracy: 0.9966216216216216\n",
"INFO:rasa_nlu.test:Classification report: \n",
" precision recall f1-score support\n",
"\n",
"Asssessmet_Page_Activity 1.00 1.00 1.00 12\n",
" Course_Creation 1.00 1.00 1.00 14\n",
" Course_Launch_Issue 1.00 1.00 1.00 16\n",
" Course_Next_Button 1.00 1.00 1.00 17\n",
" LIVEWIRE 1.00 1.00 1.00 11\n",
" User_Login_Issue 1.00 1.00 1.00 20\n",
" acces_livewire 1.00 1.00 1.00 20\n",
" allcourse 1.00 1.00 1.00 6\n",
" app_LiveWire 1.00 1.00 1.00 4\n",
" botage 1.00 1.00 1.00 14\n",
" botlanguage 1.00 1.00 1.00 10\n",
" botname 1.00 1.00 1.00 8\n",
" comp_training 1.00 1.00 1.00 4\n",
" enroll_course 1.00 1.00 1.00 5\n",
" goodbye 1.00 0.91 0.95 11\n",
" greet 0.93 1.00 0.96 13\n",
" helpuser 1.00 1.00 1.00 8\n",
" inform 1.00 1.00 1.00 6\n",
" java_search 1.00 1.00 1.00 5\n",
" launch_course 1.00 1.00 1.00 4\n",
" mood_affirm 1.00 1.00 1.00 5\n",
" mood_deny 1.00 1.00 1.00 6\n",
" mood_great 1.00 1.00 1.00 22\n",
" mood_unhappy 1.00 1.00 1.00 25\n",
" my_enroll_course 1.00 1.00 1.00 5\n",
" paper_search 1.00 1.00 1.00 15\n",
" softskill_course 1.00 1.00 1.00 3\n",
" start_course 1.00 1.00 1.00 4\n",
" technical_course 1.00 1.00 1.00 3\n",
"\n",
" micro avg 1.00 1.00 1.00 296\n",
" macro avg 1.00 1.00 1.00 296\n",
" weighted avg 1.00 1.00 1.00 296\n",
"\n",
"INFO:rasa_nlu.test:Model prediction errors saved to errors.json.\n",
"INFO:rasa_nlu.test:Entity evaluation results:\n",
"INFO:rasa_nlu.test:Evaluation for entity extractor: CRFEntityExtractor \n",
"INFO:rasa_nlu.test:F1-Score: 1.0\n",
"INFO:rasa_nlu.test:Precision: 1.0\n",
"INFO:rasa_nlu.test:Accuracy: 1.0\n",
"INFO:rasa_nlu.test:Classification report: \n",
" precision recall f1-score support\n",
"\n",
" java_type 1.00 1.00 1.00 6\n",
" no_entity 1.00 1.00 1.00 1195\n",
" paper_type 1.00 1.00 1.00 20\n",
"\n",
" micro avg 1.00 1.00 1.00 1221\n",
" macro avg 1.00 1.00 1.00 1221\n",
"weighted avg 1.00 1.00 1.00 1221\n",
"\n"
]
},
{
"data": {
"text/plain": [
"{'intent_evaluation': {'predictions': [{'text': 'hey',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9254536032676697},\n",
" {'text': 'hello there',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9022641181945801},\n",
" {'text': 'hi',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9370531439781189},\n",
" {'text': 'hello there',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9022641181945801},\n",
" {'text': 'good morning',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.8606653809547424},\n",
" {'text': 'good evening',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.8869237899780273},\n",
" {'text': 'hi chatbot',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9227420687675476},\n",
" {'text': 'hey there',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.8940584659576416},\n",
" {'text': \"let's go\",\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9234314560890198},\n",
" {'text': 'hey dude',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.9325219988822937},\n",
" {'text': 'goodmorning',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.8900388479232788},\n",
" {'text': 'goodevening',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.903065025806427},\n",
" {'text': 'good afternoon',\n",
" 'intent': 'greet',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.7740420699119568},\n",
" {'text': 'livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9595136046409607},\n",
" {'text': 'livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9595136046409607},\n",
" {'text': 'what is livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9456059336662292},\n",
" {'text': 'define livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9535816311836243},\n",
" {'text': 'context of livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9572943449020386},\n",
" {'text': 'could you please give what do you meant livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9330459833145142},\n",
" {'text': 'content of livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9578143954277039},\n",
" {'text': 'what do you meant livewire?',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9549999237060547},\n",
" {'text': 'say something about livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.955808162689209},\n",
" {'text': 'about livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9572390913963318},\n",
" {'text': 'brief about livewire',\n",
" 'intent': 'LIVEWIRE',\n",
" 'predicted': 'LIVEWIRE',\n",
" 'confidence': 0.9486264586448669},\n",
" {'text': 'Error in the assessment launch',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9631736874580383},\n",
" {'text': 'error in the assessment launch',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9631736874580383},\n",
" {'text': 'Issue while taking the assessment',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.7954310178756714},\n",
" {'text': 'Issue with the reference material mapping in Curriculum',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9493960738182068},\n",
" {'text': 'Gentle reminder -Get Enabled, Get Certified || NOC Services',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9016208052635193},\n",
" {'text': 'Gentle reminder',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9258095622062683},\n",
" {'text': 'Issue with the reference material',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9532598853111267},\n",
" {'text': 'Issue with the Question Display',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9698700308799744},\n",
" {'text': 'Unable to access the assessment - TCERT-0016-CS01-V1-0-2018',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9605975151062012},\n",
" {'text': 'Unable to read one of the assessment post editing few questions',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9382939338684082},\n",
" {'text': 'unable to take one of the assessment post',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.946078360080719},\n",
" {'text': 'Issue with the Question Display in one of the assessment',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9725423455238342},\n",
" {'text': 'customer is unable to access enroll courses',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9661334156990051},\n",
" {'text': 'unable to access enroll',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9500719308853149},\n",
" {'text': 'unable to read the course',\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9722700715065002},\n",
" {'text': \"i can't access course\",\n",
" 'intent': 'Course_Launch_Issue',\n",
" 'predicted': 'Course_Launch_Issue',\n",
" 'confidence': 0.9528241157531738},\n",
" {'text': 'User is unable to login into Indostar Site',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9151421189308167},\n",
" {'text': 'User was getting error while logging in Indostar Site',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.8508082628250122},\n",
" {'text': 'getting error while logging',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.7828798890113831},\n",
" {'text': 'User unable to login into Indostar site',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9149685502052307},\n",
" {'text': 'User was getting error while logging',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.7309585213661194},\n",
" {'text': 'Unable to login into the Indostar',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9096689820289612},\n",
" {'text': 'Unable to login',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9415640830993652},\n",
" {'text': 'User unable to access the application',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.901797354221344},\n",
" {'text': 'User unable to access',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9222075939178467},\n",
" {'text': \"i can't login\",\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9123464822769165},\n",
" {'text': 'login issue is there',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9011364579200745},\n",
" {'text': 'login error',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9022604823112488},\n",
" {'text': 'can not login',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9065878987312317},\n",
" {'text': \"can't login into indostar\",\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.8925567865371704},\n",
" {'text': 'while logging i can get error',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.8765984773635864},\n",
" {'text': 'login issue',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9304103851318359},\n",
" {'text': 'login problem',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.929642379283905},\n",
" {'text': 'i may facing login problems',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9229692220687866},\n",
" {'text': 'facing login errors',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.904554009437561},\n",
" {'text': 'facing login issues',\n",
" 'intent': 'User_Login_Issue',\n",
" 'predicted': 'User_Login_Issue',\n",
" 'confidence': 0.9279022216796875},\n",
" {'text': 'User Not able to exit after completing the course',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9195239543914795},\n",
" {'text': 'Not able to exit',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9689516425132751},\n",
" {'text': 'User was not getting next button',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9553978443145752},\n",
" {'text': 'not getting next button',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9604031443595886},\n",
" {'text': 'User unable to click next button after assessment',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.910749614238739},\n",
" {'text': 'unable to click next button',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9360637664794922},\n",
" {'text': 'Next button was not working',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9652144908905029},\n",
" {'text': 'next button not working',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9675159454345703},\n",
" {'text': 'User is not moving to next page',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.944220781326294},\n",
" {'text': 'not moving to next page',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9556053876876831},\n",
" {'text': 'User is unable to move to next slide',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9457254409790039},\n",
" {'text': 'unable to move to next slide',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9156231880187988},\n",
" {'text': 'cant able to exit',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9478616714477539},\n",
" {'text': \"can't able to click next button\",\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9453643560409546},\n",
" {'text': 'cant able to move',\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9174501299858093},\n",
" {'text': \"i can't move next slides\",\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.9525419473648071},\n",
" {'text': \"can't process after assessment\",\n",
" 'intent': 'Course_Next_Button',\n",
" 'predicted': 'Course_Next_Button',\n",
" 'confidence': 0.8787007927894592},\n",
" {'text': 'Unable to edit the course',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.8962083458900452},\n",
" {'text': 'unable to edit',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9550005793571472},\n",
" {'text': 'able to edit the course',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9602668285369873},\n",
" {'text': 'edit course',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9581416249275208},\n",
" {'text': 'cannot able to edit the course',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9393367767333984},\n",
" {'text': \"can't edit the course\",\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9298780560493469},\n",
" {'text': 'Map a course to a curriculum',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9858109354972839},\n",
" {'text': 'course to a curriculum',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9872171878814697},\n",
" {'text': 'Assessment movement from Staging to Production - AC Essentials and Troubleshooting',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9666379690170288},\n",
" {'text': 'Assessment movement',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.8915387988090515},\n",
" {'text': 'Assessment movement from Staging to Production',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.967227041721344},\n",
" {'text': 'AC Essentials and Troubleshooting',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9249686002731323},\n",
" {'text': \"i can't edit the course\",\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9059675931930542},\n",
" {'text': 'cannot edit the course',\n",
" 'intent': 'Course_Creation',\n",
" 'predicted': 'Course_Creation',\n",
" 'confidence': 0.9645700454711914},\n",
" {'text': 'Unable to select the options in the assessment',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9470490217208862},\n",
" {'text': 'Unable to select the options',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9743822813034058},\n",
" {'text': 'User was getting error while taking assessment',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9677754640579224},\n",
" {'text': 'getting error while taking assessment',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9656959176063538},\n",
" {'text': 'User was unable to mark scenerios like yes/no while taking assessment',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9597727656364441},\n",
" {'text': 'unable to mark scenerios like yes/no',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.948271632194519},\n",
" {'text': 'User unable to drag/drop the correct answer',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9619223475456238},\n",
" {'text': 'unable to drag/drop',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.964490532875061},\n",
" {'text': \"i can't select the option in the assessment\",\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9392887949943542},\n",
" {'text': 'i facing trouble in assessment page',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9673318266868591},\n",
" {'text': \"can't able to mark scenerios like yes/no\",\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9475916028022766},\n",
" {'text': 'problem on error in taking assessment',\n",
" 'intent': 'Asssessmet_Page_Activity',\n",
" 'predicted': 'Asssessmet_Page_Activity',\n",
" 'confidence': 0.9690083861351013},\n",
" {'text': 'access livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.8581943511962891},\n",
" {'text': 'how to access livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9097824096679688},\n",
" {'text': 'how to access sify livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.8837205767631531},\n",
" {'text': 'connect livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.8461143970489502},\n",
" {'text': 'How to connect livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9178878664970398},\n",
" {'text': 'Do you want to access livewire?',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.893445611000061},\n",
" {'text': 'how can i do livewire login',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9192942380905151},\n",
" {'text': 'login livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9031237363815308},\n",
" {'text': 'livewire enroll',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.8330518007278442},\n",
" {'text': 'how livewire can connect',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9121878743171692},\n",
" {'text': 'how could i connect livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9008564352989197},\n",
" {'text': 'i meant want to login livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9380010366439819},\n",
" {'text': 'livewire login',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9031237363815308},\n",
" {'text': 'connect access livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9461126923561096},\n",
" {'text': 'lw',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9435375928878784},\n",
" {'text': 'want to login livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.952838122844696},\n",
" {'text': 'live wire login',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9451026916503906},\n",
" {'text': 'login livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9031237363815308},\n",
" {'text': 'hey show livewire login link',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.9289373755455017},\n",
" {'text': 'enroll livewire',\n",
" 'intent': 'acces_livewire',\n",
" 'predicted': 'acces_livewire',\n",
" 'confidence': 0.8330518007278442},\n",
" {'text': 'advantage of livewire',\n",
" 'intent': 'app_LiveWire',\n",
" 'predicted': 'app_LiveWire',\n",
" 'confidence': 0.8991666436195374},\n",
" {'text': 'application of livewire',\n",
" 'intent': 'app_LiveWire',\n",
" 'predicted': 'app_LiveWire',\n",
" 'confidence': 0.9334148168563843},\n",
" {'text': 'which application livewire can be used',\n",
" 'intent': 'app_LiveWire',\n",
" 'predicted': 'app_LiveWire',\n",
" 'confidence': 0.9266462922096252},\n",
" {'text': 'livewire applications',\n",
" 'intent': 'app_LiveWire',\n",
" 'predicted': 'app_LiveWire',\n",
" 'confidence': 0.929597020149231},\n",
" {'text': 'cu',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9161240458488464},\n",
" {'text': 'good by',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9379647970199585},\n",
" {'text': 'cee you later',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.938867449760437},\n",
" {'text': 'good night',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9211503267288208},\n",
" {'text': 'good afternoon',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'greet',\n",
" 'confidence': 0.7740420699119568},\n",
" {'text': 'bye',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9484391808509827},\n",
" {'text': 'goodbye',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9479509592056274},\n",
" {'text': 'have a nice day',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.8917917013168335},\n",
" {'text': 'see you around',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9423503875732422},\n",
" {'text': 'bye bye',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.932583749294281},\n",
" {'text': 'see you later',\n",
" 'intent': 'goodbye',\n",
" 'predicted': 'goodbye',\n",
" 'confidence': 0.9390046000480652},\n",
" {'text': 'yes',\n",
" 'intent': 'mood_affirm',\n",
" 'predicted': 'mood_affirm',\n",
" 'confidence': 0.9129822850227356},\n",
" {'text': 'indeed',\n",
" 'intent': 'mood_affirm',\n",
" 'predicted': 'mood_affirm',\n",
" 'confidence': 0.917166531085968},\n",
" {'text': 'of course',\n",
" 'intent': 'mood_affirm',\n",
" 'predicted': 'mood_affirm',\n",
" 'confidence': 0.9122549891471863},\n",
" {'text': 'that sounds good',\n",
" 'intent': 'mood_affirm',\n",
" 'predicted': 'mood_affirm',\n",
" 'confidence': 0.9279609322547913},\n",
" {'text': 'correct',\n",
" 'intent': 'mood_affirm',\n",
" 'predicted': 'mood_affirm',\n",
" 'confidence': 0.9278704524040222},\n",
" {'text': 'no',\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.9244726896286011},\n",
" {'text': 'never',\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.91646808385849},\n",
" {'text': \"I don't think so\",\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.9176357388496399},\n",
" {'text': \"don't like that\",\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.9162029027938843},\n",
" {'text': 'no way',\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.908715009689331},\n",
" {'text': 'not really',\n",
" 'intent': 'mood_deny',\n",
" 'predicted': 'mood_deny',\n",
" 'confidence': 0.9160557985305786},\n",
" {'text': 'perfect',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9144071340560913},\n",
" {'text': 'not bad',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.8785459399223328},\n",
" {'text': 'very good',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9492309093475342},\n",
" {'text': 'great',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9199885129928589},\n",
" {'text': 'i am fine',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9388322830200195},\n",
" {'text': 'am fine',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9276975989341736},\n",
" {'text': 'yeah i am fine',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9134509563446045},\n",
" {'text': 'amazing',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9424682259559631},\n",
" {'text': 'feeling like a king',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9096189737319946},\n",
" {'text': 'wonderful',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9198298454284668},\n",
" {'text': 'I am feeling very good',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.958373486995697},\n",
" {'text': 'I am great',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9681733846664429},\n",
" {'text': 'I am amazing',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9524843096733093},\n",
" {'text': 'I am going to save the world',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9335938096046448},\n",
" {'text': 'super',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9487914443016052},\n",
" {'text': 'fine',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9296410083770752},\n",
" {'text': 'ok',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9541624188423157},\n",
" {'text': 'good',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9337866902351379},\n",
" {'text': 'extremely good',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9405960440635681},\n",
" {'text': 'so so perfect',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9565496444702148},\n",
" {'text': 'so good',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9585181474685669},\n",
" {'text': 'so perfect',\n",
" 'intent': 'mood_great',\n",
" 'predicted': 'mood_great',\n",
" 'confidence': 0.9607237577438354},\n",
" {'text': 'what is your name ?',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9441007375717163},\n",
" {'text': 'your name pls',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9444134831428528},\n",
" {'text': 'your name',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.944445013999939},\n",
" {'text': 'say your name',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9385982155799866},\n",
" {'text': 'your sweet name please',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9433069825172424},\n",
" {'text': 'tell your name',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9343447089195251},\n",
" {'text': 'hey bot what is your name',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9527814984321594},\n",
" {'text': 'hey what is you name',\n",
" 'intent': 'botname',\n",
" 'predicted': 'botname',\n",
" 'confidence': 0.9397709369659424},\n",
" {'text': 'how did you help me?',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.96607905626297},\n",
" {'text': 'how can you help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9687458276748657},\n",
" {'text': 'how did you help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.96607905626297},\n",
" {'text': 'you can help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9673898220062256},\n",
" {'text': 'will u help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9361573457717896},\n",
" {'text': 'how could you help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9668366312980652},\n",
" {'text': 'could you please help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9652597904205322},\n",
" {'text': 'pls help me',\n",
" 'intent': 'helpuser',\n",
" 'predicted': 'helpuser',\n",
" 'confidence': 0.9674379229545593},\n",
" {'text': 'which language did you speak',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9136074781417847},\n",
" {'text': 'which language do you speak ?',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.909766674041748},\n",
" {'text': 'what language you will speak',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.917372465133667},\n",
" {'text': 'how many language you may know',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9374856948852539},\n",
" {'text': 'how many language you may know',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9374856948852539},\n",
" {'text': 'did you know english',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9501946568489075},\n",
" {'text': 'which language do you speak',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.909766674041748},\n",
" {'text': 'language you can speak',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9222475290298462},\n",
" {'text': 'which langauge do you speak',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9074387550354004},\n",
" {'text': 'do u know english',\n",
" 'intent': 'botlanguage',\n",
" 'predicted': 'botlanguage',\n",
" 'confidence': 0.9328184127807617},\n",
" {'text': 'how old are you ?',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9542023539543152},\n",
" {'text': 'how old r u',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9702245593070984},\n",
" {'text': 'how old are you',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9542023539543152},\n",
" {'text': 'old are you',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.958070695400238},\n",
" {'text': 'old are u',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.962678074836731},\n",
" {'text': 'old r u',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9634266495704651},\n",
" {'text': 'your age',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9664025902748108},\n",
" {'text': 'what is your age',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9430180191993713},\n",
" {'text': 'your age please',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9406349658966064},\n",
" {'text': 'age please',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9393379092216492},\n",
" {'text': 'age',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9508044719696045},\n",
" {'text': 'what is your age',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9430180191993713},\n",
" {'text': 'your age please?',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9406349658966064},\n",
" {'text': 'how old are you?',\n",
" 'intent': 'botage',\n",
" 'predicted': 'botage',\n",
" 'confidence': 0.9542023539543152},\n",
" {'text': 'where do i find all courses?',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.8879339098930359},\n",
" {'text': 'where i find allcourses',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.911815345287323},\n",
" {'text': 'find all course',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.9441965818405151},\n",
" {'text': 'find courses',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.889059841632843},\n",
" {'text': 'where i can find all courses',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.9028177261352539},\n",
" {'text': 'how can i find all courses',\n",
" 'intent': 'allcourse',\n",
" 'predicted': 'allcourse',\n",
" 'confidence': 0.8772210478782654},\n",
" {'text': 'where do i find my enrolled courses',\n",
" 'intent': 'my_enroll_course',\n",
" 'predicted': 'my_enroll_course',\n",
" 'confidence': 0.8568020462989807},\n",
" {'text': 'how can i find my enrolledcourse',\n",
" 'intent': 'my_enroll_course',\n",
" 'predicted': 'my_enroll_course',\n",
" 'confidence': 0.9243870973587036},\n",
" {'text': 'can i find my enrolled courses',\n",
" 'intent': 'my_enroll_course',\n",
" 'predicted': 'my_enroll_course',\n",
" 'confidence': 0.9268881678581238},\n",
" {'text': 'take my enrolled courses',\n",
" 'intent': 'my_enroll_course',\n",
" 'predicted': 'my_enroll_course',\n",
" 'confidence': 0.8711749315261841},\n",
" {'text': 'i want to find my enrolled courses',\n",
" 'intent': 'my_enroll_course',\n",
" 'predicted': 'my_enroll_course',\n",
" 'confidence': 0.9199061989784241},\n",
" {'text': 'where do i find technical courses',\n",
" 'intent': 'technical_course',\n",
" 'predicted': 'technical_course',\n",
" 'confidence': 0.9441145658493042},\n",
" {'text': 'how can i find technical course',\n",
" 'intent': 'technical_course',\n",
" 'predicted': 'technical_course',\n",
" 'confidence': 0.9324712157249451},\n",
" {'text': 'find technical courses',\n",
" 'intent': 'technical_course',\n",
" 'predicted': 'technical_course',\n",
" 'confidence': 0.9494857788085938},\n",
" {'text': 'where do i find my completed training',\n",
" 'intent': 'comp_training',\n",
" 'predicted': 'comp_training',\n",
" 'confidence': 0.915101170539856},\n",
" {'text': 'find my completed training courses',\n",
" 'intent': 'comp_training',\n",
" 'predicted': 'comp_training',\n",
" 'confidence': 0.9075383543968201},\n",
" {'text': 'how can i find my completed training courses',\n",
" 'intent': 'comp_training',\n",
" 'predicted': 'comp_training',\n",
" 'confidence': 0.9117183685302734},\n",
" {'text': 'find my training completed training courses',\n",
" 'intent': 'comp_training',\n",
" 'predicted': 'comp_training',\n",
" 'confidence': 0.9149595499038696},\n",
" {'text': 'where do i find softskill courses',\n",
" 'intent': 'softskill_course',\n",
" 'predicted': 'softskill_course',\n",
" 'confidence': 0.9042088985443115},\n",
" {'text': 'how can i find softskill course',\n",
" 'intent': 'softskill_course',\n",
" 'predicted': 'softskill_course',\n",
" 'confidence': 0.9414992332458496},\n",
" {'text': 'how i can find my softskill course',\n",
" 'intent': 'softskill_course',\n",
" 'predicted': 'softskill_course',\n",
" 'confidence': 0.9454231858253479},\n",
" {'text': 'how do i launch a course',\n",
" 'intent': 'launch_course',\n",
" 'predicted': 'launch_course',\n",
" 'confidence': 0.8783479928970337},\n",
" {'text': 'how can i launch my course',\n",
" 'intent': 'launch_course',\n",
" 'predicted': 'launch_course',\n",
" 'confidence': 0.8909519910812378},\n",
" {'text': 'how i can launch a course',\n",
" 'intent': 'launch_course',\n",
" 'predicted': 'launch_course',\n",
" 'confidence': 0.8938199877738953},\n",
" {'text': 'can i launch a course',\n",
" 'intent': 'launch_course',\n",
" 'predicted': 'launch_course',\n",
" 'confidence': 0.862049400806427},\n",
" {'text': 'how do i enroll for a course',\n",
" 'intent': 'enroll_course',\n",
" 'predicted': 'enroll_course',\n",
" 'confidence': 0.9549375176429749},\n",
" {'text': 'how can i enroll a course',\n",
" 'intent': 'enroll_course',\n",
" 'predicted': 'enroll_course',\n",
" 'confidence': 0.9592815041542053},\n",
" {'text': 'how can course enroll',\n",
" 'intent': 'enroll_course',\n",
" 'predicted': 'enroll_course',\n",
" 'confidence': 0.9517952799797058},\n",
" {'text': 'i can enroll course',\n",
" 'intent': 'enroll_course',\n",
" 'predicted': 'enroll_course',\n",
" 'confidence': 0.9584643244743347},\n",
" {'text': 'where i can enroll my course',\n",
" 'intent': 'enroll_course',\n",
" 'predicted': 'enroll_course',\n",
" 'confidence': 0.955869197845459},\n",
" {'text': 'how do i start a course',\n",
" 'intent': 'start_course',\n",
" 'predicted': 'start_course',\n",
" 'confidence': 0.957624614238739},\n",
" {'text': 'how can i start a course',\n",
" 'intent': 'start_course',\n",
" 'predicted': 'start_course',\n",
" 'confidence': 0.9610071778297424},\n",
" {'text': 'how can i start my course',\n",
" 'intent': 'start_course',\n",
" 'predicted': 'start_course',\n",
" 'confidence': 0.9502424597740173},\n",
" {'text': 'i want to start my course',\n",
" 'intent': 'start_course',\n",
" 'predicted': 'start_course',\n",
" 'confidence': 0.9481368660926819},\n",
" {'text': 'my day was horrible',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9457724094390869},\n",
" {'text': 'I am sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9610917568206787},\n",
" {'text': 'i am not fine',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9449355602264404},\n",
" {'text': 'not fine',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9400454163551331},\n",
" {'text': 'shamefull',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9558016061782837},\n",
" {'text': 'not good',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9387643337249756},\n",
" {'text': 'so bad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9588065147399902},\n",
" {'text': \"don't well\",\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9346068501472473},\n",
" {'text': 'very very bad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9162862300872803},\n",
" {'text': \"I don't feel very well\",\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9307224154472351},\n",
" {'text': 'I am disappointed',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9394508004188538},\n",
" {'text': 'super sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9535143375396729},\n",
" {'text': \"I'm so sad\",\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9580322504043579},\n",
" {'text': 'sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9532513618469238},\n",
" {'text': 'very sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9299768805503845},\n",
" {'text': 'feeling bad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9557260870933533},\n",
" {'text': 'unhappy',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.95769202709198},\n",
" {'text': 'bad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9525614380836487},\n",
" {'text': 'very bad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9276782274246216},\n",
" {'text': 'awful',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9735655188560486},\n",
" {'text': 'terrible',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.8954636454582214},\n",
" {'text': 'not so good',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9499956965446472},\n",
" {'text': 'not very good',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9294439554214478},\n",
" {'text': 'extremly sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9369766712188721},\n",
" {'text': 'so sad',\n",
" 'intent': 'mood_unhappy',\n",
" 'predicted': 'mood_unhappy',\n",
" 'confidence': 0.9600715637207031},\n",
" {'text': 'Looking for course about java',\n",
" 'intent': 'java_search',\n",
" 'predicted': 'java_search',\n",
" 'confidence': 0.9433403015136719},\n",
" {'text': 'Looking for java.',\n",
" 'intent': 'java_search',\n",
" 'predicted': 'java_search',\n",
" 'confidence': 0.9463621377944946},\n",
" {'text': 'Any interesting course java technology you can recommend?',\n",
" 'intent': 'java_search',\n",
" 'predicted': 'java_search',\n",
" 'confidence': 0.9535201787948608},\n",
" {'text': 'Would like to read some course about java.',\n",
" 'intent': 'java_search',\n",
" 'predicted': 'java_search',\n",
" 'confidence': 0.9504712224006653},\n",
" {'text': 'Any recommendataions for java course to read?',\n",
" 'intent': 'java_search',\n",
" 'predicted': 'java_search',\n",
" 'confidence': 0.9425235986709595},\n",
" {'text': 'Looking for papers about chatbots',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9384145736694336},\n",
" {'text': 'Please suggest some interesting papers to read',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9365559220314026},\n",
" {'text': 'Looking for some new papers about machine learning',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.935952365398407},\n",
" {'text': 'Any recommendataions for statistics papers to read?',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.8861579298973083},\n",
" {'text': 'I have some spare time and would like to read some mathematics papers.',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9397373795509338},\n",
" {'text': 'Looking for papers to read',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9453741312026978},\n",
" {'text': 'Do you have new papers to recommend?',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9421024918556213},\n",
" {'text': 'Would like to read some papers about physics.',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9378237128257751},\n",
" {'text': 'Please recommend some interesting papers about astronomy.',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9295266270637512},\n",
" {'text': 'Looking for econometrics papers.',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9464879631996155},\n",
" {'text': 'Looking for papers in artificial intelligence',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9406561851501465},\n",
" {'text': 'Please share some papers to read',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9391187429428101},\n",
" {'text': 'Looking for Physics to read.',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9180237650871277},\n",
" {'text': 'Can you recommend me Mathematics?',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.8822610974311829},\n",
" {'text': 'Any interesting papers statistics you can recommend?',\n",
" 'intent': 'paper_search',\n",
" 'predicted': 'paper_search',\n",
" 'confidence': 0.9366065263748169},\n",
" {'text': 'about statistics',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.9316861629486084},\n",
" {'text': 'maybe mathematics',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.9519217014312744},\n",
" {'text': 'about geography',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.9544830322265625},\n",
" {'text': 'for machine learning',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.9385757446289062},\n",
" {'text': 'about differentiation',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.954918384552002},\n",
" {'text': 'maybe about calculus',\n",
" 'intent': 'inform',\n",
" 'predicted': 'inform',\n",
" 'confidence': 0.9000011086463928}],\n",
" 'report': ' precision recall f1-score support\\n\\nAsssessmet_Page_Activity 1.00 1.00 1.00 12\\n Course_Creation 1.00 1.00 1.00 14\\n Course_Launch_Issue 1.00 1.00 1.00 16\\n Course_Next_Button 1.00 1.00 1.00 17\\n LIVEWIRE 1.00 1.00 1.00 11\\n User_Login_Issue 1.00 1.00 1.00 20\\n acces_livewire 1.00 1.00 1.00 20\\n allcourse 1.00 1.00 1.00 6\\n app_LiveWire 1.00 1.00 1.00 4\\n botage 1.00 1.00 1.00 14\\n botlanguage 1.00 1.00 1.00 10\\n botname 1.00 1.00 1.00 8\\n comp_training 1.00 1.00 1.00 4\\n enroll_course 1.00 1.00 1.00 5\\n goodbye 1.00 0.91 0.95 11\\n greet 0.93 1.00 0.96 13\\n helpuser 1.00 1.00 1.00 8\\n inform 1.00 1.00 1.00 6\\n java_search 1.00 1.00 1.00 5\\n launch_course 1.00 1.00 1.00 4\\n mood_affirm 1.00 1.00 1.00 5\\n mood_deny 1.00 1.00 1.00 6\\n mood_great 1.00 1.00 1.00 22\\n mood_unhappy 1.00 1.00 1.00 25\\n my_enroll_course 1.00 1.00 1.00 5\\n paper_search 1.00 1.00 1.00 15\\n softskill_course 1.00 1.00 1.00 3\\n start_course 1.00 1.00 1.00 4\\n technical_course 1.00 1.00 1.00 3\\n\\n micro avg 1.00 1.00 1.00 296\\n macro avg 1.00 1.00 1.00 296\\n weighted avg 1.00 1.00 1.00 296\\n',\n",
" 'precision': 0.9968629343629343,\n",
" 'f1_score': 0.9966037466037465,\n",
" 'accuracy': 0.9966216216216216},\n",
" 'entity_evaluation': {'CRFEntityExtractor': {'report': ' precision recall f1-score support\\n\\n java_type 1.00 1.00 1.00 6\\n no_entity 1.00 1.00 1.00 1195\\n paper_type 1.00 1.00 1.00 20\\n\\n micro avg 1.00 1.00 1.00 1221\\n macro avg 1.00 1.00 1.00 1221\\nweighted avg 1.00 1.00 1.00 1221\\n',\n",
" 'precision': 1.0,\n",
" 'f1_score': 1.0,\n",
" 'accuracy': 1.0}}}"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from rasa_nlu.test import run_evaluation\n",
"\n",
"run_evaluation(\"nlu.md\", model_directory)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Teaching the bot to respond using Rasa Core\n",
"### 1. Defining a Domain"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing 'domain_yml' (str) to file 'domain.yml'.\n"
]
}
],
"source": [
"domain_yml = \"\"\"\n",
"intents:\n",
"- greet\n",
"- LIVEWIRE\n",
"- Course_Launch_Issue\n",
"- User_Login_Issue\n",
"- Course_Next_Button\n",
"- Course_Creation\n",
"- Asssessmet_Page_Activity\n",
"- acces_livewire\n",
"- app_LiveWire\n",
"- goodbye\n",
"- mood_affirm\n",
"- mood_deny\n",
"- mood_great\n",
"- botname\n",
"- helpuser\n",
"- botlanguage\n",
"- botage\n",
"- allcourse\n",
"- my_enroll_course\n",
"- technical_course\n",
"- comp_training\n",
"- softskill_course\n",
"- launch_course\n",
"- enroll_course\n",
"- start_course\n",
"- mood_unhappy\n",
"- java_search\n",
"- paper_search\n",
"- inform\n",
"\n",
"slots:\n",
" paper_type:\n",
" type: text\n",
" \n",
"entities:\n",
"- paper_type\n",
"\n",
"actions:\n",
"- utter_greet\n",
"- utter_livewire\n",
"- utter_course_launch_issue\n",
"- utter_user_login_issue\n",
"- utter_course_next_button\n",
"- utter_course_creation\n",
"- utter_assesment_page_activity\n",
"- utter_access_livewire\n",
"- utter_app_livewire\n",
"- utter_did_that_help\n",
"- utter_happy\n",
"- utter_goodbye\n",
"- utter_unclear\n",
"- utter_botname\n",
"- utter_helpuser\n",
"- utter_botlang\n",
"- utter_botage\n",
"- utter_all_course\n",
"- utter_my_enroll_Course\n",
"- utter_technical_Course\n",
"- utter_completed_Course\n",
"- utter_softskill_course\n",
"- utter_launch_course\n",
"- utter_enroll_course\n",
"- utter_start_course\n",
"- utter_ask_picture\n",
"- utter_default\n",
"- action_paper_search\n",
"\n",
"templates:\n",
" utter_greet:\n",
" - text: \"Hey! How are you?\"\n",
" buttons:\n",
" - title: \"great\"\n",
" payload: \"great\"\n",
" - title: \"super sad\"\n",
" payload: \"super sad\"\n",
" \n",
" utter_livewire:\n",
" - text: \"Sify livewire is a e-Learning application for managing training and educational records and software for distributing courses over the Internet with features for online collaboration. Do you want to access Livewire?\"\n",
" \n",
" utter_user_login_issue:\n",
" - text: \"Sify Livewire is completely integrated with your (Company) AD system. In order to get logged into livewire portal please use your AD credentials. For most of the cases your AD credentials is nothing but your corporate email id and your system password (System which is provided by company). Incase issue persists please reach out to your (company) IT team.\"\n",
" \n",
" utter_course_next_button:\n",
" - text: \"We strongly recommend you view all the information of current slide/page or else next button may not enabled for you. You should not skip any contents of the slide.\"\n",
" \n",
" utter_course_creation:\n",
" - text: \"Forwarded to project team and resolved\"\n",
" \n",
" \n",
" utter_assesment_page_activity:\n",
" - text: \"Assessments can consists of multiple questions with each having more than one answer options. Options may be single choice demoted by radio buttons or multiple choices denoted by check boxes.You can select only one option if you see raido buttons and multiple options if there are check boxes\"\n",
" \n",
" \n",
" utter_course_launch_issue:\n",
" - text: \"If you are trying the launch a course or an asssessment and see an exception page or a page not found error, please clear your browser cache and retry. Useful link to watch how to clear your browser cache. https://www.youtube.com/results?search_query=how+to+clear+browser+cache \"\n",
" \n",
" utter_access_livewire:\n",
" - text: \"Livewire can be access via http://lwawstest2.sifylivewire.com/ \"\n",
" \n",
" utter_app_livewire:\n",
" - text: \"Application of livewire is ... \"\n",
" \n",
" utter_did_that_help:\n",
" - text: \"Did that help you?\"\n",
"\n",
" utter_unclear:\n",
" - text: \"I am not sure what you are aiming for.\"\n",
" \n",
" utter_happy:\n",
" - text: \"Great carry on!\"\n",
"\n",
" utter_goodbye:\n",
" - text: \"Bye\"\n",
" \n",
" utter_botname:\n",
" - text: \"My name is SIFYBOT , and i am a chatbot \"\n",
" \n",
" utter_helpuser:\n",
" - text: \"sure i makes you feel good \"\n",
" \n",
" utter_botlang:\n",
" - text: \"I can understand Only English \"\n",
" \n",
" utter_botage:\n",
" - text: \"Age has no meaning , because i am Virtual . i was created in september-2019. But i am probabaly already smater than you \"\n",
" \n",
" utter_all_course:\n",
" - text: \"All course...\"\n",
" \n",
" utter_my_enroll_Course:\n",
" - text: \"My enroll course\"\n",
" \n",
" utter_technical_Course:\n",
" - text: \"technical course..\"\n",
" \n",
" utter_completed_Course:\n",
" - text: \"completed course\"\n",
" \n",
" utter_softskill_course:\n",
" - text: \"softskill course\"\n",
" \n",
" utter_launch_course:\n",
" - text: \"launch course\"\n",
" \n",
" utter_enroll_course:\n",
" - text: \"enroll course\"\n",
" \n",
" utter_start_course:\n",
" - text: \"start course\"\n",
" \n",
" utter_ask_picture:\n",
" - text: \"To cheer you up, Don't loose your hope , all is well \"\n",
" \n",
" utter_default:\n",
" - text: \"I am sorry, I am yet to learn about this. you may ask me something about PAL, Livewire,FORUM, instead .\"\n",
"\"\"\"\n",
"\n",
"%store domain_yml > domain.yml"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Writing Stories"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing 'stories_md' (str) to file 'stories.md'.\n"
]
}
],
"source": [
"stories_md = \"\"\"\n",
"## happy path \n",
"* greet \n",
" - utter_greet\n",
"* mood_great \n",
" - utter_happy\n",
"* mood_affirm\n",
" - utter_happy\n",
"* mood_affirm\n",
" - utter_goodbye\n",
" \n",
"## sad path 1 \n",
"* greet\n",
" - utter_greet \n",
"* mood_unhappy\n",
" - utter_ask_picture\n",
"* mood_affirm\n",
" - utter_happy\n",
"\n",
"## sad path 2\n",
"* greet\n",
" - utter_greet\n",
"* mood_unhappy\n",
" - utter_ask_picture\n",
"* mood_deny\n",
" - utter_goodbye\n",
" \n",
"## sad path 3\n",
"* greet\n",
" - utter_greet\n",
"* mood_affirm\n",
" - utter_happy\n",
" \n",
"## strange user\n",
"* mood_affirm\n",
" - utter_happy\n",
"* mood_affirm\n",
" - utter_unclear\n",
"\n",
"## course launch issue\n",
"* Course_Launch_Issue\n",
" - utter_course_launch_issue\n",
"* Course_Creation\n",
" - utter_course_creation\n",
"* User_Login_Issue\n",
" - utter_user_login_issue\n",
"\n",
"## user login issue\n",
"* User_Login_Issue\n",
" - utter_user_login_issue\n",
"\n",
"## course next button\n",
"* Course_Next_Button\n",
" - utter_course_next_button\n",
"* Asssessmet_Page_Activity\n",
" - utter_assesment_page_activity\n",
"\n",
"## course creation\n",
"* Course_Creation\n",
" - utter_course_creation\n",
"\n",
"## assesment page activity \n",
"* Asssessmet_Page_Activity\n",
" - utter_assesment_page_activity\n",
" \n",
"## Live wire\n",
"* LIVEWIRE\n",
" - utter_livewire\n",
" \n",
"## access livewire\n",
"*acces_livewire\n",
" - utter_access_livewire\n",
" \n",
"## application Livewire\n",
"* app_LiveWire\n",
" - utter_app_livewire\n",
"\n",
"## say goodbye\n",
"* goodbye\n",
" - utter_goodbye\n",
" \n",
"##asking bot name\n",
"* botname\n",
" - utter_botname\n",
"\n",
"## user helping\n",
"* helpuser\n",
" - utter_helpuser\n",
"\n",
"## bot language\n",
"* botlanguage\n",
" - utter_botlang\n",
"\n",
"## age of bot\n",
"*botage\n",
" - utter_botage\n",
"\n",
"## all course\n",
"* allcourse\n",
" - utter_all_course\n",
"\n",
"## my enroll course\n",
"* my_enroll_course\n",
" - utter_my_enroll_Course\n",
"\n",
"## technical course\n",
"* technical_course\n",
" - utter_technical_Course\n",
"\n",
"## my completed course\n",
"* comp_training\n",
" - utter_completed_Course\n",
"\n",
"## softskill course\n",
"* softskill_course\n",
" - utter_softskill_course\n",
"\n",
"## launch course\n",
"* launch_course\n",
" - utter_launch_course\n",
"\n",
"## enroll course\n",
"* enroll_course\n",
" - utter_enroll_course\n",
"\n",
"## start course\n",
"* start_course\n",
" - utter_start_course\n",
" \n",
"## Suggestion path 9\n",
"* greet\n",
" - utter_greet\n",
"* paper_search{\"paper_type\":\"physics\"}\n",
" - action_paper_search\n",
" - utter_happy\n",
"* mood_affirm\n",
" - utter_happy\n",
"\n",
"\n",
"## fallback\n",
" - utter_unclear\n",
"\n",
"\"\"\"\n",
"\n",
"%store stories_md > stories.md"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing 'custom_action' (str) to file 'actions.py'.\n"
]
}
],
"source": [
"custom_action = \"\"\"\n",
"\n",
"from rasa_core_sdk import Action\n",
"from rasa_core_sdk.events import SlotSet\n",
"\n",
"import requests\n",
"\n",
"class ApiAction(Action):\n",
" def name(self):\n",
" return \"action_paper_search\"\n",
"\n",
" def run(self, dispatcher, tracker, domain):\n",
"\n",
" paper_type = tracker.get_slot('paper_type')\n",
" \n",
" response = requests.get('http://dblp.org/search/publ/api?q={}&format=json&h=1'.format(paper_type)).json()\n",
" title = response['result']['hits']['hit'][0]['info']['title']\n",
" authors = response['result']['hits']['hit'][0]['info']['authors']['author'][0]\n",
" link = response['result']['hits']['hit'][0]['info']['url']\n",
"\n",
" dispatcher.utter_message(\"I found a paper called {}\".format(title))\n",
" return [SlotSet(\"link\",link), SlotSet(\"authors\",authors)]\n",
" \n",
"\"\"\"\n",
"%store custom_action > actions.py"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:apscheduler.scheduler:Scheduler started\n",
"Processed Story Blocks: 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 1276.18it/s, # trackers=1]\n"
]
},
{
"data": {
"image/png": "PCFET0NUWVBFIGh0bWw+DQo8aHRtbD4NCjxoZWFkPg0KICAgIDxtZXRhIGNoYXJzZXQ9InV0Zi04Ij4NCiAgICA8dGl0bGU+UmFzYSBDb3JlIFZpc3VhbGlzYXRpb248L3RpdGxlPg0KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2RhZ3JlanMuZ2l0aHViLmlvL3Byb2plY3QvZGFncmUtZDMvbGF0ZXN0L2RhZ3JlLWQzLm1pbi5qcyI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCBzcmM9Imh0dHBzOi8vZGFncmVqcy5naXRodWIuaW8vcHJvamVjdC9kYWdyZS9sYXRlc3QvZGFncmUubWluLmpzIj48L3NjcmlwdD4NCiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9kM2pzLm9yZy9kMy52NC5qcyI+PC9zY3JpcHQ+DQogICAgPHNjcmlwdCBzcmM9Imh0dHBzOi8vZGFncmVqcy5naXRodWIuaW8vcHJvamVjdC9ncmFwaGxpYi1kb3QvdjAuNi4zL2dyYXBobGliLWRvdC5qcyI+PC9zY3JpcHQ+DQo8L2hlYWQ+DQo8Ym9keT4NCjxkaXYgaWQ9ImVycm9ybXNnIiBzdHlsZT0iY29sb3I6ICNiMDAiPjwvZGl2Pg0KPHN2Zz4NCiAgICA8c3R5bGU+DQogICAgICAgIC5ub2RlLmludmlzaWJsZSA+IHJlY3Qgew0KICAgICAgICAgICAgZGlzcGxheTogbm9uZTsNCiAgICAgICAgfQ0KDQogICAgICAgIC5ub2RlLnN0YXJ0ID4gcmVjdCB7DQogICAgICAgICAgICBmaWxsOiAjN2Y3Ow0KICAgICAgICAgICAgcng6IDMwOw0KICAgICAgICAgICAgcnk6IDE4Ow0KICAgICAgICB9DQoNCiAgICAgICAgLm5vZGUuZW5kID4gcmVjdCB7DQogICAgICAgICAgICBmaWxsOiAjZjc3Ow0KICAgICAgICAgICAgcng6IDMwOw0KICAgICAgICAgICAgcnk6IDE4Ow0KICAgICAgICB9DQoNCiAgICAgICAgLm5vZGU6bm90KC5hY3RpdmUpID4gcmVjdCwgLm5vZGU6bm90KC5hY3RpdmUpID4gLmxhYmVsIHsNCiAgICAgICAgICAgIG9wYWNpdHk6IDAuNDsNCiAgICAgICAgfQ0KDQogICAgICAgIC5lZGdlUGF0aDpub3QoLmFjdGl2ZSkgcGF0aCB7DQogICAgICAgICAgICBvcGFjaXR5OiAwLjQ7DQogICAgICAgIH0NCg0KICAgICAgICAubm9kZS5lbGxpcHNpcyA+IHJlY3Qgew0KICAgICAgICAgICAgZmlsbDogI0NDQzsNCiAgICAgICAgfQ0KDQogICAgICAgIC5ub2RlLmludGVudCA+IHJlY3Qgew0KICAgICAgICAgICAgZmlsbDogIzdmZjsNCiAgICAgICAgfQ0KDQogICAgICAgIC5ub2RlLmRhc2hlZCA+IHJlY3Qgew0KICAgICAgICAgICAgc3Ryb2tlLWRhc2hhcnJheTogNTsNCiAgICAgICAgfQ0KDQogICAgICAgIHRleHQgew0KICAgICAgICAgICAgZm9udC13ZWlnaHQ6IDMwMDsNCiAgICAgICAgICAgIGZvbnQtZmFtaWx5OiAiSGVsdmV0aWNhIE5ldWUiLCBIZWx2ZXRpY2EsIEFyaWFsLCBzYW5zLXNlcmYsIHNlcmlmOw0KICAgICAgICAgICAgZm9udC1zaXplOiAxNHB4Ow0KICAgICAgICAgICAgY29sb3I6ICMxZjFkMWQ7DQogICAgICAgIH0NCg0KICAgICAgICAubm9kZSByZWN0IHsNCiAgICAgICAgICAgIHN0cm9rZTogIzQ0NDsNCiAgICAgICAgICAgIGZpbGw6ICNmZmY7DQogICAgICAgICAgICBzdHJva2Utd2lkdGg6IDEuNXB4Ow0KICAgICAgICB9DQoNCiAgICAgICAgLmVkZ2VQYXRoIHBhdGggew0KICAgICAgICAgICAgc3Ryb2tlOiAjMzMzOw0KICAgICAgICAgICAgc3Ryb2tlLXdpZHRoOiAxLjVweDsNCiAgICAgICAgfQ0KDQogICAgICAgIHN2ZyB7DQogICAgICAgICAgICBwb3NpdGlvbjogZml4ZWQ7DQogICAgICAgICAgICB0b3A6IDEwcHg7DQogICAgICAgICAgICBsZWZ0OiAwOw0KICAgICAgICAgICAgaGVpZ2h0OiAxMDAlOw0KICAgICAgICAgICAgd2lkdGg6IDEwMCUNCiAgICAgICAgfQ0KICAgIDwvc3R5bGU+DQogICAgPGc+PC9nPg0KPC9zdmc+DQo8c2NyaXB0Pg0KDQogIGZ1bmN0aW9uIHNlcnZlR3JhcGgoKSB7DQogICAgbGV0IG9sZElucHV0R3JhcGhWYWx1ZTsNCg0KICAgIGNvbnN0IHVybCA9ICd2aXN1YWxpemF0aW9uLmRvdCc7DQogICAgY29uc3QgcmVmcmVzaEludGVydmFsID0gNTAwOw0KDQogICAgLy8gdHJpZ2dlciBhIHJlZnJlc2ggYnkgZmV0Y2hpbmcgYW4gdXBkYXRlZCBncmFwaA0KICAgIHNldEludGVydmFsKGZ1bmN0aW9uICgpIHsNCiAgICAgIGZldGNoKHVybCkudGhlbihyID0+IHIudGV4dCgpKS50aGVuKGRvdCA9PiB7DQogICAgICAgIGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKCdlcnJvcm1zZycpLmlubmVySFRNTCA9ICcnOw0KICAgICAgICBpZiAob2xkSW5wdXRHcmFwaFZhbHVlID09PSBkb3QpIHJldHVybjsNCg0KICAgICAgICBvbGRJbnB1dEdyYXBoVmFsdWUgPSBkb3Q7DQogICAgICAgIGRyYXdHcmFwaChkb3QpOw0KICAgICAgfSkuY2F0Y2goZXJyID0+IHsNCiAgICAgICAgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ2Vycm9ybXNnJykuaW5uZXJIVE1MID0NCiAgICAgICAgICAnRmFpbGVkIHRvIHVwZGF0ZSBwbG90LiAoJyArIGVyci5tZXNzYWdlICsgJyknOw0KICAgICAgfSk7DQogICAgfSwgcmVmcmVzaEludGVydmFsKTsNCiAgfQ0KDQogIGZ1bmN0aW9uIGRyYXdHcmFwaChncmFwaCkgew0KICAgIGxldCBnID0gZ3JhcGhsaWJEb3QucmVhZChncmFwaCk7DQogICAgLy8gU2V0IG1hcmdpbnMsIGlmIG5vdCBwcmVzZW50DQogICAgaWYgKCFnLmdyYXBoKCkuaGFzT3duUHJvcGVydHkoIm1hcmdpbngiKSAmJg0KICAgICAgIWcuZ3JhcGgoKS5oYXNPd25Qcm9wZXJ0eSgibWFyZ2lueSIpKSB7DQogICAgICBnLmdyYXBoKCkubWFyZ2lueCA9IDIwOw0KICAgICAgZy5ncmFwaCgpLm1hcmdpbnkgPSAyMDsNCiAgICB9DQogICAgZy5ncmFwaCgpLnRyYW5zaXRpb24gPSBmdW5jdGlvbiAoc2VsZWN0aW9uKSB7DQogICAgICByZXR1cm4gc2VsZWN0aW9uLnRyYW5zaXRpb24oKS5kdXJhdGlvbigzMDApOw0KICAgIH07DQogICAgLy8gUmVuZGVyIHRoZSBncmFwaCBpbnRvIHN2ZyBnDQogICAgZDMuc2VsZWN0KCJzdmcgZyIpLmNhbGwocmVuZGVyLCBnKTsNCiAgfQ0KICAvLyBTZXQgdXAgem9vbSBzdXBwb3J0DQogIGNvbnN0IHN2ZyA9IGQzLnNlbGVjdCgic3ZnIiksDQogICAgaW5uZXIgPSBkMy5zZWxlY3QoInN2ZyBnIiksDQogICAgem9vbSA9IGQzLnpvb20oKS5vbigiem9vbSIsIGZ1bmN0aW9uICgpIHsNCiAgICAgIGlubmVyLmF0dHIoInRyYW5zZm9ybSIsIGQzLmV2ZW50LnRyYW5zZm9ybSk7DQogICAgfSk7DQogIHN2Zy5jYWxsKHpvb20pOw0KDQogIC8vIENyZWF0ZSBhbmQgY29uZmlndXJlIHRoZSByZW5kZXJlcg0KICBjb25zdCByZW5kZXIgPSBkYWdyZUQzLnJlbmRlcigpOw0KDQogIGxldCBpc0NsaWVudCA9IGZhbHNlOw0KICBpc0NsaWVudCA9IHRydWU7DQoNCiAgaWYgKGlzQ2xpZW50KSB7DQogICAgLy8gTWFyayBhbGwgbm9kZXMgYW5kIHRoZWlyIGVkZ2VzIGFzIGFjdGl2ZQ0KICAgIGNzc1J1bGVzID0gZG9jdW1lbnQuc3R5bGVTaGVldHNbMF0uY3NzUnVsZXM7DQogICAgY3NzUnVsZXNbM10uc3R5bGUub3BhY2l0eSA9IDE7DQogICAgY3NzUnVsZXNbNF0uc3R5bGUub3BhY2l0eSA9IDE7DQoNCiAgICBsZXQgZ3JhcGg7DQogICAgZ3JhcGggPSBgZGlncmFwaCAgew0KMCBbY2xhc3M9InN0YXJ0IGFjdGl2ZSIsIGZpbGxjb2xvcj1ncmVlbiwgZm9udHNpemU9MTIsIGxhYmVsPVNUQVJULCBzdHlsZT1maWxsZWRdOw0KIi0xIiBbY2xhc3M9ZW5kLCBmaWxsY29sb3I9cmVkLCBmb250c2l6ZT0xMiwgbGFiZWw9RU5ELCBzdHlsZT1maWxsZWRdOw0KMSBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9hc3Nlc21lbnRfcGFnZV9hY3Rpdml0eV07DQoyIFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2xpdmV3aXJlXTsNCjMgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfYWNjZXNzX2xpdmV3aXJlXTsNCjQgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfYXBwX2xpdmV3aXJlXTsNCjUgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfZ29vZGJ5ZV07DQo2IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2JvdG5hbWVdOw0KNyBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9oZWxwdXNlcl07DQo4IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2JvdGxhbmddOw0KOSBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9ib3RhZ2VdOw0KMTAgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfYWxsX2NvdXJzZV07DQoxMSBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9ncmVldF07DQoxMiBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9oYXBweV07DQoxNSBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9teV9lbnJvbGxfQ291cnNlXTsNCjE2IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX3RlY2huaWNhbF9Db3Vyc2VdOw0KMTcgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfY29tcGxldGVkX0NvdXJzZV07DQoxOCBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9zb2Z0c2tpbGxfY291cnNlXTsNCjE5IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2xhdW5jaF9jb3Vyc2VdOw0KMjAgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfZW5yb2xsX2NvdXJzZV07DQoyMSBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9zdGFydF9jb3Vyc2VdOw0KMjMgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9YWN0aW9uX3BhcGVyX3NlYXJjaF07DQoyNCBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl9oYXBweV07DQoyNiBbY2xhc3M9IiIsIGZvbnRzaXplPTEyLCBsYWJlbD11dHRlcl91bmNsZWFyXTsNCjI4IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2Fza19waWN0dXJlXTsNCjM1IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2hhcHB5XTsNCjM3IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2NvdXJzZV9sYXVuY2hfaXNzdWVdOw0KMzggW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfY291cnNlX2NyZWF0aW9uXTsNCjM5IFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX3VzZXJfbG9naW5faXNzdWVdOw0KNDEgW2NsYXNzPSIiLCBmb250c2l6ZT0xMiwgbGFiZWw9dXR0ZXJfY291cnNlX25leHRfYnV0dG9uXTsNCjQzIFtjbGFzcz0iIiwgZm9udHNpemU9MTIsIGxhYmVsPXV0dGVyX2NvdXJzZV9jcmVhdGlvbl07DQo0NCBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1Bc3NzZXNzbWV0X1BhZ2VfQWN0aXZpdHksIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo0NSBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1MSVZFV0lSRSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjQ2IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPWFjY2VzX2xpdmV3aXJlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNDcgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9YXBwX0xpdmVXaXJlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNDggW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9Z29vZGJ5ZSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjQ5IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPWJvdG5hbWUsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo1MCBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1oZWxwdXNlciwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjUxIFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPWJvdGxhbmd1YWdlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNTIgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9Ym90YWdlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNTMgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9YWxsY291cnNlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNTQgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9Z3JlZXQsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo1NSBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1teV9lbnJvbGxfY291cnNlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNTYgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9dGVjaG5pY2FsX2NvdXJzZSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjU3IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPWNvbXBfdHJhaW5pbmcsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo1OCBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1zb2Z0c2tpbGxfY291cnNlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNTkgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9bGF1bmNoX2NvdXJzZSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjYwIFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPWVucm9sbF9jb3Vyc2UsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo2MSBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1zdGFydF9jb3Vyc2UsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo2MiBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1tb29kX2FmZmlybSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjYzIFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPUNvdXJzZV9MYXVuY2hfSXNzdWUsIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo2NCBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1Db3Vyc2VfTmV4dF9CdXR0b24sIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo2NSBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1Db3Vyc2VfQ3JlYXRpb24sIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo2NiBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1Vc2VyX0xvZ2luX0lzc3VlLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNjcgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9bW9vZF9ncmVhdCwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjY4IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPXBhcGVyX3NlYXJjaHBhcGVyX3R5cGVwaHlzaWNzLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNjkgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9bW9vZF91bmhhcHB5LCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNzAgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9bW9vZF9hZmZpcm0sIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo3MSBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1tb29kX2FmZmlybSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjcyIFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPW1vb2RfYWZmaXJtLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNzMgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9bW9vZF9hZmZpcm0sIHNoYXBlPXJlY3QsIHN0eWxlPWZpbGxlZF07DQo3NCBbY2xhc3M9aW50ZW50LCBmaWxsY29sb3I9bGlnaHRibHVlLCBsYWJlbD1tb29kX2FmZmlybSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjc1IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPW1vb2RfZGVueSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjc2IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPW1vb2RfYWZmaXJtLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNzcgW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9Q291cnNlX0NyZWF0aW9uLCBzaGFwZT1yZWN0LCBzdHlsZT1maWxsZWRdOw0KNzggW2NsYXNzPWludGVudCwgZmlsbGNvbG9yPWxpZ2h0Ymx1ZSwgbGFiZWw9VXNlcl9Mb2dpbl9Jc3N1ZSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjc5IFtjbGFzcz1pbnRlbnQsIGZpbGxjb2xvcj1saWdodGJsdWUsIGxhYmVsPUFzc3Nlc3NtZXRfUGFnZV9BY3Rpdml0eSwgc2hhcGU9cmVjdCwgc3R5bGU9ZmlsbGVkXTsNCjAgLT4gMjYgIFtjbGFzcz0iIiwga2V5PU5PTkUsIGxhYmVsPSIiXTsNCjAgLT4gNDQgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA0NSAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDQ2ICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNDcgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA0OCAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDQ5ICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNTAgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA1MSAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDUyICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNTMgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA1NCAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDU1ICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNTYgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA1NyAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDU4ICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNTkgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA2MCAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDYxICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNjIgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA2MyAgW2NsYXNzPSIiLCBrZXk9MF07DQowIC0+IDY0ICBbY2xhc3M9IiIsIGtleT0wXTsNCjAgLT4gNjUgIFtjbGFzcz0iIiwga2V5PTBdOw0KMCAtPiA2NiAgW2NsYXNzPSIiLCBrZXk9MF07DQoxIC0+ICItMSIgIFtjbGFzcz0iIiwga2V5PU5PTkUsIGxhYmVsPSIiXTsNCjIgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KMyAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQo0IC0+ICItMSIgIFtjbGFzcz0iIiwga2V5PU5PTkUsIGxhYmVsPSIiXTsNCjUgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KNiAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQo3IC0+ICItMSIgIFtjbGFzcz0iIiwga2V5PU5PTkUsIGxhYmVsPSIiXTsNCjggLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KOSAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxMCAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxMSAtPiA2NyAgW2NsYXNzPSIiLCBrZXk9MF07DQoxMSAtPiA2OCAgW2NsYXNzPSIiLCBrZXk9MF07DQoxMSAtPiA2OSAgW2NsYXNzPSIiLCBrZXk9MF07DQoxMSAtPiA3MCAgW2NsYXNzPSIiLCBrZXk9MF07DQoxMiAtPiA3MSAgW2NsYXNzPSIiLCBrZXk9MF07DQoxMiAtPiA3MiAgW2NsYXNzPSIiLCBrZXk9MF07DQoxNSAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxNiAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxNyAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxOCAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoxOSAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoyMCAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoyMSAtPiAiLTEiICBbY2xhc3M9IiIsIGtleT1OT05FLCBsYWJlbD0iIl07DQoyMyAtPiAyNCAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KMjQgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KMjQgLT4gNzMgIFtjbGFzcz0iIiwga2V5PTBdOw0KMjYgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KMjggLT4gNzQgIFtjbGFzcz0iIiwga2V5PTBdOw0KMjggLT4gNzUgIFtjbGFzcz0iIiwga2V5PTBdOw0KMzUgLT4gNzYgIFtjbGFzcz0iIiwga2V5PTBdOw0KMzcgLT4gNzcgIFtjbGFzcz0iIiwga2V5PTBdOw0KMzggLT4gNzggIFtjbGFzcz0iIiwga2V5PTBdOw0KMzkgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KNDEgLT4gNzkgIFtjbGFzcz0iIiwga2V5PTBdOw0KNDMgLT4gIi0xIiAgW2NsYXNzPSIiLCBrZXk9Tk9ORSwgbGFiZWw9IiJdOw0KNDQgLT4gMSAgW2NsYXNzPSIiLCBrZXk9MF07DQo0NSAtPiAyICBbY2xhc3M9IiIsIGtleT0wXTsNCjQ2IC0+IDMgIFtjbGFzcz0iIiwga2V5PTBdOw0KNDcgLT4gNCAgW2NsYXNzPSIiLCBrZXk9MF07DQo0OCAtPiA1ICBbY2xhc3M9IiIsIGtleT0wXTsNCjQ5IC0+IDYgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTAgLT4gNyAgW2NsYXNzPSIiLCBrZXk9MF07DQo1MSAtPiA4ICBbY2xhc3M9IiIsIGtleT0wXTsNCjUyIC0+IDkgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTMgLT4gMTAgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTQgLT4gMTEgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTUgLT4gMTUgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTYgLT4gMTYgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTcgLT4gMTcgIFtjbGFzcz0iIiwga2V5PTBdOw0KNTggLT4gMTggIFtjbGFzcz0iIiwga2V5PTBdOw0KNTkgLT4gMTkgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjAgLT4gMjAgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjEgLT4gMjEgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjIgLT4gMzUgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjMgLT4gMzcgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjQgLT4gNDEgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjUgLT4gNDMgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjYgLT4gMzkgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjcgLT4gMTIgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjggLT4gMjMgIFtjbGFzcz0iIiwga2V5PTBdOw0KNjkgLT4gMjggIFtjbGFzcz0iIiwga2V5PTBdOw0KNzAgLT4gMjQgIFtjbGFzcz0iIiwga2V5PTBdOw0KNzEgLT4gNSAgW2NsYXNzPSIiLCBrZXk9MF07DQo3MiAtPiAxMiAgW2NsYXNzPSIiLCBrZXk9MF07DQo3MyAtPiAyNCAgW2NsYXNzPSIiLCBrZXk9MF07DQo3NCAtPiAyNCAgW2NsYXNzPSIiLCBrZXk9MF07DQo3NSAtPiA1ICBbY2xhc3M9IiIsIGtleT0wXTsNCjc2IC0+IDI2ICBbY2xhc3M9IiIsIGtleT0wXTsNCjc3IC0+IDM4ICBbY2xhc3M9IiIsIGtleT0wXTsNCjc4IC0+IDM5ICBbY2xhc3M9IiIsIGtleT0wXTsNCjc5IC0+IDEgIFtjbGFzcz0iIiwga2V5PTBdOw0KfQ0KYDsNCiAgICBkcmF3R3JhcGgoZ3JhcGgpOw0KICB9IGVsc2Ugew0KICAgIHNlcnZlR3JhcGgoKTsNCiAgfQ0KDQoNCjwvc2NyaXB0Pg0KPC9ib2R5Pg0KPC9odG1sPg0K\n",
"text/plain": [
""
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import Image\n",
"from rasa_core.agent import Agent\n",
"\n",
"agent = Agent('domain.yml')\n",
"agent.visualize(\"stories.md\", \"story_graph.png\", max_history=2)\n",
"Image(filename=\"story_graph.png\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training a Dialogue Model"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Processed Story Blocks: 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 1336.94it/s, # trackers=1]\n",
"Processed Story Blocks: 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 452.82it/s, # trackers=20]\n",
"Processed Story Blocks: 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 413.03it/s, # trackers=20]\n",
"Processed Story Blocks: 100%|██████████████████████████████████████████| 28/28 [00:00<00:00, 379.39it/s, # trackers=20]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"masking (Masking) (None, 3, 67) 0 \n",
"_________________________________________________________________\n",
"lstm (LSTM) (None, 32) 12800 \n",
"_________________________________________________________________\n",
"dense (Dense) (None, 36) 1188 \n",
"_________________________________________________________________\n",
"activation (Activation) (None, 36) 0 \n",
"=================================================================\n",
"Total params: 13,988\n",
"Trainable params: 13,988\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:rasa_core.policies.keras_policy:Fitting model with 444 total samples and a validation split of 0.1\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/200\n",
"444/444 [==============================] - ETA: 7s - loss: 3.5614 - acc: 0.031 - 1s 1ms/sample - loss: 3.5014 - acc: 0.3063\n",
"Epoch 2/200\n",
"444/444 [==============================] - ETA: 0s - loss: 3.4272 - acc: 0.437 - 0s 106us/sample - loss: 3.3802 - acc: 0.5090\n",
"Epoch 3/200\n",
"444/444 [==============================] - ETA: 0s - loss: 3.2994 - acc: 0.437 - 0s 88us/sample - loss: 3.2161 - acc: 0.5090\n",
"Epoch 4/200\n",
"444/444 [==============================] - ETA: 0s - loss: 3.1365 - acc: 0.437 - 0s 85us/sample - loss: 2.9991 - acc: 0.5090\n",
"Epoch 5/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.8813 - acc: 0.437 - 0s 81us/sample - loss: 2.7251 - acc: 0.5090\n",
"Epoch 6/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.6499 - acc: 0.437 - 0s 79us/sample - loss: 2.4867 - acc: 0.5090\n",
"Epoch 7/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.6003 - acc: 0.437 - 0s 81us/sample - loss: 2.3783 - acc: 0.5090\n",
"Epoch 8/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.4942 - acc: 0.437 - 0s 79us/sample - loss: 2.2990 - acc: 0.5090\n",
"Epoch 9/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.4786 - acc: 0.437 - 0s 76us/sample - loss: 2.2641 - acc: 0.5090\n",
"Epoch 10/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.4401 - acc: 0.437 - 0s 76us/sample - loss: 2.2303 - acc: 0.5090\n",
"Epoch 11/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.3958 - acc: 0.437 - 0s 81us/sample - loss: 2.1878 - acc: 0.5090\n",
"Epoch 12/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.3915 - acc: 0.437 - 0s 76us/sample - loss: 2.1612 - acc: 0.5090\n",
"Epoch 13/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.3375 - acc: 0.437 - 0s 76us/sample - loss: 2.1383 - acc: 0.5090\n",
"Epoch 14/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.3217 - acc: 0.437 - 0s 76us/sample - loss: 2.1152 - acc: 0.5090\n",
"Epoch 15/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.3173 - acc: 0.437 - 0s 79us/sample - loss: 2.0744 - acc: 0.5090\n",
"Epoch 16/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.2486 - acc: 0.437 - 0s 79us/sample - loss: 2.0518 - acc: 0.5090\n",
"Epoch 17/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.1986 - acc: 0.437 - 0s 76us/sample - loss: 2.0105 - acc: 0.5090\n",
"Epoch 18/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.2019 - acc: 0.437 - 0s 81us/sample - loss: 1.9966 - acc: 0.5090\n",
"Epoch 19/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.1773 - acc: 0.437 - 0s 76us/sample - loss: 1.9704 - acc: 0.5090\n",
"Epoch 20/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.1777 - acc: 0.437 - 0s 81us/sample - loss: 1.9532 - acc: 0.5090\n",
"Epoch 21/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.0738 - acc: 0.437 - 0s 79us/sample - loss: 1.9218 - acc: 0.5090\n",
"Epoch 22/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.0951 - acc: 0.437 - 0s 76us/sample - loss: 1.9054 - acc: 0.5090\n",
"Epoch 23/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.1515 - acc: 0.437 - 0s 76us/sample - loss: 1.8958 - acc: 0.5090\n",
"Epoch 24/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.0753 - acc: 0.437 - 0s 79us/sample - loss: 1.8573 - acc: 0.5090\n",
"Epoch 25/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.0018 - acc: 0.437 - 0s 76us/sample - loss: 1.8423 - acc: 0.5090\n",
"Epoch 26/200\n",
"444/444 [==============================] - ETA: 0s - loss: 2.0390 - acc: 0.437 - 0s 79us/sample - loss: 1.8188 - acc: 0.5090\n",
"Epoch 27/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.9838 - acc: 0.437 - 0s 79us/sample - loss: 1.7680 - acc: 0.5090\n",
"Epoch 28/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.9496 - acc: 0.437 - 0s 76us/sample - loss: 1.7854 - acc: 0.5090\n",
"Epoch 29/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.9335 - acc: 0.437 - 0s 81us/sample - loss: 1.7341 - acc: 0.5090\n",
"Epoch 30/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.8862 - acc: 0.437 - 0s 76us/sample - loss: 1.6977 - acc: 0.5090\n",
"Epoch 31/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.9507 - acc: 0.437 - 0s 81us/sample - loss: 1.6765 - acc: 0.5135\n",
"Epoch 32/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.8114 - acc: 0.437 - 0s 76us/sample - loss: 1.6500 - acc: 0.5135\n",
"Epoch 33/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.8382 - acc: 0.468 - 0s 88us/sample - loss: 1.6395 - acc: 0.5225\n",
"Epoch 34/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.7839 - acc: 0.468 - 0s 79us/sample - loss: 1.6186 - acc: 0.5360\n",
"Epoch 35/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6892 - acc: 0.468 - 0s 79us/sample - loss: 1.5734 - acc: 0.5450\n",
"Epoch 36/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6834 - acc: 0.468 - 0s 81us/sample - loss: 1.5450 - acc: 0.5473\n",
"Epoch 37/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6412 - acc: 0.562 - 0s 79us/sample - loss: 1.5021 - acc: 0.5788\n",
"Epoch 38/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6648 - acc: 0.500 - 0s 79us/sample - loss: 1.4690 - acc: 0.5698\n",
"Epoch 39/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6249 - acc: 0.593 - 0s 76us/sample - loss: 1.4605 - acc: 0.5991\n",
"Epoch 40/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.6100 - acc: 0.562 - 0s 79us/sample - loss: 1.4304 - acc: 0.5968\n",
"Epoch 41/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.5683 - acc: 0.531 - 0s 76us/sample - loss: 1.4006 - acc: 0.6171\n",
"Epoch 42/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.4388 - acc: 0.656 - 0s 79us/sample - loss: 1.3809 - acc: 0.6419\n",
"Epoch 43/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.4967 - acc: 0.625 - 0s 76us/sample - loss: 1.3605 - acc: 0.6464\n",
"Epoch 44/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.4744 - acc: 0.593 - 0s 81us/sample - loss: 1.3302 - acc: 0.6622\n",
"Epoch 45/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.4548 - acc: 0.656 - 0s 76us/sample - loss: 1.3171 - acc: 0.6757\n",
"Epoch 46/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.4154 - acc: 0.687 - 0s 85us/sample - loss: 1.3022 - acc: 0.7050\n",
"Epoch 47/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.3827 - acc: 0.718 - 0s 76us/sample - loss: 1.2601 - acc: 0.7387\n",
"Epoch 48/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.3996 - acc: 0.718 - 0s 81us/sample - loss: 1.2527 - acc: 0.7410\n",
"Epoch 49/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.3350 - acc: 0.750 - 0s 79us/sample - loss: 1.2388 - acc: 0.7387\n",
"Epoch 50/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.2653 - acc: 0.812 - 0s 81us/sample - loss: 1.2118 - acc: 0.7342\n",
"Epoch 51/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.3258 - acc: 0.750 - 0s 76us/sample - loss: 1.1836 - acc: 0.7545\n",
"Epoch 52/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.2825 - acc: 0.781 - 0s 79us/sample - loss: 1.1626 - acc: 0.7658\n",
"Epoch 53/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.2304 - acc: 0.718 - 0s 79us/sample - loss: 1.1208 - acc: 0.7995\n",
"Epoch 54/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1806 - acc: 0.875 - 0s 74us/sample - loss: 1.0996 - acc: 0.8063\n",
"Epoch 55/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1172 - acc: 0.906 - 0s 74us/sample - loss: 1.1064 - acc: 0.8018\n",
"Epoch 56/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1494 - acc: 0.781 - 0s 81us/sample - loss: 1.0684 - acc: 0.8018\n",
"Epoch 57/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1264 - acc: 0.875 - 0s 76us/sample - loss: 1.0804 - acc: 0.7973\n",
"Epoch 58/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1914 - acc: 0.750 - 0s 79us/sample - loss: 1.0591 - acc: 0.8108\n",
"Epoch 59/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.0482 - acc: 0.812 - 0s 79us/sample - loss: 1.0011 - acc: 0.8356\n",
"Epoch 60/200\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"444/444 [==============================] - ETA: 0s - loss: 1.1079 - acc: 0.843 - 0s 81us/sample - loss: 1.0133 - acc: 0.8153\n",
"Epoch 61/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1453 - acc: 0.875 - 0s 76us/sample - loss: 0.9905 - acc: 0.8311\n",
"Epoch 62/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.9328 - acc: 0.906 - 0s 81us/sample - loss: 0.9639 - acc: 0.8356\n",
"Epoch 63/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.0395 - acc: 0.906 - 0s 76us/sample - loss: 0.9474 - acc: 0.8468\n",
"Epoch 64/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.1234 - acc: 0.812 - 0s 76us/sample - loss: 0.9434 - acc: 0.8401\n",
"Epoch 65/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.9500 - acc: 0.875 - 0s 76us/sample - loss: 0.9164 - acc: 0.8401\n",
"Epoch 66/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.9300 - acc: 0.937 - 0s 76us/sample - loss: 0.8804 - acc: 0.8739\n",
"Epoch 67/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.8917 - acc: 0.906 - 0s 79us/sample - loss: 0.8563 - acc: 0.8964\n",
"Epoch 68/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.8720 - acc: 0.937 - 0s 79us/sample - loss: 0.8701 - acc: 0.8581\n",
"Epoch 69/200\n",
"444/444 [==============================] - ETA: 0s - loss: 1.0012 - acc: 0.812 - 0s 81us/sample - loss: 0.8007 - acc: 0.8784\n",
"Epoch 70/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.8624 - acc: 0.875 - 0s 76us/sample - loss: 0.8133 - acc: 0.8851\n",
"Epoch 71/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7907 - acc: 0.937 - 0s 79us/sample - loss: 0.7967 - acc: 0.8761\n",
"Epoch 72/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7576 - acc: 0.906 - 0s 79us/sample - loss: 0.7886 - acc: 0.8761\n",
"Epoch 73/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6630 - acc: 0.968 - 0s 81us/sample - loss: 0.7350 - acc: 0.8806\n",
"Epoch 74/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7457 - acc: 0.937 - 0s 79us/sample - loss: 0.6743 - acc: 0.9369\n",
"Epoch 75/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.8869 - acc: 0.906 - 0s 81us/sample - loss: 0.7702 - acc: 0.8739\n",
"Epoch 76/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7200 - acc: 0.968 - 0s 79us/sample - loss: 0.7234 - acc: 0.9054\n",
"Epoch 77/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5622 - acc: 0.968 - 0s 79us/sample - loss: 0.6619 - acc: 0.9234\n",
"Epoch 78/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6440 - acc: 0.875 - 0s 76us/sample - loss: 0.6933 - acc: 0.8896\n",
"Epoch 79/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6605 - acc: 0.937 - 0s 79us/sample - loss: 0.6589 - acc: 0.8941\n",
"Epoch 80/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7218 - acc: 0.906 - 0s 79us/sample - loss: 0.6309 - acc: 0.9212\n",
"Epoch 81/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7187 - acc: 0.937 - 0s 76us/sample - loss: 0.6111 - acc: 0.9144\n",
"Epoch 82/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5005 - acc: 0.937 - 0s 76us/sample - loss: 0.6482 - acc: 0.9009\n",
"Epoch 83/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6133 - acc: 0.937 - 0s 76us/sample - loss: 0.5452 - acc: 0.9279\n",
"Epoch 84/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6656 - acc: 0.875 - 0s 79us/sample - loss: 0.6152 - acc: 0.9032\n",
"Epoch 85/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6762 - acc: 0.843 - 0s 79us/sample - loss: 0.5378 - acc: 0.9212\n",
"Epoch 86/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3937 - acc: 1.000 - 0s 76us/sample - loss: 0.5180 - acc: 0.9324\n",
"Epoch 87/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.7331 - acc: 0.812 - 0s 79us/sample - loss: 0.5551 - acc: 0.9032\n",
"Epoch 88/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3796 - acc: 0.968 - 0s 79us/sample - loss: 0.4834 - acc: 0.9347\n",
"Epoch 89/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5116 - acc: 0.937 - 0s 79us/sample - loss: 0.4777 - acc: 0.9347\n",
"Epoch 90/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4464 - acc: 0.968 - 0s 76us/sample - loss: 0.5041 - acc: 0.9257\n",
"Epoch 91/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3637 - acc: 0.968 - 0s 76us/sample - loss: 0.4284 - acc: 0.9482\n",
"Epoch 92/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5013 - acc: 0.906 - 0s 79us/sample - loss: 0.4542 - acc: 0.9369\n",
"Epoch 93/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5709 - acc: 0.937 - 0s 76us/sample - loss: 0.4667 - acc: 0.9189\n",
"Epoch 94/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3848 - acc: 0.968 - 0s 76us/sample - loss: 0.4230 - acc: 0.9414\n",
"Epoch 95/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3156 - acc: 1.000 - 0s 79us/sample - loss: 0.4061 - acc: 0.9572\n",
"Epoch 96/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.6829 - acc: 0.843 - 0s 79us/sample - loss: 0.4050 - acc: 0.9527\n",
"Epoch 97/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3269 - acc: 0.968 - 0s 74us/sample - loss: 0.3930 - acc: 0.9392\n",
"Epoch 98/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4699 - acc: 0.937 - 0s 79us/sample - loss: 0.4015 - acc: 0.9527\n",
"Epoch 99/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.5105 - acc: 0.968 - 0s 79us/sample - loss: 0.3988 - acc: 0.9459\n",
"Epoch 100/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4568 - acc: 0.906 - 0s 76us/sample - loss: 0.3871 - acc: 0.9527\n",
"Epoch 101/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3731 - acc: 0.968 - 0s 81us/sample - loss: 0.3781 - acc: 0.9482\n",
"Epoch 102/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3439 - acc: 0.968 - 0s 88us/sample - loss: 0.3814 - acc: 0.9414\n",
"Epoch 103/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3453 - acc: 0.937 - 0s 83us/sample - loss: 0.3839 - acc: 0.9347\n",
"Epoch 104/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3793 - acc: 0.937 - 0s 81us/sample - loss: 0.3510 - acc: 0.9392\n",
"Epoch 105/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4114 - acc: 0.937 - 0s 83us/sample - loss: 0.3777 - acc: 0.9302\n",
"Epoch 106/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4572 - acc: 0.937 - 0s 83us/sample - loss: 0.3547 - acc: 0.9437\n",
"Epoch 107/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1749 - acc: 0.968 - 0s 79us/sample - loss: 0.3279 - acc: 0.9505\n",
"Epoch 108/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1594 - acc: 1.000 - 0s 81us/sample - loss: 0.2738 - acc: 0.9550\n",
"Epoch 109/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3544 - acc: 0.937 - 0s 85us/sample - loss: 0.2841 - acc: 0.9595\n",
"Epoch 110/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1207 - acc: 1.000 - 0s 85us/sample - loss: 0.3253 - acc: 0.9459\n",
"Epoch 111/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2607 - acc: 0.968 - 0s 81us/sample - loss: 0.3062 - acc: 0.9505\n",
"Epoch 112/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3105 - acc: 0.968 - 0s 81us/sample - loss: 0.2904 - acc: 0.9505\n",
"Epoch 113/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2311 - acc: 1.000 - 0s 79us/sample - loss: 0.2824 - acc: 0.9572\n",
"Epoch 114/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2268 - acc: 0.968 - 0s 85us/sample - loss: 0.2566 - acc: 0.9459\n",
"Epoch 115/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3448 - acc: 0.906 - 0s 83us/sample - loss: 0.2421 - acc: 0.9640\n",
"Epoch 116/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3524 - acc: 0.906 - 0s 81us/sample - loss: 0.2851 - acc: 0.9459\n",
"Epoch 117/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3350 - acc: 0.937 - 0s 81us/sample - loss: 0.2847 - acc: 0.9595\n",
"Epoch 118/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.4243 - acc: 0.937 - 0s 79us/sample - loss: 0.2215 - acc: 0.9685\n",
"Epoch 119/200\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"444/444 [==============================] - ETA: 0s - loss: 0.2180 - acc: 0.968 - 0s 81us/sample - loss: 0.2388 - acc: 0.9595\n",
"Epoch 120/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2903 - acc: 0.968 - 0s 81us/sample - loss: 0.2713 - acc: 0.9459\n",
"Epoch 121/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3198 - acc: 1.000 - 0s 79us/sample - loss: 0.2774 - acc: 0.9572\n",
"Epoch 122/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1982 - acc: 0.968 - 0s 81us/sample - loss: 0.2475 - acc: 0.9527\n",
"Epoch 123/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3999 - acc: 0.937 - 0s 76us/sample - loss: 0.3069 - acc: 0.9437\n",
"Epoch 124/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1424 - acc: 0.968 - 0s 81us/sample - loss: 0.2559 - acc: 0.9550\n",
"Epoch 125/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3245 - acc: 0.937 - 0s 85us/sample - loss: 0.2544 - acc: 0.9550\n",
"Epoch 126/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2788 - acc: 1.000 - 0s 79us/sample - loss: 0.2301 - acc: 0.9617\n",
"Epoch 127/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2039 - acc: 0.968 - 0s 76us/sample - loss: 0.2843 - acc: 0.9482\n",
"Epoch 128/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1822 - acc: 0.968 - 0s 83us/sample - loss: 0.2752 - acc: 0.9617\n",
"Epoch 129/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2954 - acc: 0.937 - 0s 76us/sample - loss: 0.2534 - acc: 0.9505\n",
"Epoch 130/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2043 - acc: 1.000 - 0s 81us/sample - loss: 0.2056 - acc: 0.9752\n",
"Epoch 131/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3666 - acc: 0.937 - 0s 79us/sample - loss: 0.2729 - acc: 0.9550\n",
"Epoch 132/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3446 - acc: 0.968 - 0s 79us/sample - loss: 0.2423 - acc: 0.9572\n",
"Epoch 133/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2101 - acc: 1.000 - 0s 79us/sample - loss: 0.2302 - acc: 0.9617\n",
"Epoch 134/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1385 - acc: 1.000 - 0s 79us/sample - loss: 0.1908 - acc: 0.9640\n",
"Epoch 135/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1345 - acc: 1.000 - 0s 81us/sample - loss: 0.2466 - acc: 0.9527\n",
"Epoch 136/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3275 - acc: 0.937 - 0s 76us/sample - loss: 0.2222 - acc: 0.9572\n",
"Epoch 137/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0655 - acc: 1.000 - 0s 81us/sample - loss: 0.1809 - acc: 0.9820\n",
"Epoch 138/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2505 - acc: 1.000 - 0s 81us/sample - loss: 0.2053 - acc: 0.9640\n",
"Epoch 139/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2836 - acc: 0.968 - 0s 79us/sample - loss: 0.2165 - acc: 0.9685\n",
"Epoch 140/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1177 - acc: 1.000 - 0s 79us/sample - loss: 0.2269 - acc: 0.9685\n",
"Epoch 141/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1946 - acc: 1.000 - 0s 76us/sample - loss: 0.2180 - acc: 0.9662\n",
"Epoch 142/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2108 - acc: 1.000 - 0s 79us/sample - loss: 0.2072 - acc: 0.9685\n",
"Epoch 143/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1689 - acc: 0.968 - 0s 79us/sample - loss: 0.2119 - acc: 0.9572\n",
"Epoch 144/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1182 - acc: 1.000 - 0s 76us/sample - loss: 0.2011 - acc: 0.9730\n",
"Epoch 145/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0322 - acc: 1.000 - 0s 76us/sample - loss: 0.1830 - acc: 0.9662\n",
"Epoch 146/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2965 - acc: 0.937 - 0s 79us/sample - loss: 0.1830 - acc: 0.9640\n",
"Epoch 147/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1002 - acc: 1.000 - 0s 76us/sample - loss: 0.1659 - acc: 0.9752\n",
"Epoch 148/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2453 - acc: 0.968 - 0s 79us/sample - loss: 0.1911 - acc: 0.9662\n",
"Epoch 149/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1711 - acc: 1.000 - 0s 81us/sample - loss: 0.1773 - acc: 0.9730\n",
"Epoch 150/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2258 - acc: 0.968 - 0s 81us/sample - loss: 0.1723 - acc: 0.9685\n",
"Epoch 151/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2825 - acc: 1.000 - 0s 81us/sample - loss: 0.1732 - acc: 0.9685\n",
"Epoch 152/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1491 - acc: 0.968 - 0s 81us/sample - loss: 0.1682 - acc: 0.9685\n",
"Epoch 153/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1751 - acc: 0.968 - 0s 79us/sample - loss: 0.1662 - acc: 0.9797\n",
"Epoch 154/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0953 - acc: 1.000 - 0s 83us/sample - loss: 0.1636 - acc: 0.9775\n",
"Epoch 155/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0447 - acc: 1.000 - 0s 83us/sample - loss: 0.1550 - acc: 0.9707\n",
"Epoch 156/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2481 - acc: 1.000 - 0s 85us/sample - loss: 0.2014 - acc: 0.9640\n",
"Epoch 157/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0885 - acc: 1.000 - 0s 83us/sample - loss: 0.1780 - acc: 0.9707\n",
"Epoch 158/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1147 - acc: 1.000 - 0s 92us/sample - loss: 0.1820 - acc: 0.9707\n",
"Epoch 159/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1397 - acc: 0.968 - 0s 81us/sample - loss: 0.1722 - acc: 0.9662\n",
"Epoch 160/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1851 - acc: 1.000 - 0s 88us/sample - loss: 0.1717 - acc: 0.9662\n",
"Epoch 161/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2280 - acc: 0.968 - 0s 81us/sample - loss: 0.1589 - acc: 0.9775\n",
"Epoch 162/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2086 - acc: 0.968 - 0s 85us/sample - loss: 0.1841 - acc: 0.9662\n",
"Epoch 163/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1800 - acc: 0.968 - 0s 76us/sample - loss: 0.1652 - acc: 0.9865\n",
"Epoch 164/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1897 - acc: 1.000 - 0s 83us/sample - loss: 0.1643 - acc: 0.9797\n",
"Epoch 165/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1220 - acc: 1.000 - 0s 76us/sample - loss: 0.1607 - acc: 0.9707\n",
"Epoch 166/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2358 - acc: 0.968 - 0s 81us/sample - loss: 0.1366 - acc: 0.9865\n",
"Epoch 167/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1710 - acc: 1.000 - 0s 79us/sample - loss: 0.1551 - acc: 0.9730\n",
"Epoch 168/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1152 - acc: 1.000 - 0s 83us/sample - loss: 0.1451 - acc: 0.9797\n",
"Epoch 169/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0885 - acc: 1.000 - 0s 76us/sample - loss: 0.1358 - acc: 0.9932\n",
"Epoch 170/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2732 - acc: 0.968 - 0s 79us/sample - loss: 0.1398 - acc: 0.9820\n",
"Epoch 171/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1497 - acc: 1.000 - 0s 79us/sample - loss: 0.1552 - acc: 0.9797\n",
"Epoch 172/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0371 - acc: 1.000 - 0s 81us/sample - loss: 0.1048 - acc: 0.9887\n",
"Epoch 173/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1070 - acc: 0.968 - 0s 81us/sample - loss: 0.1395 - acc: 0.9820\n",
"Epoch 174/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0925 - acc: 1.000 - 0s 81us/sample - loss: 0.1289 - acc: 0.9887\n",
"Epoch 175/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1037 - acc: 1.000 - 0s 79us/sample - loss: 0.1650 - acc: 0.9775\n",
"Epoch 176/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1375 - acc: 0.968 - 0s 76us/sample - loss: 0.1338 - acc: 0.9775\n",
"Epoch 177/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2436 - acc: 1.000 - 0s 81us/sample - loss: 0.1503 - acc: 0.9775\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 178/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1701 - acc: 1.000 - 0s 79us/sample - loss: 0.1740 - acc: 0.9707\n",
"Epoch 179/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1136 - acc: 1.000 - 0s 79us/sample - loss: 0.1010 - acc: 0.9955\n",
"Epoch 180/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.3732 - acc: 0.906 - 0s 76us/sample - loss: 0.1343 - acc: 0.9842\n",
"Epoch 181/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0999 - acc: 1.000 - 0s 81us/sample - loss: 0.1817 - acc: 0.9662\n",
"Epoch 182/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1845 - acc: 0.937 - 0s 76us/sample - loss: 0.1480 - acc: 0.9730\n",
"Epoch 183/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0500 - acc: 1.000 - 0s 81us/sample - loss: 0.1236 - acc: 0.9797\n",
"Epoch 184/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2499 - acc: 0.968 - 0s 81us/sample - loss: 0.1411 - acc: 0.9797\n",
"Epoch 185/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0535 - acc: 1.000 - 0s 79us/sample - loss: 0.0944 - acc: 0.9887\n",
"Epoch 186/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1045 - acc: 0.968 - 0s 76us/sample - loss: 0.1426 - acc: 0.9775\n",
"Epoch 187/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0569 - acc: 1.000 - 0s 81us/sample - loss: 0.1349 - acc: 0.9842\n",
"Epoch 188/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1661 - acc: 0.968 - 0s 81us/sample - loss: 0.1596 - acc: 0.9752\n",
"Epoch 189/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1731 - acc: 0.968 - 0s 76us/sample - loss: 0.1303 - acc: 0.9752\n",
"Epoch 190/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0658 - acc: 1.000 - 0s 81us/sample - loss: 0.1270 - acc: 0.9865\n",
"Epoch 191/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0921 - acc: 1.000 - 0s 79us/sample - loss: 0.1309 - acc: 0.9797\n",
"Epoch 192/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.2334 - acc: 0.968 - 0s 76us/sample - loss: 0.1348 - acc: 0.9842\n",
"Epoch 193/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1349 - acc: 1.000 - 0s 76us/sample - loss: 0.1168 - acc: 0.9820\n",
"Epoch 194/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1278 - acc: 1.000 - 0s 79us/sample - loss: 0.1189 - acc: 0.9775\n",
"Epoch 195/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0570 - acc: 1.000 - 0s 79us/sample - loss: 0.1122 - acc: 0.9887\n",
"Epoch 196/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1045 - acc: 1.000 - 0s 79us/sample - loss: 0.1508 - acc: 0.9797\n",
"Epoch 197/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.1559 - acc: 0.968 - 0s 76us/sample - loss: 0.1327 - acc: 0.9797\n",
"Epoch 198/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0624 - acc: 1.000 - 0s 79us/sample - loss: 0.1302 - acc: 0.9775\n",
"Epoch 199/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0996 - acc: 0.968 - 0s 79us/sample - loss: 0.1455 - acc: 0.9752\n",
"Epoch 200/200\n",
"444/444 [==============================] - ETA: 0s - loss: 0.0503 - acc: 1.000 - 0s 79us/sample - loss: 0.1247 - acc: 0.9865\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:rasa_core.policies.keras_policy:Done fitting keras policy model\n",
"Processed actions: 75it [00:00, 1566.68it/s, # examples=73]\n",
"INFO:rasa_core.agent:Model directory models/dialogue exists and contains old model files. All files will be overwritten.\n",
"INFO:rasa_core.agent:Persisted model to 'C:\\Users\\015864\\Sify_Chat_bot_livewire1.1 (Custom API action)\\models\\dialogue'\n"
]
}
],
"source": [
"from rasa_core.policies import FallbackPolicy, KerasPolicy, MemoizationPolicy \n",
"from rasa_core.agent import Agent\n",
"\n",
"from rasa_core import config as policy_config\n",
"\n",
"\n",
"# this will catch predictions the model isn't very certain about\n",
"# there is a threshold for the NLU predictions as well as the action predictions\n",
"fallback = FallbackPolicy(fallback_action_name=\"utter_unclear\",\n",
" core_threshold=0.2,\n",
" nlu_threshold=0.26)\n",
"\n",
"#agent = Agent('domain.yml', policies=[MemoizationPolicy(), KerasPolicy(), fallback])\n",
"policiess = policy_config.load(\"policy.yml\")\n",
"agent = Agent(\"domain.yml\", policies=policiess)\n",
"\n",
"# loading our neatly defined training dialogues\n",
"training_data = agent.load_data('stories.md') \n",
"\n",
"agent.train(\n",
" training_data\n",
")\n",
"''' agent.train(\n",
" training_data,\n",
" validation_split=0.2,\n",
" epochs=200\n",
")'''\n",
"\n",
"agent.persist('models/dialogue')"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2019-09-26 10:18:26 INFO rasa_sdk.endpoint - Starting action endpoint server...\n",
"E:\\ANACONDA\\lib\\site-packages\\rasa_core_sdk\\__init__.py:12: UserWarning: The 'rasa_core_sdk' package has been renamed. You should change your imports to use 'rasa_sdk' instead.\n",
" UserWarning,\n",
"2019-09-26 10:18:26 INFO rasa_sdk.executor - Registered function for 'action_paper_search'.\n",
"Traceback (most recent call last):\n",
" File \"E:\\ANACONDA\\lib\\runpy.py\", line 193, in _run_module_as_main\n",
" \"__main__\", mod_spec)\n",
" File \"E:\\ANACONDA\\lib\\runpy.py\", line 85, in _run_code\n",
" exec(code, run_globals)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\rasa_sdk\\endpoint.py\", line 140, in \n",
" rasa_sdk.__main__.main()\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\rasa_sdk\\__main__.py\", line 26, in main\n",
" main_from_args(cmdline_args)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\rasa_sdk\\__main__.py\", line 18, in main_from_args\n",
" run(args.actions, args.port, args.cors)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\rasa_sdk\\endpoint.py\", line 131, in run\n",
" http_server.start()\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\gevent\\baseserver.py\", line 305, in start\n",
" self.init_socket()\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\gevent\\pywsgi.py\", line 1490, in init_socket\n",
" StreamServer.init_socket(self)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\gevent\\server.py\", line 146, in init_socket\n",
" self.socket = self.get_listener(self.address, self.backlog, self.family)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\gevent\\server.py\", line 157, in get_listener\n",
" return _tcp_listener(address, backlog=backlog, reuse_addr=cls.reuse_addr, family=family)\n",
" File \"E:\\ANACONDA\\lib\\site-packages\\gevent\\server.py\", line 256, in _tcp_listener\n",
" sock.bind(address)\n",
"OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted: ('0.0.0.0', 5055)\n"
]
}
],
"source": [
"#!python -m rasa_sdk.endpoint --actions actions\n",
"#!python -m rasa_sdk --actions actions"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Restoring parameters from models/nlu/default/current\\component_6_EmbeddingIntentClassifier.ckpt\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"********** Bot is ready to talk! Type your messages here or send 'stop' ******* \n",
"user : hi\n",
"Bot : Hey! How are you?\n",
"\n",
"user : fine\n",
"Bot : Great carry on!\n",
"\n",
"user : bye\n",
"Bot : Bye\n",
"\n",
"user : hello\n",
"Bot : Hey! How are you?\n",
"\n",
"user : looking for paper about Physics\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"ERROR:rasa_core.actions.action:Failed to run custom action 'action_paper_search'. Action server responded with a non 200 status code of 404. Make sure your action server properly runs actions and returns a 200 once the action is executed. Error: 404 Client Error: NOT FOUND for url: http://127.0.0.1:5055/home\n",
"ERROR:rasa_core.processor:Encountered an exception while running action 'action_paper_search'. Bot will continue, but the actions events are lost. Make sure to fix the exception in your custom code.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Bot : Great carry on!\n",
"\n",
"user : stop\n"
]
}
],
"source": [
"import IPython\n",
"from IPython.display import clear_output\n",
"from rasa_core.agent import Agent\n",
"from rasa_core.interpreter import NaturalLanguageInterpreter\n",
"from rasa_core.utils import EndpointConfig\n",
"import time\n",
"\n",
"messages = [\"Hi! you can chat in this window. Type 'stop' to end the conversation.\"]\n",
"interpreter = NaturalLanguageInterpreter.create('models/nlu/default/current')\n",
"endpoint = EndpointConfig('http://127.0.0.1:5055/home')\n",
"agent = Agent.load('models/dialogue', interpreter=interpreter, action_endpoint = endpoint)\n",
"\n",
"print(\"********** Bot is ready to talk! Type your messages here or send 'stop' ******* \")\n",
"while True:\n",
" a = input(\"user : \")\n",
" if a == 'stop':\n",
" break\n",
" responses = agent.handle_text(a)\n",
" for response in responses:\n",
" print(\"Bot :\",response[\"text\"])\n",
" print()\n",
"\n",
"# looking for paper about Physics"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}