Error while trying to create a look up table

the config i used:

language: en pipeline:

  • name: “WhitespaceTokenizer”
  • name: “EntitySynonymMapper”
  • name: “intent_entity_featurizer_regex”
  • name: “EmbeddingIntentClassifier”
  • name: “ner_crf” features: [ [“low”, “title”, “upper”], [“bias”, “low”, “prefix5”, “prefix2”, “suffix5”, “suffix3”,“suffix2”, “upper”, “title”, “digit”, “pattern”], [“low”, “title”, “upper”] ]

policies:

  • name: MemoizationPolicy
  • name: KerasPolicy
  • name: MappingPolicy

Training Data i used:

intent:searchBookIntent

lookup:sBook

  • harry potter and the chamber of secrets
  • to kill a mockingbird
  • the great gatsby
  • the adventure of huckleberry finn
  • the catcher in the rye
  • the grapes of wrath
  • of mice and men
  • fahrenheit
  • the scarlet letter
  • in cold blood
  • east
  • harry porter
  • the hobbit
  • walk to freedom
  • wings of fire
  • the walking dead

Error i Got:

<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="shortcut icon" href="/icons/favicon.ico"><link rel="apple-touch-icon" href="/icons/apple-touch-icon-57x57.png"/><link rel="apple-touch-icon" sizes="72x72" href="/icons/apple-touch-icon-72x72.png"/><link rel="apple-touch-icon" sizes="114x114" href="/icons/apple-touch-icon-114x114.png"/><link rel="apple-touch-icon" sizes="144x144" href="/icons/apple-touch-icon-144x144.png"/><script>try {
    window.SERVER_DATA = __SERVER_DATA__;
  } catch {}</script><title>Rasa X</title><link href="/static/css/2.465d5a02.chunk.css" rel="stylesheet"></head><body><div id="root"></div><script>!function(c){function e(e){for(var r,t,n=e[0],o=e[1],u=e[2],i=0,a=[];i<n.length;i++)t=n[i],l[t]&&a.push(l[t][0]),l[t]=0;for(r in o)Object.prototype.hasOwnProperty.call(o,r)&&(c[r]=o[r]);for(d&&d(e);a.length;)a.shift()();return p.push.apply(p,u||[]),f()}function f(){for(var e,r=0;r<p.length;r++){for(var t=p[r],n=!0,o=1;o<t.length;o++){var u=t[o];0!==l[u]&&(n=!1)}n&&(p.splice(r--,1),e=s(s.s=t[0]))}return e}var t={},l={1:0},p=[];function s(e){if(t[e])return t[e].exports;var r=t[e]={i:e,l:!1,exports:{}};return c[e].call(r.exports,r,r.exports,s),r.l=!0,r.exports}s.e=function(u){var e=[],t=l[u];if(0!==t)if(t)e.push(t[2]);else{var r=new Promise(function(e,r){t=l[u]=[e,r]});e.push(t[2]=r);var n,i=document.createElement("script");i.charset="utf-8",i.timeout=120,s.nc&&i.setAttribute("nonce",s.nc),i.src=s.p+"static/js/"+({}[u]||u)+"."+{3:"634070b1",4:"fd0c0280",5:"55d21e5f",6:"9c26f39b"}[u]+".chunk.js",n=function(e){i.onerror=i.onload=null,clearTimeout(a);var r=l[u];if(0!==r){if(r){var t=e&&("load"===e.type?"missing":e.type),n=e&&e.target&&e.target.src,o=new Error("Loading chunk "+u+" failed.\n("+t+": "+n+")");o.type=t,o.request=n,r[1](o)}l[u]=void 0}};var a=setTimeout(function(){n({type:"timeout",target:i})},12e4);i.onerror=i.onload=n,document.head.appendChild(i)}return Promise.all(e)},s.m=c,s.c=t,s.d=function(e,r,t){s.o(e,r)||Object.defineProperty(e,r,{enumerable:!0,get:t})},s.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},s.t=function(r,e){if(1&e&&(r=s(r)),8&e)return r;if(4&e&&"object"==typeof r&&r&&r.__esModule)return r;var t=Object.create(null);if(s.r(t),Object.defineProperty(t,"default",{enumerable:!0,value:r}),2&e&&"string"!=typeof r)for(var n in r)s.d(t,n,function(e){return r[e]}.bind(null,n));return t},s.n=function(e){var r=e&&e.__esModule?function(){return e.default}:function(){return e};return s.d(r,"a",r),r},s.o=function(e,r){return Object.prototype.hasOwnProperty.call(e,r)},s.p="/",s.oe=function(e){throw console.error(e),e};var r=window.webpackJsonp=window.webpackJsonp||[],n=r.push.bind(r);r.push=e,r=r.slice();for(var o=0;o<r.length;o++)e(r[o]);var d=n;f()}([])</script><script src="/static/js/2.bf1eb599.chunk.js"></script><script src="/static/js/main.a02e112f.chunk.js"></script></body></html>

i got this when i tried to parse a text from postman in localhost:5002/model/parse. i have also attached a screenshot below please look into it

can anyone explain whats the problem here ,i am new to lookup table .can anyone share a solution for it ??

make sure your some of examples of training data matches to your lookup examples

as mentioned on the official website " Note For lookup tables to be effective, there must be a few examples of matches in your training data. Otherwise the model will not learn to use the lookup table match features. "

revert if this does not work.

@shubham, i have chaged the missing training data but still when i tried to hit through postman i got the response as

<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="shortcut icon" href="/icons/favicon.ico"><link rel="apple-touch-icon" href="/icons/apple-touch-icon-57x57.png"/><link rel="apple-touch-icon" sizes="72x72" href="/icons/apple-touch-icon-72x72.png"/><link rel="apple-touch-icon" sizes="114x114" href="/icons/apple-touch-icon-114x114.png"/><link rel="apple-touch-icon" sizes="144x144" href="/icons/apple-touch-icon-144x144.png"/><script>try {
    window.SERVER_DATA = __SERVER_DATA__;
  } catch {}</script><title>Rasa X</title><link href="/static/css/2.465d5a02.chunk.css" rel="stylesheet"></head><body><div id="root"></div><script>!function(c){function e(e){for(var r,t,n=e[0],o=e[1],u=e[2],i=0,a=[];i<n.length;i++)t=n[i],l[t]&&a.push(l[t][0]),l[t]=0;for(r in o)Object.prototype.hasOwnProperty.call(o,r)&&(c[r]=o[r]);for(d&&d(e);a.length;)a.shift()();return p.push.apply(p,u||[]),f()}function f(){for(var e,r=0;r<p.length;r++){for(var t=p[r],n=!0,o=1;o<t.length;o++){var u=t[o];0!==l[u]&&(n=!1)}n&&(p.splice(r--,1),e=s(s.s=t[0]))}return e}var t={},l={1:0},p=[];function s(e){if(t[e])return t[e].exports;var r=t[e]={i:e,l:!1,exports:{}};return c[e].call(r.exports,r,r.exports,s),r.l=!0,r.exports}s.e=function(u){var e=[],t=l[u];if(0!==t)if(t)e.push(t[2]);else{var r=new Promise(function(e,r){t=l[u]=[e,r]});e.push(t[2]=r);var n,i=document.createElement("script");i.charset="utf-8",i.timeout=120,s.nc&&i.setAttribute("nonce",s.nc),i.src=s.p+"static/js/"+({}[u]||u)+"."+{3:"634070b1",4:"fd0c0280",5:"55d21e5f",6:"9c26f39b"}[u]+".chunk.js",n=function(e){i.onerror=i.onload=null,clearTimeout(a);var r=l[u];if(0!==r){if(r){var t=e&&("load"===e.type?"missing":e.type),n=e&&e.target&&e.target.src,o=new Error("Loading chunk "+u+" failed.\n("+t+": "+n+")");o.type=t,o.request=n,r[1](o)}l[u]=void 0}};var a=setTimeout(function(){n({type:"timeout",target:i})},12e4);i.onerror=i.onload=n,document.head.appendChild(i)}return Promise.all(e)},s.m=c,s.c=t,s.d=function(e,r,t){s.o(e,r)||Object.defineProperty(e,r,{enumerable:!0,get:t})},s.r=function(e){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(e,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(e,"__esModule",{value:!0})},s.t=function(r,e){if(1&e&&(r=s(r)),8&e)return r;if(4&e&&"object"==typeof r&&r&&r.__esModule)return r;var t=Object.create(null);if(s.r(t),Object.defineProperty(t,"default",{enumerable:!0,value:r}),2&e&&"string"!=typeof r)for(var n in r)s.d(t,n,function(e){return r[e]}.bind(null,n));return t},s.n=function(e){var r=e&&e.__esModule?function(){return e.default}:function(){return e};return s.d(r,"a",r),r},s.o=function(e,r){return Object.prototype.hasOwnProperty.call(e,r)},s.p="/",s.oe=function(e){throw console.error(e),e};var r=window.webpackJsonp=window.webpackJsonp||[],n=r.push.bind(r);r.push=e,r=r.slice();for(var o=0;o<r.length;o++)e(r[o]);var d=n;f()}([])</script><script src="/static/js/2.bf1eb599.chunk.js"></script><script src="/static/js/main.a02e112f.chunk.js"></script></body></html>

Hi @prasanth55555. At what point do you get the error above?

Hi @Juste i was trying to add a lookup table as with the above mentioned data. after training rasa using (rasa train) i was about to see how NLU classifies the model, so i tried to hit the localhost:5002/model/parse with the text i wanted to parse, i got the above error. i am adding a screenshot of my postman request .Thanks.

hi @Juste @shubham @akelad @erohmensing can any one say whats wrong with the training data?

Which version of Rasa are you using?

@akelad Am using rasa==1.1.4 rasa-sdk==1.1.0 rasa-x==0.19.2

the packages am using are:

absl-py==0.7.1 aiofiles==0.4.0 aiohttp==3.5.4 alembic==1.0.10 APScheduler==3.6.0 asn1crypto==0.24.0 astor==0.8.0 async-generator==1.10 async-timeout==3.0.1 attrs==19.1.0 blis==0.2.4 boto3==1.9.175 botocore==1.12.175 certifi==2019.6.16 cffi==1.12.3 chardet==3.0.4 Click==7.0 colorclass==2.2.0 coloredlogs==10.0 colorhash==1.0.2 conda==4.6.14 ConfigArgParse==0.14.0 cryptography==2.7 cycler==0.10.0 cymem==2.0.2 decorator==4.4.0 docopt==0.6.2 docutils==0.14 en-core-web-md==2.1.0 fakeredis==1.0.3 fbmessenger==6.0.0 Flask==1.0.3 Flask-Cors==3.0.8 future==0.17.1 gast==0.2.2 gevent==1.4.0 greenlet==0.4.15 grpcio==1.21.1 h5py==2.9.0 httptools==0.0.13 humanfriendly==4.18 idna==2.8 idna-ssl==1.1.0 isodate==0.6.0 itsdangerous==1.1.0 Jinja2==2.10.1 jmespath==0.9.4 joblib==0.13.2 jsonpickle==1.2 jsonschema==2.6.0 kafka-python==1.4.6 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 kiwisolver==1.1.0 Mako==1.0.12 Markdown==3.1.1 MarkupSafe==1.1.1 matplotlib==3.1.0 mattermostwrapper==2.1 mixpanel==4.4.0 mock==3.0.5 multidict==4.5.2 murmurhash==1.0.2 networkx==2.3 numpy==1.16.4 packaging==19.0 pika==1.0.1 plac==0.9.6 preshed==2.0.1 prompt-toolkit==2.0.9 protobuf==3.8.0 pycosat==0.6.3 pycparser==2.19 pydot==1.4.1 PyJWT==1.7.1 pykwalify==1.7.0 pymongo==3.8.0 pyOpenSSL==19.0.0 pyparsing==2.4.0 PySocks==1.7.0 python-crfsuite==0.9.6 python-dateutil==2.8.0 python-editor==1.0.4 python-engineio==3.8.1 python-socketio==4.1.0 python-telegram-bot==11.1.0 pytz==2019.1 PyYAML==5.1.1 questionary==1.1.1 rasa==1.1.4 rasa-sdk==1.1.0 rasa-x==0.19.2 redis==3.2.1 requests==2.22.0 requests-toolbelt==0.9.1 rocketchat-API==0.6.32 ruamel-yaml==0.15.46 ruamel.yaml==0.15.97 s3transfer==0.2.1 sanic==19.3.1 Sanic-Cors==0.9.8.post1 sanic-jwt==1.3.2 Sanic-Plugins-Framework==0.7.0 scikit-learn==0.21.2 scipy==1.3.0 simplejson==3.16.0 six==1.12.0 sklearn-crfsuite==0.3.6 slackclient==1.3.2 sortedcontainers==2.1.0 spacy==2.1.4 SQLAlchemy==1.3.5 srsly==0.0.7 tabulate==0.8.3 tensorboard==1.13.1 tensorflow==1.13.1 tensorflow-estimator==1.13.0 termcolor==1.1.0 terminaltables==3.1.0 thinc==7.0.5 tqdm==4.32.2 twilio==6.28.0 typing-extensions==3.7.4 tzlocal==1.5.1 ujson==1.35 urllib3==1.24.2 uvloop==0.12.2 wasabi==0.2.2 wcwidth==0.1.7 webexteamssdk==1.1.1 websocket-client==0.54.0 websockets==6.0 Werkzeug==0.15.4 yarl==1.3.0

hey @prasanth55555, can you try out testing the api as mentioned here:

also can try to change the port number to “5005

@JiteshGaikwad thanks. once i downgraded the rasa version it seems to be working but the issue am facing is my nlu is not able to identify the slot values from look up table can you help me with it.?please look into the pipeline as well

intent:searchBookIntent

lookup:sBook

  • harry potter
  • the hobbit
  • walk to freedom
  • wings of fire
  • the walking dead
  • harry potter and the chamber of secrets
  • to kill a mockingbird
  • the great gatsby
  • the adventure of huckleberry finn
  • the catcher in the rye
  • the grapes of wrath
  • of mice and men
  • fahrenheit
  • the scarlet letter
  • in cold blood
  • east

lookup:sAuthor

  • abdul kalam
  • rowling
  • marie
  • maria
  • mahesh
  • keerthana
  • prasanth
  • praveen
  • mohan
  • mahesh

pipeline am using is:

  • name: “WhitespaceTokenizer”#- name: “intent_entity_featurizer_regex”
  • name: “ner_crf” features: [ [“low”, “title”, “upper”], [“bias”, “low”, “prefix5”, “prefix2”, “suffix5”, “suffix3”,“suffix2”, “upper”, “title”, “digit”, “pattern”], [“low”, “title”, “upper”] ]
  • name: “RegexFeaturizer”
  • name: “CRFEntityExtractor”
  • name: “EntitySynonymMapper”
  • name: “CountVectorsFeaturizer”
  • name: “EmbeddingIntentClassifier”

thanks in advance

1 Like

hey @prasanth55555, can you try changing your nlu config to below and test:

language: en

pipeline: supervised_embeddings

@JiteshGaikwad jiteshgaikwad ,still now after changing the pipeline the model was not able to recognize the data that was given in the look up table.

1 Like

@JiteshGaikwad I am facing the same issue. with Rasa 1.1.4. Anyone get solution or mistake or missing things ?

hey @shivangpatel you refer this example they are using lookup tables.

GitHub - RasaHQ/tutorial-knowledge-base: Integrating Rasa with a knowledge base to encode domain knowledge and resolve entities

1 Like

Hi @JiteshGaikwad , I checked your link. everything looks fine to me with my code. still unable use lookup table. I tried both ways - added lookup in seprate file and nlu file. and also checked with both version 1.1.4 and 1.1.6

hey @shivangpatel, I had used lookup tables in Restaurant bot for locations & cuisines entity extraction, it works for me, you can check it out here:

1 Like

@JiteshGaikwad : It’s a working. My application also working now. What I do is, just added more examples for location entity, like your nlu.md file. Now, It’s a working perfectly fine. Thank you for great efforts and demo.

@shivangpatel, glad that it worked :smile:

1 Like

The trick is you have to include enough examples in your nlu.md file, so that the model can create a regex pattern that includes all the different possibilities in your lookup table. Because first it checks with the regex pattern can this be in my lookup table and then it matches the value.