What kind of information does DM get from NLU? nature of DM vs NLU integration

Rasa Open Source
1

Hi,

I am trying to get a sense of the kinds of inputs that DM gets from the NLU pipeline.

Is it only the output from the end of the pipeline that gets passed to the DM?

As a newbie, my sense is that NLU only sends intents and entities to the DM. All of the examples of NLU output shown at Components and intent and/or entity information. (If there is a spec describing the full extent of stuff DM can receive from NLU, please do tell! (-: — haven’t been able to find one yet)

I mean, yeah, there’s a lot of fine-grained detail about intents and entities (e.g. confidences, intent rankings, start and end char positions for entities etc etc).

However, my question is this: is there important non-entity and non-intent info being passed from the various native Rasa pipeline configurations to the DM?

For example, does, say, the tokenizer pass info to the DM?

Here’s the motivation for my question: we have a strong need to use our in house NLU. I’ve been reading up on helpful resources such as Using Custom NLU with Rasa X - #5 by Tobias_Wochinger and NLU-Only Server.

We take interest in @Tobias_Wochinger’s comment at Using Custom NLU with Rasa X - #5 by Tobias_Wochinger where he says " I can only strongly recommend to use the integrated NLU in Rasa Open Source" and “team which is continuously working on improvements for NLU and also integrating the dialogue management with the NLU predictions more and more which should lead to more robust decisions by the bot”.

So, we are trying to get a sense of the nature of the tight integration between Rasa NLU and Rasa DM.

If Rasa NLU / Rasa DM integration is built around the idea that only intent and entity related into gets passed from NLU to DM, that’s one thing. But if all sorts of random info apart from intents and entities get passed to DM, that’s another thing. (e.g. we stand to loose more if we roll our own NLU).

Even worse would be if Rasa DM passed information back to the Rasa NLU. Does this happen? (If so, this would be an even bigger “loose” for those of us wanting to use our own NLU bc our in house NLU (currently) does not understand such inputs).

2

A good place to start would be to look at the tracker store which contains an array of events. This json structure will show you the data that is available. You can access the tracker store to control the dialog.

Here’s an example custom action that reviews the confidence scores to decide the next step in the conversation.

1 Like
3

Thanks @stephens .

It looks like if I am going to use this Tracker thing I need to be running an Action Server, right?

And how to start that up? See Running a Rasa SDK Action Server. I’ve tried rasa run actions as suggested on that page and it is not working. I will open this problem on another thread soon (assuming I don’t find a solution in the forum).

4

Yes, nearly all Rasa chatbots will have an action server however you can also use the http api to access the tracker.

1 Like
5

Thanks.

Okay, well, after a very brief try I did not have any luck with “use the http api”, but there are many more tricks up my sleeve before I will ask for help regarding that.

What I DID do was fix my earlier problem. My earlier problem, to recap, was that I could not run the action server.

The instructions at Running a Rasa SDK Action Server say that one way you can start an action server is to do this:

unix% rasa run actions

Well, I tried that and I got the following error…

unix% rasa run actions 2021-10-26 09:27:04 INFO rasa_sdk.endpoint - Starting action endpoint server… 2021-10-26 09:27:04 ERROR rasa_sdk.executor - Failed to register package ‘actions’. Traceback (most recent call last): File “/Users/tess/Dialog/Rasa/Code/env/lib/python3.8/site-packages/rasa_sdk/executor.py”, line 262, in register_package self._import_submodules(package) File “/Users/tess/Dialog/Rasa/Code/env/lib/python3.8/site-packages/rasa_sdk/executor.py”, line 218, in _import_submodules package = self._import_module(package) File “/Users/tess/Dialog/Rasa/Code/env/lib/python3.8/site-packages/rasa_sdk/executor.py”, line 239, in import_module module = importlib.import_module(name) File “/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/importlib/init.py”, line 127, in import$ return _bootstrap._gcd_import(name[level:], package, level) File “”, line 1014, in _gcd_import File “”, line 991, in _find_and_load File “”, line 973, in _find_and_load_unlocked ModuleNotFoundError: No module named ‘actions’

…well, it turns out the fix was very simple. Alas, the instructions at the above page do not tell you this but I figured out that I need to have a subdirectory in my project directory like so…

unix% ls -R actions/

init.py

actions.py

…and what is the content of actions.py? Well, you can get a nice starter stub for actions.py if you do this…

rasa init

…it will make a nicer fresh “starter kit” of the needed files e.g. data/nlu.yml, etc etc AND this actions/ directory I speak of.

So, IN SUM: to start an actions sever you can call ‘rasa run actions’ and this is what it will probably look like…

% rasa run actions 2021-10-26 15:03:20 INFO rasa_sdk.endpoint - Starting action endpoint server…

2021-10-26 15:03:20 INFO rasa_sdk.endpoint - Action endpoint is up and running on http://0.0.0.0:5055

…BUT to make that work, you need to have an actions/ directory in your rasa project dir as described above.

Now, what one can do with http://0.0.0.0:5050 is currently unknown to me (and critically, maybe that is involved in using the Tracker). That is what I will be trying to figure that out next.

6

@stephens Thanks for the suggestion to use the “http to access the tracker.”

After about 1 or 2 hours of banging around I finally think I have figured out how to start the API.

The key was googling “rasa api”.

The answer came from a search result to “rasa api”. The answer is on this page: Rasa Open Source HTTP API

The way to “use the api” in a “Hello World” sense is the following:

unix % rasa run --enable-api

What you will see is the following:

(env) tess@MyMac project % rasa run --enable-api 2021-10-26 16:05:44 INFO root - Starting Rasa server on http://localhost:5005 2021-10-26 16:05:44 INFO rasa.model - Loading model models/20211026-142006.tar.gz… /Users/tess/dialog/Code/env/lib/python3.8/site-packages/rasa/utils/train_utils.py:641: UserWarning: constrain_similarities is set to False. It is recommended to set it to True when using cross-entropy loss. It will be set to True by default, Rasa Open Source 3.0.0 onwards. rasa.shared.utils.io.raise_warning( /Users/tess/dialog/Code/env/lib/python3.8/site-packages/rasa/shared/core/slots.py:311: FutureWarning: UnfeaturizedSlot is deprecated and will be removed in Rasa Open Source 3.0. Please change the type and configure the ‘influence_conversation’ flag for slot ‘email’ instead. rasa.shared.utils.io.raise_warning( 2021-10-26 16:05:59 INFO root - Rasa server is up and running.

That will lock up that terminal window.

However, in another terminal window you can do things like this…

unix% curl http://localhost:5005

…and the result will be this…

Hello from Rasa: 2.8.7

And you can try other commands from certain aspect of the API documentation (e.g. Rasa Open Source Documentation)

For example, you can do this…

unix% curl http://localhost:5005/version

…and get back…

{“version”:“2.8.7”,“minimum_compatible_version”:“2.8.0”}

7

@stephens Sorry to keep tagging you. At each post, I think I’m stuck and/or I feel I need to document what I have figured out so far just to make sure I am not flying off in foolish or irrelevant direction.

NOW, I think I am finally “getting” the full force of what you are tell me.

So, what I am asking you in this message is this: Have I basically gotten your main point.

You main point was this:

If one wants to see what the totality of what DM gets from NLU, then one solution is to use ‘the tracker’. Now, how does one actually use the tracker? Well, here is one way for a newbie to use it…

(1) Step 1: get a Rasa project configured (e.g. you need intents, stories, etc etc. For example get the simple project here: Rasa Playground. You click on “download project”. Unzip the downloaded file. cd into the resultant directory.

(2) Step 2: The Key Magic: Invoke the Bot using this fancy command:

rasa shell --enable-api --conversation-id myconversation

(hrm, you might have had to do unix % rasa train first. Whatever it is a detail).

If you succeed at the above you should see all sorts of trace messages spewing by. And then at the end of the spew you will see something like this…

2021-10-26 16:25:43 INFO root - Rasa server is up and running. Bot loaded. Type a message and press enter (use ‘/stop’ to exit): Your input →

(3) Step 3: Actually interrogate the tracker.

Got to a different terminal window, where you have a unix command line prompt. Paste this to your unix prompt…

curl http://localhost:5005/conversations/myconversation/tracker

Nothing will happen. That is expected.

Next, go back to the terminal window that has the bot prompt, i.e. “Your input ->”. Type in some text there. For example you might do this…

Your Inputs → hello

…The bot will answer with something like…

Hello! How can I help you?

…NOW, you have the “gold” you want. What you do is go back to the other terminal, the one where you did the curl command. Remember how above I said “Nothing will happen”. Well, finally something DID happen. When you go to that terminal window, viola you will see a HUGE json blob. That blob, must be the tracker showing you what it has tracked. (RIght?)

Here’s an example of the first couple hundred words of the spew that showed up in response to my curl command…

{“sender_id”:“myconversation”,“slots”:{“email”:null,“requested_slot”:“email”,“session_started_metadata”:null},“latest_message”:{“intent”:{“id”:3083014051725182201,“name”:“subscribe”,“confidence”:0.9810560345649719},“entities”:[],"tex

So, @stephens my question to you, is this a very specific “how to” of what you were pointing me at?

If yes, great.

If not, can you please explain a little what would be a better direction to go in, or what is wrong about my approach?

8

I recommend you go thru the Rasa certification course.

9

The course is not allowed to register, any other choice?