Conversation log using Redis

[jbasix2002 (Dan)]

Hi, I am not an expert programmer. My rasa core is 0.11.12. I am wondering in the tracker store, can we generate the bot log file in these two formats below. If so, can anyone suggest how I can achieve this?

Format 1

Format 2

Basically, I want the log file, or the log file needs to contain the information:

1. user ID
2. time
3. messages they send to the bot
4. the bot’s reply

Does the rasa tracker store store these information? and can we export the frequency of bot’s reply and intents to another log file?

Thank you

[ricwo]

Hi @jbasix2002, yes, the Redis tracker store contains all of that information. Check out the docs on the RedisTrackerStore here

[jbasix2002 (Dan)]

Hi, thank you, @ricwo. I am wondering if there’s a way to capture the intents in the RedisTrackerStore?

I found that in the rasa tracker store, there’s file stored these pieces of information like below

But it does not label the intents, and the bot utterances.

Thank you so much for your help.

[ricwo]

@jbasix2002 which version of Rasa Core are you running?

The RedisTrackerStore persists the entire conversation. User messages show up under the events key and have entries like

{
      "timestamp": 1517821726.200036,
      "parse_data": {
        "entities": [],
        "intent": {
          "confidence": 0.54,
          "name": "greet"
        },
        "text": "/greet",
        "intent_ranking": [
          {
            "confidence": 0.54,
            "name": "greet"
          },
          {
            "confidence": 0.31,
            "name": "goodbye"
          },
          {
            "confidence": 0.15,
            "name": "default"
          }
        ]
      },
      "event": "user",
      "text": "/greet",
      "input_channel": null
    }

from which you can read or process the intent.

[jbasix2002 (Dan)]

@ricwo: Thank you so much. My rasa core is 0.11.12.

[ricwo]

@jbasix2002 please upgrade to 0.12.3, and you should be able to see events like the one I posted in your Redis tracker store

[jbasix2002 (Dan)]

@jbasix2002

0.11.12.

@ricwo: once we updated it, then we need to re-work the whole custom action…Is there a way to log without updating it? Happy New Year…

[ricwo]

All the event information should be included in your RedisTrackerStore, even in 0.11.12. You can read out the tracker store with

from rasa_core.tracker_store import RedisTrackerStore
import json

tracker_store = RedisTrackerStore(domain, host='localhost',
                 port=6379, db=0, password=None, event_broker=None,
                 record_exp=None)
tracker = tracker_store.retrieve(sender_id)
print(json.dumps(tracker.current_state(), indent=2))

[ASINGH (Ashutosh)]

My conversation is being stored in Redis and I can see in some unreadable format. I want to read stored data from Redis I have used your and passed the sender_id. But I am getting error

AttributeError: ‘NoneType’ object has no attribute ‘current_state’

In below code I am able to connect with Redis

tracker_store = RedisTrackerStore(domain, host='localhost',
             port=6379, db=0, password=None, event_broker=None,
             record_exp=None)

But beyond this it is not working. How can we read conversation for all the senders or individual sender. I am using RASA core version: 0.11. Please help and post python code for the same task

[ASINGH (Ashutosh)]

@ricwo can you please help in above problem

[ricwo]

Please confirm that your sender_id exists in Redis:

import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter():
       print(key)

[ASINGH (Ashutosh)]

@ricwo No my sender_id does not exist in listed keys. However I am passing sender ID to RASA HTTP server using below URL:

http://WEB-SER-DEVP:5005/conversations/31912110/respond

But I can see some information stored in Redis under a different key in Hexa format:

127.0.0.1:6379> get 581fbc3d89dc4513a4f2a6a7f9ceaed7

Edit:

Now what i did I thought I would clear all data from redis and will start fresh. I used command FLUSHDB . It cleared all data from db 0. Now when I again started communication, it is not connecting with Redis at all. But chat data is available in InMemoryTracker(Default)` which I can see using

curl --request GET --url http://localhost:5005/conversations/31912110/tracker at my rasa server.

I have defined following in endpoints.yml

tracker_store:
    store_type: redis
    url: 'localhost'
    port: 6379
    db: 0
    password: 'JOHN@123'

and While loading agent I am passing following:

tracker_stores = RedisTrackerStore('domain.yml', host='localhost',port=6379, db=0, password="JOHN@123", event_broker=None)

agent = Agent('domain.yml', policies = [MemoizationPolicy(max_history = 3), KerasPolicy(),fallback],tracker_store=tracker_stores)

data = agent.load_data('./data/stories.md')

However while doing interactive learning data is being stored in Redis in Hexa format under ker default

Kindly tell me what is the mistake I am making. Thanks

[ricwo]

@ASINGH, interactive learning intentionally assigns user ids in hex format. this is because they’re not real user conversations

[ASINGH (Ashutosh)]

@ricwo , Thanks. I am able to store data now into redis and it is being stored in Hexadecimal format. When I execute this below code I get key 31912110:

import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter():
   print(key)

But in below given code I am getting error:

tracker_store = RedisTrackerStore(domain, host='localhost',
             port=6379, db=0, password=None, event_broker=None,
             record_exp=None)
tracker = tracker_store.retrieve(sender_id)

print(json.dumps(tracker.current_state(), indent=2))

Traceback (most recent call last):

File "E:\Python and Tensorflow\weatherbot_smalltalk\ReadTracker.py", line 28, in <module>

tracker = tracker_stores.retrieve(‘31912110’)

File "c:\users\31960460\appdata\local\programs\python\python36\lib\site-packages\rasa_core\tracker_store.py", line 180, in retrieve

return self.deserialise_tracker(sender_id, stored)

File "c:\users\31960460\appdata\local\programs\python\python36\lib\site-packages\rasa_core\tracker_store.py", line 123, in deserialise_tracker

tracker = self.init_tracker(sender_id)

File "c:\users\31960460\appdata\local\programs\python\python36\lib\site-packages\rasa_core\tracker_store.py", line 77, in init_tracker

self.domain.slots)

AttributeError: ‘str’ object has no attribute ‘slots’

Can you please provide code snippet to this to get this conversation in readable format.

    rasa-core==0.12.3

    rasa-core-sdk==0.12.1

    rasa-nlu==0.13.7
    redis==2.10.6

[mohamedHassanKa (Mohamed Hassan Kadri)]

can you share with us how you managed to connect redis with ur bot and u managed to get that output of redis because i am failling to get it

[mohamedHassanKa (Mohamed Hassan Kadri)]

the information are stored in dump.rdb redis file as hexadecimal i was wandering how are you converting this file guys and how to structure exactly the data like the format 1 or in csv file

[vigneshp826]

@ASINGH , did you resolved this issue? if yes, can you share the solution?

Update:

Issue resolved by adding below two lines to the Ricwo script.

from rasa.core.domain import Domain

domain = Domain.load("/home/centos/rasa-x/domain.yml") —>Path of domain file

[ASINGH (Ashutosh)]

Thanks, Yes it was resolved at that time by same approach you mentioned. Missing domain file was key. Sorry for very very very late confirmation

[samruddhi02 (samruddhi)]

I am using RedisTrackerStore to store all the conversations between a user and a bot in to Redis database and I tried retrieving those conversation using the following method: tracker_store = RedisTrackerStore(domain, host=‘localhost’, port=6379, db=0, password=None, event_broker=None, record_exp=None) tracker = tracker_store.retrieve(sender_id)

print(json.dumps(tracker.current_state(), indent=2))

after getting the key from below script: import redis r = redis.StrictRedis(host=‘localhost’, port=6379, db=0) for key in r.scan_iter(): print(key)

but I am getting only the latest or the last message of the conversation, is there any way to retrieve the whole conversation of the bot and the user?

[lokkrish (lokesh)]

@ricwo In the redis tracker store configuration, what does ‘record_exp’ parameter mean? If we don’t set this, does it mean the events are saved forever and i can expect memory and latency issues in future?