What was the reasoning behind `new_request` in ActionQueryKnowledgeBase?

Hi there!

I started creating a POC with the Knowledge Base concept before going further into the breadth of custom actions. Of course when your needs are a tiny bit outside of the current simple (yet guiding!) ActionQueryKnowledgeBase implementation, you need to extend/customize it a lot.

I encountered this new_request condition, seen below:

This basically prevents direct questions about an object without prior history or an uttering of a list of objects. Was there a specific reason for this limitation? When I remove that condition, it does work the way I want. I am just curious if you had some insights for it.

Thank you,

Hey @oytun, thanks for the question! I think the condition is there due to the basic use cases we considered when creating the action: Users ask for objects of specific type and only once some relevant objects are retrieved and brought into the conversation, asking about the attribute of an object makes sense.

Of course, now I’m curious: What’s your use case? What would be an example user utterance where they ask for objects of a specific time and also ask about the attribute of an object?

By the way, you’re more than welcome to change the code; as I said, it’s designed to accommodate some basic use cases but definitely not all!

Thanks for the input, @SamS!

We’ve come a good way through implementing our own KnowledgeBase and ActionQueryKnowledgeBase objects by using your initial implementation in the SDK.

Our use case is initially to support our operations team. MotaWord is a human translation platform with 20K translators, tens or hundreds of projects coming in and out every day in 100 languages. I am creating a conversational experience for our community/project managers to access information faster.

Some intent examples:

  • Hey bot, what is the progress of project 288822?
  • Great that it’s 100% completed. Can you deliver this to the client?
  • Who is the client of that project?
  • Hmm, can you tell me about their recent completed projects?
  • Who is the Italian proofreader of project 199282?
  • When did they become a proofreader?
  • Hmm, I’d better look at this in detail. Can you give me the vendor’s profile link in admin panel?

The domain at MotaWord is huge and quite unique/complex in the language-related industry (we are the pioneering collaborative translation platform, so everything is realtime with thousands of stakeholders moving at any given time).

The bigger things we’ve changed so far:

  • Our own KnowledgeBase object that talks to our API
    • No more JSON file data like in InMemoryKB… which was quite useful initially while making non-data related changes in the codebase. We kept this for a while until our code changes were more stable. Made the testing much easier than talking to our API in each action.
    • Our own object type and attributes domain shapes. For instance, we now have a list of attributes per object type and an attribute can configure its own utter_template.
    • Some smaller decisions such as “get object by ID or representation” are moved from ActionQuery to APIKnowledgeBase. So it can decide whether to get an object from our API by ID or search by name or something.
  • In the SDK implementation, it requires a list of objects to exist beforehand to query its attributes, like you said. This has changed in our implementation. A user can directly query about an object. We will go fetch that object from API and return its attribute.
  • One important change was to make our fork of ActionQueryKnowledgeBase an abstract class. For this, we used abc and defined name method as abstract. Something like this:
class ActionQueryKnowledgeBase(Action, metaclass=abc.ABCMeta):
...
    # you need to define at least 1 abstractmethod, otherwise Rasa will try to register this Action
    # which is not what we want. This Action is to be extended by Knowledge Base implementations.
    @abc.abstractmethod
    def name(self) -> Text:
        return "action_query_knowledge_base"
  • So we extend this ActionQueryKnowledgeBase in our GenericKnowledgeBaseAction which is the actual action registered in action server. This GenericKnowledgeBaseAction is a very very thin class.
  • ActionQueryKnowledgeBase does not simply utter messages anymore, it utters templates.
  • As we had already previously integrated our NLG server (with our own , uttering templates should let us use our NLG in KnowledgeBase responses.
    • This was fun, too. We created our own NLG class by extending NaturalLanguageGenerator. It first tries to use our NLG server with template name, if not available, falls back to Rasa’s existing TemplatedNaturalLanguageGenerator.
    • It was even more fun to debug TemplatedNaturalLanguageGenerator.templates. I figured it loads the templates async, so we assign templates to this variable in generate method :frowning: it was a sad moment, suggestions are welcome.

Overall, our fork of ActionQueryKnowledgeBase and KnowledgeBase has quite a bit differences, but not completely unrecognizable.

Context switching is the biggest problem in KnowledgeBase capabilities. If the mention mechanism is improved, I think KnowledgeBase concept can go a very long way. (implement coreference libraries like huggingface’s?)

I wanted to give you guys a long summary of what I have been dealing with recently, I hope it all makes sense and can contribute to your understanding of Rasa. I am always open to live discussions and even codebase walk-through.

Oytun

2 Likes

Thanks a lot @oytun, I really appreciate the details of how you made things work for your challenging use case, especially how you effectively added more layers in both directions to improve search and responses. While I personally don’t work on KBs, I think some others will definitely want to hear more from you – perhaps now, but certainly later when some more related research projects are on the table.

You’re right, there’s a lot of potential in KBs – though the accompanying challenges are also very tough and no existing library is perfect in this respect. Especially in dialogue where the context changes with every utterance, resolving mentions has to account for these dynamics – and things get really hard the moment we step beyond the most simplistic use cases :slightly_smiling_face:

1 Like