Custom Policy Not Working

I’m trying to create a custom policy that predicts one specific action all the time when a slot is set, but it is not working for some reason. Below is my policy’s code:

from rasa.core.policies import Policy
from rasa.core.policies.policy import confidence_scores_for
import numpy as np

class MyPolicy(Policy):
    def predict_action_probabilities(self, tracker, domain):
        if tracker.get_slot("my_slot") == True:
            return confidence_scores_for("action_my_custom_action", 1.0, domain) 
        else:
            return np.zeros(domain.num_actions)

the policy file is in a directory called custom_policies/ in a file called testpolicy.py. I also have a file called __init__.py in that directory. Below is the relevant part of my config.yml:

...
policies:
  - name: MemoizationPolicy
    max_history: 1
  - name: MappingPolicy
  - name: FormPolicy
  - name: TwoStageFallbackPolicy
  - name: "custom_policies.testpolicy.MyPolicy"
    priority: 15
1 Like

Hi @basil-chatha, when you say it doesn’t work what do you mean? Is there an error? Or does the policy not get used? If it’s the second case, could you post the debug logs of the conversation?

Also, are you sure using a custom policy is the right course here? It could also be achieved with a story that predicts a handoff and then pauses the conversation for example.

@akelad so the error I was getting before was because my model was not training using the rasax ui so I had to fix that. Here are the error logs I get for rasa-production now:

rasa-production_1  | Traceback (most recent call last):
rasa-production_1  |   File "/opt/venv/lib/python3.7/site-packages/rasa/core/agent.py", line 100, in _load_and_set_updated_model
rasa-production_1  |     policy_ensemble = PolicyEnsemble.load(core_path)
rasa-production_1  |   File "/opt/venv/lib/python3.7/site-packages/rasa/core/policies/ensemble.py", line 253, in load
rasa-production_1  |     policy_cls = registry.policy_from_module_path(policy_name)
rasa-production_1  |   File "/opt/venv/lib/python3.7/site-packages/rasa/core/registry.py", line 24, in policy_from_module_path
rasa-production_1  |     raise ImportError(f"Cannot retrieve policy from path '{module_path}'")
rasa-production_1  | ImportError: Cannot retrieve policy from path 'custom_policies.testpolicy.MyPolicy'
rasa-production_1  | 2020-07-16 02:25:57 ERROR    rasa.core.agent  - Failed to load policy and update agent. The previous model will stay loaded instead.
rasa-production_1  | Traceback (most recent call last):
rasa-production_1  |   File "/opt/venv/lib/python3.7/site-packages/rasa/core/registry.py", line 21, in policy_from_module_path
rasa-production_1  |     module_path, lookup_path="rasa.core.policies.registry"
rasa-production_1  |   File "/opt/venv/lib/python3.7/site-packages/rasa/utils/common.py", line 198, in class_from_module_path
rasa-production_1  |     m = importlib.import_module(module_name)
rasa-production_1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
rasa-production_1  |     return _bootstrap._gcd_import(name[level:], package, level)
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
rasa-production_1  |   File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
rasa-production_1  | ModuleNotFoundError: No module named 'custom_policies'

Have you mounted this file to the rasa-production container in your docker-compose.override.yml?

@akelad Ok thanks yes now I’m doing that and am no longer getting that error. For some reason this method is not triggering the action I want it to run for some reason. Here’s my code:

is_triggered = tracker.get_slot('my_slot')

if is_triggered:
    logger.info('if statement ' + str(is_triggered) + ' ' + str(type(is_triggered)))
    return confidence_scores_for("action_my_action", 1.0, domain)
else:
    logger.info('in else statement ' + str(is_triggered) + ' ' + str(type(is_triggered)))
    return self._default_predictions(domain)

I’m not getting any errors in my logs and I know it is triggering this policy now because I was getting errors from within the policy earlier and am no longer getting them. Any suggestions?

@basil-chatha can you post the debug logs of the conversation where the action isn’t being run? I want to make sure that the policy is actually the one predicting the action

@akelad Actually I retried it and it seems to be working now. However, I’m getting a circuit breaker tripped error. Is that likely because my model is not trained on stories with this new conversation flow?

Yes, that’s usually the issue with the circuit braker tripped error

@akelad I have been trying to train a model with a custom policy in rasax version 0.32.2 and rasa open source version 1.10.2, but when I include the following in domain.yml it is not able to train the model. When I don’t include this bit it trains fine. Do you know what the issue could be? I have mounted the custom_policies directory to my rasa-production container.

- name: "custom_policies.testpolicy.MyPolicy"
  priority: 6

i am trying to create a custom policy “MyPolicy” in botfront, which would just return a value.My main intention is to see whether the custom policy is getting registered in rasa or not.

below is my policy code and i have placed that in my botfront/rasa folder:

from rasa_sdk import Action, Tracker

#from rasa_sdk.events import SlotSet

from rasa_sdk.events import SessionStarted, ActionExecuted

import logging

from rasa_sdk.executor import CollectingDispatcher

from rasa_sdk.events import UserUtteranceReverted, ActionReverted

from rasa_sdk.events import SlotSet

from rasa_sdk.events import AllSlotsReset

from rasa_sdk.events import UserUtteranceReverted, ActionReverted, FollowupAction

from typing import List, Dict, Text, Optional, Any, Union, Tuple

import rasa.shared.utils.common

import rasa.shared.utils.io

from rasa.core.policies.policy import Policy, PolicyPrediction

class MyPolicy(Policy):

def __init__(

    self,

    lookup: Optional[Dict] = None,

    **kwargs: Any,

) -> None:

    # max history is set to 2 in order to capture

    # previous meaningful action before action listen

    super().__init__(

        lookup=lookup,

        **kwargs,

    )

    print("in policy*****************************************************************")

def predict_action_probabilities(self, tracker, domain):

    if tracker.get_slot("version") == "15.9.0":

        return confidence_scores_for("action_hi", 1.0, domain)

    else:

        return "hiiiiii"

def _metadata(self) -> Dict[Text, Any]:

    return {"lookup": self.lookup}

In my policies pipeline i have the below code: policies:

  • name: TEDPolicy epochs: 50 max_history: 5 batch_size:
    • 32
    • 64
  • name: RulePolicy
  • name: AugmentedMemoizationPolicy
  • name: MyPolicy

But it is throwing the below error:

2021-03-12 04:46:23 WARNING rasa.utils.common - Failed to write global config. Error: [Errno 13] Permission denied: '/.config'. Skipping.

2021-03-12 04:46:23 ERROR rasa.server - Traceback (most recent call last):

File "/opt/venv/lib/python3.7/site-packages/rasa/core/policies/ensemble.py", line 430, in from_dict

constr_func = registry.policy_from_module_path(policy_name)

File "/opt/venv/lib/python3.7/site-packages/rasa/core/registry.py", line 28, in policy_from_module_path

module_path, lookup_path="rasa.core.policies.registry"

File "/opt/venv/lib/python3.7/site-packages/rasa/shared/utils/common.py", line 31, in class_from_module_path

return getattr(m, module_path)

AttributeError: module 'rasa.core.policies.registry' has no attribute 'MyPolicy'

``

During handling of the above exception, another exception occurred:

``

Traceback (most recent call last):

File "/opt/venv/lib/python3.7/site-packages/rasa/server.py", line 1021, in train

training_result = await train_async(**training_payload)

File "/opt/venv/lib/python3.7/site-packages/rasa/train.py", line 178, in train_async

finetuning_epoch_fraction=finetuning_epoch_fraction,

File "/opt/venv/lib/python3.7/site-packages/rasa/train.py", line 368, in _train_async_internal

finetuning_epoch_fraction=finetuning_epoch_fraction,

File "/opt/venv/lib/python3.7/site-packages/rasa/train.py", line 431, in _do_training

finetuning_epoch_fraction=finetuning_epoch_fraction,

File "/opt/venv/lib/python3.7/site-packages/rasa/train.py", line 626, in _train_core_with_validated_data

model_to_finetune=model_to_finetune,

File "/opt/venv/lib/python3.7/site-packages/rasa/core/train.py", line 45, in train

policies = config.load(policy_config)

File "/opt/venv/lib/python3.7/site-packages/rasa/core/config.py", line 56, in load

return PolicyEnsemble.from_dict(config_data)

File "/opt/venv/lib/python3.7/site-packages/rasa/core/policies/ensemble.py", line 438, in from_dict

f"Module for policy '{policy_name}' could not "

rasa.core.policies.ensemble.InvalidPolicyConfig: Module for policy 'MyPolicy' could not be loaded. Please make sure the name is a valid policy.

``

2021-03-12 04:46:23 ERROR rasa.server - An unexpected error occurred during training. Error: Module for policy 'MyPolicy' could not be loaded. Please make sure the name is a valid policy.

@akelad Any help in this regard would be welcomed

@akelad i have two intents with the same utterance and marked as different entities in each case… For example : Intent 1 : invoke_process Utterances : PR2002 ,PR2003 Intent 2 : goto_object Utterances : PR2002 ,PR2003

So here i have same utterance marked as different entities in each case … and i have a separate story for each… but botfront does not allow me to do this … Is there any possibility to achieve this ?

I am also having problems loading custom policies, the same as above.

Should I put them in a custom folder in the root of my project, like ./policies/handoff.py ?

Then inside the handoff.py file, I have a class HandoffToHuman. So should my path in the config.yml file be then: - name: "policies.handoff.HandoffToHuman" or how should this look?

Hi,

yes, name should be a path to your handoff.py. I also added a priority and action name:

name: handoffPolicy.HandoffPolicy

priority: 10

handoff_action_name: “action_human_handoff”

Sorry, in your case, are you naming the file handoffPolicy.py and putting it in the root? And are you renaming the class to HandoffPolicy?

Since you changed the naming, it is unclear exactly where to put the file, and what to name the file, and the class inside it, to match the name in the policies section of the config.yml

I have tried dozens of permutations of the above and still unsuccessful in getting it to load while training

yes, i renamed file to handoffPolicy.py and put it in the root. and yes, i renamed a class to HandoffPolicy. I think in your case, it is suppose to work like that:

name: policies.handoff. HandoffToHuman

but what Error message do you receive ?

I would get InvalidPolicyConfig: Module for policy 'policies.handoff.HandoffToHuman' could not be loaded. Please make sure the name is a valid policy.

I tried your code above, gives me the same error even though I tried the space between the dot and the class name. If I try the following - name: "./policies/handoff. HandoffToHuman" it gives me TypeError: the 'package' argument is required to perform a relative import for './policies/handoff'

which seems closer to it actually working, but I can’t tell

@Aigerim If I put the file in the root, and rename it like you suggest above, I get:

InvalidPolicyConfig: Module for policy 'handoffPolicy.HandoffPolicy' could not be loaded. Please make sure the name is a valid policy.

So it’s similar to the first error

Which methods do you implement in your handoffPolicy.py? I had the same error, but after i implemented following methods:

predict_action_probabilities, _standard_featurizer, train, persist and load

everything worked. Look into the following post as a reference: Custom Policy Issue

I have __init__, train, and predict_action_probabilities but not the others. I didn’t find examples for the other two, but I will look into them

I attached the reference post for these methods. Custom Policy Issue

@classmethod
 
def load(cls, path: Text) -> "SpellCheckPolicy":

        meta = {}

        if os.path.exists(path):

            meta_path = os.path.join(path, "spellCheck_policy.json")

            if os.path.isfile(meta_path):

                meta = json.loads(rasa.utils.io.read_file(meta_path))

        return cls(**meta)



@staticmethod

def _standard_featurizer() -> None:

    return None


def persist(self, path: Text) -> None:

    """Persists the policy to storage."""

    config_file = os.path.join(path, "spellCheck_policy.json")

    meta = {

        "priority": self.priority,

        "spell_check_action_name": self.spell_check_action_name,    

    }

    rasa.utils.io.create_directory_for_file(config_file)

    rasa.utils.io.dump_obj_as_json_to_file(config_file, meta)
1 Like