Early Stop Callback does not work with Keras LSTM policy

Rasa Open Source
1

Hi!

I have defined an Early Stop callback for the Keras LSTM policy and I pass it to the Agent as follows:

kwargs["callbacks"] = [
        EarlyStopping(monitor="val_acc", mode="max", min_delta=0.2, patience=2)
    ]

agent.train(training_data, **kwargs)

However, it does not work. Any idea on what I am doing wrong?

Thanks,
L-P from Dialogue

2

Hi, we don’t support passing policy parameters through agent. You should initialize KerasPolicy with it

3

Thanks, this works :slight_smile:

4

Hello

I have also been trying to get EarlyStopping to work.

I tried subclassing KerasPolicy and passing the callback as kwargs.

class OurKerasPolicy(KerasPolicy):
def __init__(self, **kwargs: Any):
    kwargs["callbacks"] = [
        EarlyStopping(monitor="val_loss", mode="min", min_delta=0.5, patience=2),
        debug_callback
    ]
    print(f"KerasPolicy kwargs = {kwargs}")
    super().__init__(**kwargs)

But the callback is never triggered.

5

could you please check that callbacks are actually passed to self.model.fit?