Starting interactive learning at a certain point in the dialogue

I have a dialogue with about 10 steps, and the first 5-6 are more or less standard. User response variety occurs in the end. But every time I do interactive (online) training, I have to waste a couple of minutes going through the first 5-6 steps, plus occasionally make mistakes there that get me off the happy path. Is there anyway to go start the interactive learning at a certain point in the happy path?

Hey @lgrinberg. As of now, you would have to go trough the whole stores when you trin in interactive learning mode. Since it’s either re-training the model or loading an existing one, the way it makes predictions will depend on the training data your had in your stories.

Yes, Juste, we realized it as well. So we’re modifying the code to take the existing tracker and start of the interactive learning at a step past the “current state” of the tracker. We have a working POC, and now working to clean up the code.

Awesome! Let me know how it goes!

This is what I did: In “run.py” added this to the create_argument_parser():slight_smile:

parser.add_argument(
    '--tracker',
    type=str,
    default=None,
    help='starting with stored tracker file'
)

In the "do_interactive_learning, modify the Agent.load to take the tracker_filename:

_agent = Agent.load(cmdline_args.core,
							interpreter=_interpreter,
							generator=_endpoints.nlg,
							tracker_store=_tracker_store,
							action_endpoint=_endpoints.action,
							tracker_filename=cmdline_args.tracker)

In the Agent class definition (agent.py), modify the init method to take the tracker_filename and set the the new tracker_filename attribute. Agent class will now carry the tracker_filename containing the tracker object to start with:

def init( self, domain: Union[Text, Domain] = None, policies: Union[PolicyEnsemble, List[Policy], None] = None, interpreter: Optional[NaturalLanguageInterpreter] = None, generator: Union[EndpointConfig, ‘NLG’, None] = None, tracker_store: Optional[‘TrackerStore’] = None, action_endpoint: Optional[EndpointConfig] = None, fingerprint: Optional[Text] = None, tracker_filename: Optional[Text] = None ):

	self.tracker_filename = tracker_filename

Then, modify the load method of the Agent class, to take and return tracker_filename.

So now we have the Agent class carry the tracker_filename from the command line.

The “run_interactive_learning” function in training/interactive.py will invoke the “create_app” function in server.py: app = server.create_app(agent)

We will modify the “create_app” method in the server.py. We don’t need to modify the interactive.py as we’re passing the agent object that is now carrying the tracker_filename as one of its attributes.

def create_app(agent,
			   cors_origins: Optional[Union[Text, List[Text]]] = None,
			   auth_token: Optional[Text] = None,
			   jwt_secret: Optional[Text] = None,
			   jwt_method: Optional[Text] = "HS256",
			   ):
	"""Class representing a Rasa Core HTTP server."""
	tracker_filename = agent.tracker_filename

Then we will modify the “retrieve_tracker” function within “create_app” that invokes the get_or_create_tracker method defined in tracker_store.py:

tracker = agent.tracker_store.get_or_create_tracker(sender_id, tracker_filename). That method invokes “create_tracker” method. We will modify them as follows:

def get_or_create_tracker(self, sender_id, tracker_filename=None):
		tracker = self.retrieve(sender_id)
		if tracker is None:
			tracker = self.create_tracker(sender_id, tracker_filename)
		return tracker

def create_tracker(self, sender_id, tracker_filename=None, append_action_listen=True):
		"""Creates a new tracker for the sender_id.
		The tracker is initially listening."""
		tracker = self.init_tracker(sender_id) #initialize our new tracker
		try: 
			f = open(tracker_filename, 'rb')
			retrieved_tracker = pickle.load(f)
			f.close()
			print ('retrieved tracker id is', retrieved_tracker.sender_id)
			events = retrieved_tracker.events  #get the events of the tracker
			for event in events:				#update our new tracker with the stored events
				tracker.update(event)
			print("finished updating tracker")
		except:
			pass
		if tracker:
			if append_action_listen:
				tracker.update(ActionExecuted(ACTION_LISTEN_NAME))
			self.save(tracker)
		return tracker
1 Like

Can we/should we open a PR request and try to incorporate it into Rasa code?

1 Like