Training : Training schedule (exploration)

[elliotwoods]

Have a schedule for training. This might be best to be in discrete steps, e.g.:

Stage	Description	Explore (%)	Exploit (%)
1	Warm up	100	0
2	Training 1	20	80
3	Training 2	60	40
4	Training 3	40	60
5	Training 5	20	80
6	Test	0	100

When setting a training schedule, you set an amount of time for each step. We can try different training schedules to find a good result

We want to be able to optionally set safety boundaries during training. Perhaps with some kind of lookahead (e.g. if position + velocity * N time steps > boundary: act safely)

[elliotwoods]

We're going to want to put this exploration into the server side definition of the network

[elliotwoods]

We have a first pass at this with the runtime_parameters model and

	def update_runtime_parameters(self):
		if self.replay_memory.buffer_counter < 10000:
			self.runtime_parameters.is_training = True
			self.runtime_parameters.noise_amplitude = 1 / 2
		elif self.replay_memory.buffer_counter < 20000:
			self.runtime_parameters.is_training = True
			self.runtime_parameters.noise_amplitude = 1 / 4
		elif self.replay_memory.buffer_counter < 30000:
			self.runtime_parameters.is_training = True
			self.runtime_parameters.noise_amplitude = 1 / 8
		elif self.replay_memory.buffer_counter < 40000:
			self.runtime_parameters.is_training = True
			self.runtime_parameters.noise_amplitude = 1 / 16
		else:
			self.runtime_parameters.is_training = True
			self.runtime_parameters.noise_amplitude = 0

[elliotwoods]

Potential improvements:

• Realtime adjust / display parameters (some sort of portal for managing training
• Presets in JSON format (with filters)