Transition is NoneType in PER when using Rainbow

[hughesmc]

I'm trying to use Coach for the first time, with a custom made environment in Gym. I have a simple preset:

from rl_coach.agents.rainbow_dqn_agent import RainbowDQNAgentParameters
from rl_coach.graph_managers.basic_rl_graph_manager import BasicRLGraphManager
from rl_coach.environments.gym_environment import GymVectorEnvironment
from rl_coach.graph_managers.graph_manager import SimpleSchedule
from rl_coach.core_types import EnvironmentSteps

agent_params = RainbowDQNAgentParameters()

env_params = GymVectorEnvironment(level="rl_coach.environments.custom_environments.BandEnvironment_env:BandEnv")

schedule_params=SimpleSchedule()

schedule_params.heatup_steps=EnvironmentSteps(1000)

graph_manager = BasicRLGraphManager(agent_params=agent_params, env_params=env_params, schedule_params=schedule_params)

It seems to me as though things work okay, until I get the following error message:

Traceback (most recent call last):
  File "/home/hughes/anaconda3/envs/coachRL/bin/coach", line 10, in <module>
sys.exit(main())
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/coach.py", line 775, in main
launcher.launch()
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/coach.py", line 224, in launch
self.run_graph_manager(graph_manager, args)
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/coach.py", line 610, in run_graph_manager
self.start_single_threaded(task_parameters, graph_manager, args)
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/coach.py", line 672, in start_single_threaded
start_graph(graph_manager=graph_manager, task_parameters=task_parameters)
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/coach.py", line 87, in start_graph
graph_manager.improve()
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/graph_managers/graph_manager.py", line 549, in improve
self.train_and_act(self.steps_between_evaluation_periods)
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/graph_managers/graph_manager.py", line 484, in train_and_act
self.train()
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/graph_managers/graph_manager.py", line 410, in train
[manager.train() for manager in self.level_managers]
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/graph_managers/graph_manager.py", line 410, in <listcomp>
[manager.train() for manager in self.level_managers]
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/level_manager.py", line 187, in train
[agent.train() for agent in self.agents.values()]
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/level_manager.py", line 187, in <listcomp>
[agent.train() for agent in self.agents.values()]
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/agents/agent.py", line 724, in train
range(self.ap.algorithm.num_consecutive_training_steps)]
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/agents/agent.py", line 723, in <listcomp>
self.ap.is_batch_rl_training else [self.call_memory('sample', batch_size) for _ in
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/agents/agent.py", line 463, in call_memory
result = getattr(self.memory, func)(*args)
  File "/home/hughes/anaconda3/envs/coachRL/lib/python3.5/site-packages/rl_coach/memories/non_episodic/prioritized_experience_replay.py", line 250, in sample
transition.info['idx'] = leaf_idx
AttributeError: 'NoneType' object has no attribute 'info'

I'm really not sure what's going on. It seems like it's trying to sample from the replay memory and getting a NoneType object returned? Any help explaining what's going on, and how to avoid this would be appreciated.