The reset function in the bottleneck env have bugs

[CYBruce]

Choose an option This is a:

• [x] Question
• [x] Feature request
• [x] Bug report

Describe the issue From my point of view, the reset() function in the bottleneck env keep all other network parameters match the previous except the inflow rate. So the AV_FRAC should not change. But in the reset() function, the AV_FRAC is replaced with 0.1. If users use a different AV_FRAC like 0.3 or 0.7, the rollouts afterwards may use a wrong parameter.

    def reset(self):
        """Reset the environment with a new inflow rate.

        The diverse set of inflows are used to generate a policy that is more
        robust with respect to the inflow rate. The inflow rate is update by
        creating a new scenario similar to the previous one, but with a new
        Inflow object with a rate within the additional environment parameter
        "inflow_range", which is a list consisting of the smallest and largest
        allowable inflow rates.

        **WARNING**: The inflows assume there are vehicles of type
        "followerstopper" and "human" within the VehicleParams object.
        """
        add_params = self.env_params.additional_params
        if add_params.get("reset_inflow"):
            inflow_range = add_params.get("inflow_range")
            flow_rate = np.random.uniform(
                min(inflow_range), max(inflow_range)) * self.scaling

            # We try this for 100 trials in case unexpected errors during
            # instantiation.
            for _ in range(100):
                try:
                    # introduce new inflows within the pre-defined inflow range
                    inflow = InFlows()
                    inflow.add(
                        veh_type="followerstopper",  # FIXME: make generic
                        edge="1",
                        vehs_per_hour=flow_rate * .1,
                        departLane="random",
                        departSpeed=10)
                    inflow.add(
                        veh_type="human",
                        edge="1",
                        vehs_per_hour=flow_rate * .9,
                        departLane="random",
                        departSpeed=10)

                    # all other network parameters should match the previous
                    # environment (we only want to change the inflow)
                    additional_net_params = {
                        "scaling": self.scaling,
                        "speed_limit": self.scenario.net_params.
                        additional_params['speed_limit']
                    }
                    net_params = NetParams(
                        inflows=inflow,
                        no_internal_links=False,
                        additional_params=additional_net_params)

                    vehicles = VehicleParams()
                    vehicles.add(
                        veh_id="human",  # FIXME: make generic
                        car_following_params=SumoCarFollowingParams(
                            speed_mode=9,
                        ),
                        lane_change_controller=(SimLaneChangeController, {}),
                        routing_controller=(ContinuousRouter, {}),
                        lane_change_params=SumoLaneChangeParams(
                            lane_change_mode=0,  # 1621,#0b100000101,
                        ),
                        num_vehicles=1 * self.scaling)
                    vehicles.add(
                        veh_id="followerstopper",
                        acceleration_controller=(RLController, {}),
                        lane_change_controller=(SimLaneChangeController, {}),
                        routing_controller=(ContinuousRouter, {}),
                        car_following_params=SumoCarFollowingParams(
                            speed_mode=9,
                        ),
                        lane_change_params=SumoLaneChangeParams(
                            lane_change_mode=0,
                        ),
                        num_vehicles=1 * self.scaling)

                    # recreate the scenario object
                    self.scenario = self.scenario.__class__(
                        name=self.scenario.orig_name,
                        vehicles=vehicles,
                        net_params=net_params,
                        initial_config=self.scenario.initial_config,
                        traffic_lights=self.scenario.traffic_lights)
                    observation = super().reset()

                    # reset the timer to zero
                    self.time_counter = 0

                    return observation

                except Exception as e:
                    print('error on reset ', e)

        # perform the generic reset function
        observation = super().reset()

        # reset the timer to zero
        self.time_counter = 0

        return observation

To Reproduce path: /flow/envs/bottleneck_env.py

[AboudyKreidieh]

hi @CYBruce thank you for pointing this out. I will get someone to add this feature soon

[CYBruce]

Thank you @AboudyKreidieh . I will be a master student at Tsinghua-Berkeley Shenzhen Institute in September, so we have some connections. Perhaps I will go to Berkeley in 2020 for a master degree. I am willing to do some help with flow project. We can keep in touch. BTW, I am using flow to do some experiments for my graduation thesis and I really appreciate your job.

[AboudyKreidieh]

@CYBruce all great to hear! And thank your kind words and support in finding these features and bugs. If you'd like to get in touch with us, you can send us a personal email, see: https://flow-project.github.io/contact.html