A2C Architecture for CarRacing-v3 - Confirmation of Network Structure

[yeknafar]

📚 Documentation

Hello,

I am working on the CarRacing-v3 environment using A2C from the Stable-Baselines3 (SB3) library.

I have sketched my understanding of the network architecture used for this task (Actor-Critic with a shared feature extractor).

Could you please confirm if the Attached structure accurately represents the default A2C policy architecture (e.g., CnnPolicy) when applied to the 96 * 96 * 3 image observations of CarRacing-v3?

This is the output of model.policy :

ActorCriticCnnPolicy( (features_extractor): NatureCNN( (cnn): Sequential( (0): Conv2d(3, 32, kernel_size=(8, 8), stride=(4, 4)) (1): ReLU() (2): Conv2d(32, 64, kernel_size=(4, 4), stride=(2, 2)) (3): ReLU() (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1)) (5): ReLU() (6): Flatten(start_dim=1, end_dim=-1) ) (linear): Sequential( (0): Linear(in_features=4096, out_features=512, bias=True) (1): ReLU() ) ) (pi_features_extractor): NatureCNN( (cnn): Sequential( (0): Conv2d(3, 32, kernel_size=(8, 8), stride=(4, 4)) (1): ReLU() (2): Conv2d(32, 64, kernel_size=(4, 4), stride=(2, 2)) (3): ReLU() (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1)) (5): ReLU() (6): Flatten(start_dim=1, end_dim=-1) ) (linear): Sequential( (0): Linear(in_features=4096, out_features=512, bias=True) (1): ReLU() ) ) (vf_features_extractor): NatureCNN( (cnn): Sequential( (0): Conv2d(3, 32, kernel_size=(8, 8), stride=(4, 4)) (1): ReLU() (2): Conv2d(32, 64, kernel_size=(4, 4), stride=(2, 2)) (3): ReLU() (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1)) (5): ReLU() (6): Flatten(start_dim=1, end_dim=-1) ) (linear): Sequential( (0): Linear(in_features=4096, out_features=512, bias=True) (1): ReLU() ) ) (mlp_extractor): MlpExtractor( (policy_net): Sequential() (value_net): Sequential() ) (action_net): Linear(in_features=512, out_features=3, bias=True) (value_net): Linear(in_features=512, out_features=1, bias=True) )

I use basic setting. Best regards.

Checklist

• [x] I have checked that there is no similar issue in the repo
• [x] I have read the documentation

[araffin]

hello,

this looks good indeed =)

See https://stable-baselines3.readthedocs.io/en/master/guide/custom_policy.html