Question about the shape of map_size.

[WangzcBruce]

The original dimension [8, 200, 200] could lead to errors. Should it be adjusted to [26, 200, 200]? Reference link: https://github.com/cure-lab/MagicDrive/blob/d2fad7dd5056110afed07da821325eab84f967e8/configs/model/SDv1.5mv_rawbox.yaml#L31

python-BaseException Error executing job with overrides: ['++runner.enable_xformers_memory_efficient_attention=false'] Traceback (most recent call last): File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 222, in run_and_report raise ex File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 219, in run_and_report return func() File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 132, in run _ = ret.return_value File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) File "/workspace/[email protected]/MagicDrive/tools/train.py", line 110, in main runner.run() File "/workspace/[email protected]/MagicDrive/magicdrive/runner/base_runner.py", line 343, in run loss = self._train_one_stop(batch) File "/workspace/[email protected]/MagicDrive/magicdrive/runner/multiview_runner.py", line 300, in _train_one_stop model_pred = self.controlnet_unet( File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/accelerate/utils/operations.py", line 553, in forward return model_forward(*args, **kwargs) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/accelerate/utils/operations.py", line 541, in call return convert_to_fp32(self.model_forward(*args, **kwargs)) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/autocast_mode.py", line 198, in decorate_autocast return func(*args, **kwargs) File "/workspace/[email protected]/MagicDrive/magicdrive/runner/multiview_runner.py", line 48, in forward encoder_hidden_states_with_cam = self.controlnet( File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/workspace/[email protected]/MagicDrive/magicdrive/networks/unet_addon_rawbox.py", line 848, in forward controlnet_cond = self.controlnet_cond_embedding(controlnet_cond) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/workspace/[email protected]/MagicDrive/magicdrive/networks/map_embedder.py", line 67, in forward embedding = self.conv_in(conditioning) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward return self._conv_forward(input, self.weight, self.bias) File "/root/miniconda3/envs/MagicDrive/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Given groups=1, weight of size [16, 8, 3, 3], expected input[18, 26, 200, 200] to have 8 channels, but got 26 channels instead Steps: 0%| | 0/937700 [00:42<?, ?it/s]

[flymin]

No, it should be 8 in the latest version. We use raw_box for objects, so there is no object-related layers in the map.

[WangzcBruce]

Therefore, it is necessary to execute the command python tools/prepare_map_aux.py +process=train rather than utilizing the existing HD maps in order to match the hyperparameters-8 in the latest version?

[github-actions[bot]]

This issue is stale because it has been open for 7 days with no activity. If you do not have any follow-ups, the issue will be closed soon.