DeepSpeed
DeepSpeed copied to clipboard
[BUG] `AttributeError: 'CSVConfig' object has no attribute 'group'`
Hello,
When trying to enable W&B monitoring (as shown below in the snippet from my ds_config.json):
"wandb": {
"enabled": True,
"project": projectName,
"group": groupName
},
I get the following error:
...
File "/lus/grand/projects/datascience/foremans/locations/polaris/projects/l2hmc-qcd/src/l2hmc/trainers/pytorch/trainer.py", line 233, in _setup_deepspeed
engine, optimizer, _, _ = deepspeed.initialize(
File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/__init__.py", line 125, in initialize
engine = DeepSpeedEngine(args=args,
File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 287, in __init__
self.monitor = MonitorMaster(self._config.monitor_config)
File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/monitor/monitor.py", line 36, in __init__
self.wandb_monitor = WandbMonitor(monitor_config.csv_monitor)
File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/monitor/wandb.py", line 14, in __init__
self.group = wandb_config.group
AttributeError: 'CSVConfig' object has no attribute 'group'
I believe the issue is coming from the line:
https://github.com/microsoft/DeepSpeed/blob/8be8c012c8a247e5512276676b5e7092d88633eb/deepspeed/monitor/monitor.py#L36
which is (incorrectly) trying to use monitor_config.csv_monitor to instantiate the self.wandb_monitor object.