DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

[BUG] `AttributeError: 'CSVConfig' object has no attribute 'group'`

Open saforem2 opened this issue 2 years ago • 0 comments

Hello,

When trying to enable W&B monitoring (as shown below in the snippet from my ds_config.json):

"wandb": {
    "enabled": True,
    "project": projectName,
    "group": groupName
},

I get the following error:

  ...
  File "/lus/grand/projects/datascience/foremans/locations/polaris/projects/l2hmc-qcd/src/l2hmc/trainers/pytorch/trainer.py", line 233, in _setup_deepspeed
    engine, optimizer, _, _ = deepspeed.initialize(
  File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/__init__.py", line 125, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 287, in __init__
    self.monitor = MonitorMaster(self._config.monitor_config)
  File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/monitor/monitor.py", line 36, in __init__
    self.wandb_monitor = WandbMonitor(monitor_config.csv_monitor)
  File "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/deepspeed/monitor/wandb.py", line 14, in __init__
    self.group = wandb_config.group
AttributeError: 'CSVConfig' object has no attribute 'group'

I believe the issue is coming from the line:

https://github.com/microsoft/DeepSpeed/blob/8be8c012c8a247e5512276676b5e7092d88633eb/deepspeed/monitor/monitor.py#L36

which is (incorrectly) trying to use monitor_config.csv_monitor to instantiate the self.wandb_monitor object.

saforem2 avatar Feb 18 '23 19:02 saforem2