DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Pydantic v2 migration

Open mrwyattii opened this issue 1 year ago • 4 comments

Pydantic v2 has been out for some time now. We have been relying on using the v1 API available in v2 until now. This is a refresh of #3902 to bring proper v2 support to DeepSpeed.

Corresponding DeepSpeed-MII PR here.

@loadams

mrwyattii avatar Feb 21 '24 00:02 mrwyattii

Do you want to do what we did on the previous pydantic update of updating the nv-mii test to use your branch rather than main so that it passes here?

loadams avatar Feb 27 '24 16:02 loadams

CI with microsoft/DeepSpeed-MII#423 for nv-a6000 and nv-mii

mrwyattii avatar Mar 02 '24 00:03 mrwyattii

CI with microsoft/DeepSpeed-MII#423 for nv-a6000 and nv-mii

CI with changes updated on both DS and DS-MII for nv-a6000 and nv-mii

loadams avatar Apr 05 '24 18:04 loadams

Using Albumentation, the last version has a strict dependancy regarding pydantic >2.0 what makes it incompatible with deepspeed.

https://github.com/albumentations-team/albumentations/releases/tag/1.4.4

https://github.com/albumentations-team/albumentations/blob/a22a319b2e7079389326e5b7830e71017701b1b9/setup.py#L11

It makes this PR even more releavant.

FrsECM avatar Apr 24 '24 11:04 FrsECM

CI triggered using updates in MII

nv-a6000 nv-mii

Both tests now passing.

loadams avatar Jul 01 '24 21:07 loadams

Tested in #6382.

loadams avatar Aug 20 '24 17:08 loadams

Tested here

https://github.com/microsoft/DeepSpeed/actions/runs/10512334326 https://github.com/microsoft/DeepSpeed/actions/runs/10512339060

loadams avatar Aug 22 '24 17:08 loadams