Hai Xuan Pham

Has anyone tried compiling cntk with CUDA 10.1 and 10.2? and python 3.7+?

6

I'm not using CNTK much these days, beside old projects. I still like it nonetheless, for its elegant APIs. Wish to use it with newer CUDA version (for new GPU...

[BUG] Zero2/3 segmentation fault with CPU optimizer off-loading

2

**Describe the bug** Deepspeed got segfault when loading CPU_ADAM, both with zero-2 and zero-3 configs / Huggingface transformers integration. **Zero Configuations** - Zero-2 ``` { "fp16": { "enabled": "auto", "loss_scale":...

AMP autocast not invoked with CUDA 11.8 build of Pytorch

1

### System Info pytorch 2.1 + CUDA 11.8 transformers 4.36.2 accelerate 0.26.0 ### Who can help? @pacman100 , @muellerz ### Information - [ ] The official example scripts - [...

With deepspeed zero3 enabled, loading from_pretrained() and resize_token_embeddings() do not work correctly

2

### System Info torch 2.1.1 - CUDA 12.1 transformers 4.36.2 accelerate 0.26.0 deepspeed 0.12.3 ### Who can help? @pacman100 ### Information - [ ] The official example scripts - [X]...