StableCascade
StableCascade copied to clipboard
Add more streamlined local and single_gpu support, allow use of 8 bit adam optimizer
Tested on an Ubuntu a100 40 gb
- Provides a 1B C config
- Fixes a typo
- Allows slurm, gpu, single gpu or cpu
- bitsandbytes support for Adam 8 bit
- adds checks so FSDP isn't used in conjunction with single_gpu
- cleaned up a handful of imports