whcao

[Bug]Fix tracer bugs and make groups bugs

1

1. New backward ops are added in Pytorch 1.10.2 and 1.12. 2. Fix register_same_mutable

Following pr#8461, pr#8480 and this [commit](https://github.com/open-mmlab/mmdetection/commit/727d867119f30434921ae581586c070dd737ad71) in mmdet, 1. '--amp' option 2. auto-scale lr 3. auto-resume are supported in this pr.

Integrate pytorch poc python api

1

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

[Feature] Distillation enhancement

1

## Modification 1. Support distillation early stop. 2. Add norm connector and support the config of connectors as a list. 3. Add fgd loss. 4. Fix bugs in compute_distill_losses. If...

## Modification 1. Support Cos / Linear / MultiStep loss weight scheduler. Notes: Linear one can be used for distillation loss weight warmup and MultiStep can be used for early...

Encountering FileNotFoundError while Compiling Triton Kernel in Distributed Training

5

During the process of distributed training, I encountered the following problem when compiling Triton kernels: ``` Traceback (most recent call last): ...... File "/mnt/petrelfs/caoweihan/anaconda3/envs/deepspeed/lib/python3.10/site-packages/triton/compiler/compiler.py", line 482, in compile metadata_group[ir_filename] =...

Requirements: 1. transformers >= 4.39.0 2. peft >= 0.10.0