leo5856

hi,since during training phase the input face A is a heavily blurred face，Is B the exact same picture as original A or just a face belonging to the same identity...

Reproduction Failure : 8*A100 40G run opt-13b stage3_RLHF OOM

2

- I use offload、gradient_checkpointing and zero_stage 3, and still get OOM result - I test it in 8*A100 80G, and see about 55G GPU memory consumption via "nvidia-smi" - my...

[BUG] hybrid_engine for zero 3 seems invalid

2

``` from transformers import AutoTokenizer, AutoModel, AutoModelForCausalLM, AutoConfig, get_scheduler import deepspeed model = AutoModelForCausalLM.from_pretrained("models/opt-6.7b") tokenizer = AutoTokenizer.from_pretrained("models/opt-6.7b", fast_tokenizer=True) tokenizer.padding_side = 'left' ds_config ={ 'train_micro_batch_size_per_gpu': 4, 'steps_per_print': 10, 'zero_optimization': {'stage': 3,...