Daniel
可以贡献下tts 用的是哪 个模型？可以转onnx或ncnn？
yes, you do.
> In the paper, it was mentioned that 48G of graphics memory can train 65B of LLaMA > > ``` > We present QLORA, an efficient finetuning approach that reduces...
Is there an example for training on multiple-GPUs?
ddp_find_unused_parameters False
load another model: ValueError: Target modules ['dense_h_to_4h', 'dense', 'dense_4h_to_h', 'query_key_value'] not found in the base model. Please check the target modules and try again. python qlora.py –learning_rate 0.0001 --model_name_or_path huggyllama/llama-7b
what's wrong with my setting?
Yes. I tested it with 8* A100, it occupates 12G on each GPU and still not fast, especially alter the input length more than 500.
thanks:)
> @internlm-team You told us number of parameters at least :) We present InternLM, a multilingual foundational language model with 104B parameters. InternLM is pre-trained on a large corpora with...