InternLM
InternLM copied to clipboard
InternLM
Official release of InternLM2.5 base and chat models. 1M context support
Hi InternLM team, thank you for this open source contribution! InternLM looks like a really strong 7B model. I think the research community would greatly benefit from learning about the...
## Motivation We have `checkpoint_fraction` but have no interface in config file, this PR will support this. ## Modification 1. add `checkpoint_fraction` option into model config. 2. add `checkpoint_fraction` sanity...
感谢开源！ 我想做一组Retrieval QA的测试，请问用什么prompt比较合适，我试了几个prompt都不太好，比如下面这个internLM-7b-chat会用英文来回答，而直接使用chat接口回答都很简短。 ``` prompt_template = f"""User: 已知信息： {context} 根据上述已知信息，专业的回答用户的问题。如果无法从中得到答案，请说 “抱歉，根据已知信息无法回答该问题”，不允许在答案中添加编造成分，答案请使用中文。 问题：{question} Assistant:""" ```
1. Do we need documentation in the readthedocs of this repository? 2. The contributing guideline such as https://github.com/InternLM/opencompass/blob/main/docs/en/notes/contribution_guide.md is missing, community may have issue in contributing to this project. 3....
I run the code, but only got 90+ tflops. INFO train.py:317 in record_current_batch_training_metrics -- tflops=93.48098385143103,step=9,loss=7.502509117126465,tgs (tokens/gpu/second)=2104.89,lr=2.2e-06,loss_scale=65536.0,grad_norm=20.60409540743281,micro_num=4,num_consumed_tokens=2621440,inf_nan_skip_batches=0,num_samples_in_batch=13,largest_length=2048,largest_batch=4,smallest_batch=3,adam_beta2=0.95,fwd_bwd_time=6.15
>>> model = AutoModelForCausalLM.from_pretrained("/home/hope/work/models/internlm-chat-7b-8k", trust_remote_code=True).to(torch.bfloat16).cuda() Loading checkpoint shards: 100%|███████████████████████████████| 2/2 [00:11>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("/home/hope/work/models/internlm-chat-7b-8k", trust_remote_code=True) >>> model = model.eval()Traceback (most recent call last): File "", line 1,...
Hi, I tried to install per requirements/torch.txt and got following errors. Anything did I miss? ``` ➜ InternLM git:(main) pip install -r requirements/torch.txt Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu117 ERROR: Could...
It would be extremely helpful if you could share the list of languages that are included in the dataset used to train the model. If the dataset is available on...
