LMOps
LMOps copied to clipboard
microsoft
General technology for enabling AI capabilities w/ LLMs and MLLMs
我如果想在dolly数据集上进行teacher：llama3-70b-instruct，student：llama3-8b-instruct。的蒸馏，是否需要修改某些代码逻辑或者dolly数据集的template？
Bumps [zipp](https://github.com/jaraco/zipp) from 3.7.0 to 3.19.1. Changelog Sourced from zipp's changelog. v3.19.1 Bugfixes Improved handling of malformed zip files. (#119) v3.19.0 Features Implement is_symlink. (#117) v3.18.2 No significant changes. v3.18.1...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
我在4张A100上使用4卡模型并行训练，student是llama3-8b，teacher是llama3-70b，使用ds_config_zero2_offload运行成功时4张A100的GPU占用为47g/80g，在训练过程中会出现CUDA out of memory，请问如何解决这一问题 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2023.7.22 to 2024.7.4. Commits bd81538 2024.07.04 (#295) 06a2cbf Bump peter-evans/create-pull-request from 6.0.5 to 6.1.0 (#294) 13bba02 Bump actions/checkout from 4.1.6 to 4.1.7 (#293) e8abcd0 Bump pypa/gh-action-pypi-publish from...
Bumps [transformers](https://github.com/huggingface/transformers) from 4.26.1 to 4.38.0. Release notes Sourced from transformers's releases. v4.38: Gemma, Depth Anything, Stable LM; Static Cache, HF Quantizer, AQLM New model additions 💎 Gemma 💎 Gemma...
Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.48.2 to 4.66.3. Release notes Sourced from tqdm's releases. tqdm v4.66.3 stable cli: eval safety (fixes CVE-2024-34062, GHSA-g7vv-2v7x-gj9p) tqdm v4.66.2 stable pandas: add DataFrame.progress_map (#1549) notebook: fix...
Bumps [idna](https://github.com/kjd/idna) from 2.8 to 3.7. Release notes Sourced from idna's releases. v3.7 What's Changed Fix issue where specially crafted inputs to encode() could take exceptionally long amount of time...
Don't you have a bug in the logit processor? It seems you don't even use the processed scores in the wrapper. https://github.com/microsoft/LMOps/blob/96e7d9d086cba6cd878c98ac810902eae1323d8b/minillm/transformers/src/transformers/generation/utils.py#L2942
From the ReadMe inside MiniLLM both of the following commands return a 404: `wget -O data.tar https://unilm.blob.core.windows.net/minillm/MiniLLM/data.tar` `wget -O processed_data.tar https://unilm.blob.core.windows.net/minillm/MiniLLM/processed_data.tar` Response: `https://unilm.blob.core.windows.net/minillm/MiniLLM/data.tar Resolving unilm.blob.core.windows.net (unilm.blob.core.windows.net)... 20.60.153.132 Connecting to unilm.blob.core.windows.net...
Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.32 to 3.1.41. Release notes Sourced from gitpython's releases. 3.1.41 - fix Windows security issue The details about the Windows security issue can be found in this...