OpenCSGs
There is no problem in uploading a single model file of about 4G, but a single file larger than 4G will directly report an error.
Describe the bug
After using helmchart to deploy in the k8s cluster, git lfs cannot upload a single model file larger than 4G through http or ssh.
Environment CSGHub Version: v1.8.0｜1.9.0 OS: Linux Hardware: 512c1024G Launch: helm chart
To Reproduce Steps to reproduce the behavior:
- git clone xxx
- git lfs install
- git track "*.safesenter"
- git lfs push
Expected behavior Successfully uploaded
Screenshots
use ssh:
DeepSeek-R1-0528# GIT_TRACE=1 git lfs push --object-id origin6 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:24:17.426904 git.c:749 trace: exec: git-lfs push --object-id origin6 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:24:17.426984 run-command.c:659 trace: run_command: git-lfs push --object-id origin6 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:24:17.458136 trace git-lfs: exec: git 'version'
03:24:17.461974 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
03:24:17.466157 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
03:24:17.470500 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
03:24:17.474255 trace git-lfs: exec: git 'config' '--includes' '-l'
03:24:17.477785 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
03:24:17.481511 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
03:24:17.485804 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
03:24:17.490411 trace git-lfs: attempting pure SSH protocol connection
03:24:17.490424 trace git-lfs: spawning pure SSH connection
03:24:17.490550 trace git-lfs: run_command: ssh -oControlMaster=auto -oControlPath=/tmp/sock-291537913/sock-%C -p 30022 [email protected] git-lfs-transfer /models/root/DeepSeek-R1-0528.git upload
03:24:17.490672 trace git-lfs: exec: ssh '-oControlMaster=auto' '-oControlPath=/tmp/sock-291537913/sock-%C' '-p' '30022' '[email protected]' 'git-lfs-transfer /models/root/DeepSeek-R1-0528.git upload'
03:24:17.585427 trace git-lfs: pure SSH connection successful
03:24:17.585956 trace git-lfs: tq: running as batched queue, batch size of 100
03:24:17.586514 trace git-lfs: tq: sending batch of size 1
03:24:17.586533 trace git-lfs: api: batch 1 files
03:24:24.639786 trace git-lfs: shutting down pure SSH connection
03:24:24.639828 trace git-lfs: terminating pure SSH connection (1 -> 0)
LFS: batch response: status 500 from server (internal error)
use http:
GIT_TRACE=1 git lfs push --object-id origin5 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:26:01.665466 git.c:749 trace: exec: git-lfs push --object-id origin5 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:26:01.665533 run-command.c:659 trace: run_command: git-lfs push --object-id origin5 042be47233c9077ce4bc8dd42c124d17bb74bb2c1c368343e9db8292ef277c1e
03:26:01.693419 trace git-lfs: exec: git 'version'
03:26:01.697326 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'remote'
03:26:01.701481 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
03:26:01.705899 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' '--git-dir' '--show-toplevel'
03:26:01.710198 trace git-lfs: exec: git 'config' '--includes' '-l'
03:26:01.713950 trace git-lfs: exec: git 'rev-parse' '--is-bare-repository'
03:26:01.717074 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' ':.lfsconfig'
03:26:01.720669 trace git-lfs: exec: git 'config' '--includes' '-l' '--blob' 'HEAD:.lfsconfig'
03:26:01.725294 trace git-lfs: tq: running as batched queue, batch size of 100
03:26:01.725451 trace git-lfs: tq: sending batch of size 1
03:26:01.726024 trace git-lfs: api: batch 1 files
03:26:01.726840 trace git-lfs: HTTP: POST http://root:[email protected]:30080/models/root/DeepSeek-R1-0528.git/info/lfs/objects/batch
03:26:01.740099 trace git-lfs: HTTP: 500
03:26:01.740145 trace git-lfs: HTTP: <!DOCTYPE html>
<html>
<head>
<title>500 Server Error | CSGHub</title>
<style>
body {
height: 100vh;
display: flex;
align-items: center;
justify-content: center;
margin: 0;
background-color: #f9fafb;
}
.content {
display: flex;
align-items: center;
justify-content: center;
flex-direction: column;
gap: 32px;
max-width: 800px;
padding: 20px;
}
.content img {
width: 242px;
height: auto;
}
.
03:26:01.740332 trace git-lfs: api error: Fatal error: Unable to parse HTTP response for POST http://root:[email protected]:30080/models/root/DeepSeek-R1-0528.git/info/lfs/objects/batch: invalid character '<' looking for beginning of value
batch response: Fatal error: Unable to parse HTTP response for POST http://root:[email protected]:30080/models/root/DeepSeek-R1-0528.git/info/lfs/objects/batch: invalid character '<' looking for beginning of value
Uploading LFS objects: 0% (0/1), 0 B | 0 B/s, done.
Errors logged to '/opt/data/cgs/DeepSeek-R1-0528/.git/lfs/logs/20250724T032601.740432899.log'.
Use `git lfs logs last` to view the log.
Additional context
About 4G can be uploaded successfully.
@Bao0ne may I know which version you are using? We have fixed issue in v1.8
@samchen8008 I tried v1.8.0-ee, v1.8.0-ce and v1.9.0-ee, and they all encountered the same problem.helmchart need to make additional configuration?
@Bao0ne This is because OSS pre-signed URLs only support files up to 5GB in size. We've improved the error message in the latest version. If you need to upload files larger than 5GB, please consider using our SDK instead https://github.com/OpenCSGs/csghub-sdk.
I tried to use sdk to upload, and observed the upload traffic, but returned a 500 error.：
python3 upload.py
fail to upload model-00163-of-000163.safetensors with response code: 500, error: <!DOCTYPE html>
<html>
<head>
<title>500 Server Error | CSGHub</title>
<style>
body {
height: 100vh;
display: flex;
align-items: center;
justify-content: center;
margin: 0;
background-color: #f9fafb;
}
.content {
display: flex;
align-items: center;
justify-content: center;
flex-direction: column;
gap: 32px;
max-width: 800px;
padding: 20px;
}
.content img {
width: 242px;
height: auto;
}
.text-content {
text-align: center;
}
.text-content h1 {
color: #101828;
font-family: "PingFang SC", system-ui, -apple-system, sans-serif;
font-size: 60px;
font-weight: 500;
line-height: 72px;
letter-spacing: -1.2px;
margin: 0 0 16px 0;
}
.text-content p {
color: #475467;
font-family: "PingFang SC", system-ui, -apple-system, sans-serif;
font-size: 20px;
font-weight: 400;
line-height: 30px;
margin: 0 0 32px 0;
}
.action-button {
display: inline-block;
padding: 12px 24px;
background-color: #1e50e2;
color: white;
text-decoration: none;
border-radius: 8px;
font-family: "PingFang SC", system-ui, -apple-system, sans-serif;
font-size: 16px;
font-weight: 500;
transition: background-color 0.3s;
}
.action-button:hover {
background-color: #1a44b8;
}
.error-details {
background-color: #f8f9fc;
padding: 12px;
border-radius: 6px;
margin-top: 20px;
font-family: monospace;
color: #667085;
font-size: 14px;
}
</style>
</head>
<body>
<div class="content">
<img src="https://opencsg-public-resource.oss-cn-beijing.aliyuncs.com/0916d174-15a8-44f3-90a0-35882a4554bc" alt="Server error">
<div class="text-content">
<h1>500 Internal Server Error</h1>
<p>Something went wrong on our servers while processing your request.</p>
<div class="error-details">
Error 500: Internal Server Error<br>
</div>
</div>
</div>
<script>
// Generate a simple request ID for tracking
document.getElementById('request-id').textContent =
Math.random().toString(36).substring(2, 10).toUpperCase();
</script>
</body>
</html>
Code：
from pycsghub.file_upload import http_upload_file
token = "xxx"
endpoint = "http://xxx:30080"
repo_type = "model"
repo_id = 'root/DeepSeek-R1-0528'
result = http_upload_file(repo_id, endpoint=endpoint, token=token, repo_type='model', file_path='model-00163-of-000163.safetensors')
@Bao0ne Suggest just upload small files by http method, and upload file which size is less than 4G by http upoad method due to some network limit.
Use following command upload large folder to your local csghub.
csghub-cli upload-large-folder <repo-name> <local path> -e <local csghub server address> -k <your token>
很抱歉，根据我检索到的内容，我无法回答您关于该项目社区运营、代码原创性以及Issue处理效率的问题。我掌握的信息仅限于StarShip产品的版本更新说明，不包含社区管理、项目开发历史或用户支持方面的内容。
如果您需要了解更多关于StarShip产品的技术功能和使用方法，可以访问 https://opencsg.com/docs，技术支持团队将会为您提供帮助。
根据检索到的内容，无法回答您关于该项目维护状态或具体行为的问题。
您可以访问 https://opencsg.com/docs 获取更多信息，技术支持将会介入处理。