thanos
thanos copied to clipboard
thanos-io
s3/aliyunOSS: Support multipart upload without pre known object size.
Discussion for Reference https://github.com/improbable-eng/thanos/pull/617
Acceptance Critiria:
- Minio S3 provider will choose when to use multi part properly even with just
Upload(ctx context.Context, name string, r io.Reader)interface.
The same story is for new provider AliYun. We should either implement multipart on our own or extend objstore interface: https://github.com/thanos-io/thanos/pull/1573/files#r336389276
We have a solid use case for this issue to be fixed: https://github.com/observatorium/thanos-replicate/issues/22
Help wanted!
@bwplotka could move thanos-replicate under thanos-io to increase its exposure and make it look more important for thaons?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi, @bwplotka @adrien-f I see the minio-go has been fixed the bug to upload large file without the size of object. I try to upload a 5.7GB file to my S3 with minio-go client v6.0.49, it works. Maybe we can remove the changes by #617.
For aliyun oss, we can use the resumable upload without known object size. cc @wujinhu
What do you think?
This issue/PR has been automatically marked as stale because it has not had recent activity. Please comment on status otherwise the issue will be closed in a week. Thank you for your contributions.
This issue/PR has been automatically marked as stale because it has not had recent activity. Please comment on status otherwise the issue will be closed in a week. Thank you for your contributions.
Closing for now as promised, let us know if you need this to be reopened! 🤗
Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.
Closing for now as promised, let us know if you need this to be reopened! 🤗
Hi @wujinhu, it seems that this issue hasn't been fixed yet. Do we have any plans for that? Is there some work that I can do for solving it?
We use aliyun OSS in our production environment as Thanos bucket store. But we failed to do downsampling jobs because we always get annoying errors unsupported type of io.Reader.
And I find there is a todo comment referring to this issue in line 72 in the source file oss.go.
Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.
