rustfs
The file content has been modified
English
- Obtaining an upload link using PresignedPutObjectRequest.
- Uploading the test.txt file using the API request tool.
- The file's contents were modified after downloading it through the web interface. Adding a new line to the beginning of the file.
----------------------------055896199584227059978138 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain
This Content
Chinese
- 通过PresignedPutObjectRequest获取上传链接
- 通过api请求工具上传文件test.txt
- 通过web界面下载文件后内容被修改文件的开头被添加了
----------------------------055896199584227059978138 Content-Disposition: form-data; name="file"; filename="test.txt" Content-Type: text/plain
This Content
@Nugine In rustfs, PutObjectInput.body is used to hold the file content. When handling deserialize_http_multipart, should Multipart.File be assigned to PutObjectInput.body as the body value?
@Slf4j
@Service
public class S3FileService {
private final S3Properties s3Properties = SpringUtil.getBean(S3Properties.class);
private final S3Presigner presignerInner;
private final S3Presigner presignerPublic;
private final S3Client s3Client;
public S3FileService() {
presignerInner = S3Presigner.builder()
.endpointOverride(URI.create(s3Properties.getEndpoint()))
.region(Region.US_EAST_1)
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(s3Properties.getAccessKey(), s3Properties.getSecretKey())
)
)
.serviceConfiguration(
S3Configuration.builder()
.pathStyleAccessEnabled( true)
.build()
)
.build();
presignerPublic = S3Presigner.builder()
.endpointOverride(URI.create(s3Properties.getPublicBasisEndpoint()))
.region(Region.US_EAST_1)
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(s3Properties.getAccessKey(), s3Properties.getSecretKey())
)
)
.serviceConfiguration(
S3Configuration.builder()
.pathStyleAccessEnabled( true)
.build()
)
.build();
s3Client = S3Client.builder()
.region(Region.US_EAST_1)
.credentialsProvider(
StaticCredentialsProvider.create(
AwsBasicCredentials.create(s3Properties.getAccessKey(), s3Properties.getSecretKey())
)
)
.forcePathStyle(true)
.build();
}
/*
* 获取外部直传链接
* */
public String getPublicUploadUrl(String key) {
PutObjectRequest putRequest = PutObjectRequest.builder()
.bucket(s3Properties.getBucketName())
.key(key)
.build();
PresignedPutObjectRequest presignedPut = presignerPublic.presignPutObject(
PutObjectPresignRequest.builder()
.putObjectRequest(putRequest)
.signatureDuration(Duration.ofMinutes(30))
.build()
);
return presignedPut.url().toString();
}
/*
* 生成外部下载链接
* */
public String getPublicDownloadUrl(String key,Integer expiresInMinutes) {
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(s3Properties.getBucketName())
.key(key)
.build();
GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder()
.getObjectRequest(getObjectRequest)
.signatureDuration(Duration.ofMinutes(expiresInMinutes))
.build();
PresignedGetObjectRequest presignedRequest = presignerPublic.presignGetObject(presignRequest);
return presignedRequest.url().toString();
}
/*
* 生成内部下载地址
* */
public String getInnerDownloadUrl(String key,Integer expiresInMinutes) {
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(s3Properties.getBucketName())
.key(key)
.build();
GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder()
.getObjectRequest(getObjectRequest)
.signatureDuration(Duration.ofMinutes(expiresInMinutes))
.build();
PresignedGetObjectRequest presignedRequest = presignerInner.presignGetObject(presignRequest);
return presignedRequest.url().toString();
}
/*
* 删除对象
* */
public void deleteObject(String key) {
DeleteObjectResponse deleteObjectResponse = s3Client.deleteObject(DeleteObjectRequest.builder().bucket(s3Properties.getBucketName()).key(key).build());
}
/*
* 获取文件下载地址
* */
public String getStaticDownloadUrl(String key) {
return s3Properties.getPublicBasisEndpoint() + "/" + s3Properties.getBucketName() + "/" + key;
}
}
@Test
void s3() {
String url = s3FileService.getPublicUploadUrl("test.txt");
log.info(url);
}
@XiaoZhengRS you're using a presignedPut URL—are you uploading with a PUT request and sending the file as raw binary? If you want to use form-data instead, you need to switch to a presignedPost URL.
here is an example of using a PUT URL
<input type="file" id="u">
<script>
document.getElementById('u').onchange = async e => {
const file = e.target.files[0];
const url = 'https://bucket.s3.amazonaws.com/imgs/flower.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...'; // presignedPut URL
await fetch(url, {
method: 'PUT',
headers: {'Content-Type': file.type},
body: file
});
alert('done');
};
</script>
It seems that s3s does not yet support signature verification for presigned PUT URLs. For now, you can use the POST-URL approach; however, the official AWS Java SDK (v1/v2) still does not include a built-in generatePresignedPost method. You’ll need to rely on community-wrapped third-party libraries. The solutions are given below.
<dependency>
<groupId>io.github.dyegosutil</groupId>
<artifactId>aws-s3-presigned-post</artifactId>
<version>1.0.1</version>
</dependency>
import io.github.dyegosutil.s3presignedpost.S3PostSigner;
import io.github.dyegosutil.s3presignedpost.PostParams;
import software.amazon.awssdk.regions.Region;
import java.time.ZonedDateTime;
import java.time.temporal.ChronoUnit;
public class Demo {
public static void main(String[] args) {
PostParams params = PostParams.builder(
Region.US_EAST_1,
ZonedDateTime.now().plus(1, ChronoUnit.HOURS),
"my-bucket",
PostParams.withKey("uploads/${filename}"))
.withContentLengthRange(1, 10 * 1024 * 1024)
.withContentType("image/jpeg")
.build();
var post = S3PostSigner.sign(params);
System.out.println("URL : " + post.getUrl());
System.out.println("Fields: " + post.getFields());
}
}
我尝试使用问题中的流程进行验证，但是没有发现 Mutlipart Body 被写入文件的问题，是否是我的方向错误了？
I tried to verify using the process described in the question, but I didn't find any issue with Mutlipart Body being written to the file. Am I going in the wrong direction?
package commands
import (
"bytes"
"context"
"fmt"
"io"
"log"
"net/http"
"s3-client-demo/s3client"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/spf13/cobra"
)
func PresignedPutCommand(client *s3client.Client, bucketName string) *cobra.Command {
return &cobra.Command{
Use: "presigned-put",
Run: func(cmd *cobra.Command, args []string) {
if len(args) < 1 {
log.Fatalf("请提供对象名称")
}
objectKey := args[0]
ctx := context.Background()
// 1、生成预签名URL
presignedURL, err := client.GetPresignClient().PresignPutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
})
if err != nil {
log.Fatalf("生成预签名URL失败: %v", err)
}
// 2、通过http请求上传文件
body := bytes.NewReader([]byte("Hello, World!"))
request, err := http.NewRequest("PUT", presignedURL.URL, body)
if err != nil {
log.Fatalf("创建请求失败: %v", err)
}
request.Header.Set("Content-Type", "application/octet-stream")
httpClient := &http.Client{}
response, err := httpClient.Do(request)
if err != nil {
log.Fatalf("上传文件失败: %v", err)
}
defer response.Body.Close()
bodyBytes, err := io.ReadAll(response.Body)
if err != nil {
log.Fatalf("读取响应体失败: %v", err)
}
fmt.Println(string(bodyBytes))
// 3、生成预签名URL
presignedURL, err = client.GetPresignClient().PresignGetObject(ctx, &s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(objectKey),
})
if err != nil {
log.Fatalf("生成预签名URL失败: %v", err)
}
fmt.Println(presignedURL.URL)
// 4、通过http请求下载文件
request, _ = http.NewRequest("GET", presignedURL.URL, nil)
response, err = httpClient.Do(request)
if err != nil {
log.Fatalf("下载文件失败: %v", err)
}
defer response.Body.Close()
bodyBytes, err = io.ReadAll(response.Body)
if err != nil {
log.Fatalf("读取响应体失败: %v", err)
}
fmt.Println(string(bodyBytes))
},
}
}
output
http://my_local_host:9010/test001/empty_file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Checksum-Mode=ENABLED&X-Amz-Credential=rustfsadmin%2F20251114%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251114T142605Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=b890577bb3d128464852fe8386c274c79c7fb6ebaea1942fcd5386087162794c
Hello, World!
@XiaoZhengRS
我尝试使用问题中的流程进行验证，但是没有发现
Mutlipart Body被写入文件的问题，是否是我的方向错误了？I tried to verify using the process described in the question, but I didn't find any issue with
Mutlipart Bodybeing written to the file. Am I going in the wrong direction?package commands
import ( "bytes" "context" "fmt" "io" "log" "net/http" "s3-client-demo/s3client"
"github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/service/s3" "github.com/spf13/cobra" )
func PresignedPutCommand(client *s3client.Client, bucketName string) *cobra.Command { return &cobra.Command{ Use: "presigned-put", Run: func(cmd *cobra.Command, args []string) { if len(args) < 1 { log.Fatalf("请提供对象名称") } objectKey := args[0] ctx := context.Background()
// 1、生成预签名URL presignedURL, err := client.GetPresignClient().PresignPutObject(ctx, &s3.PutObjectInput{ Bucket: aws.String(bucketName), Key: aws.String(objectKey), }) if err != nil { log.Fatalf("生成预签名URL失败: %v", err) } // 2、通过http请求上传文件 body := bytes.NewReader([]byte("Hello, World!")) request, err := http.NewRequest("PUT", presignedURL.URL, body) if err != nil { log.Fatalf("创建请求失败: %v", err) } request.Header.Set("Content-Type", "application/octet-stream") httpClient := &http.Client{} response, err := httpClient.Do(request) if err != nil { log.Fatalf("上传文件失败: %v", err) } defer response.Body.Close() bodyBytes, err := io.ReadAll(response.Body) if err != nil { log.Fatalf("读取响应体失败: %v", err) } fmt.Println(string(bodyBytes)) // 3、生成预签名URL presignedURL, err = client.GetPresignClient().PresignGetObject(ctx, &s3.GetObjectInput{ Bucket: aws.String(bucketName), Key: aws.String(objectKey), }) if err != nil { log.Fatalf("生成预签名URL失败: %v", err) } fmt.Println(presignedURL.URL) // 4、通过http请求下载文件 request, _ = http.NewRequest("GET", presignedURL.URL, nil) response, err = httpClient.Do(request) if err != nil { log.Fatalf("下载文件失败: %v", err) } defer response.Body.Close() bodyBytes, err = io.ReadAll(response.Body) if err != nil { log.Fatalf("读取响应体失败: %v", err) } fmt.Println(string(bodyBytes)) },} } output
http://my_local_host:9010/test001/empty_file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Checksum-Mode=ENABLED&X-Amz-Credential=rustfsadmin%2F20251114%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20251114T142605Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=b890577bb3d128464852fe8386c274c79c7fb6ebaea1942fcd5386087162794c Hello, World!
We have not received a reply from you, so we will close the issue.
If you reply again later, we will reopen the issue.
@loverustfs Please reopen this issue.
Upstream: s3s will support PUT presigned url.
- https://github.com/s3s-project/s3s/issues/388
Hey @XiaoZhengRS ,
We have fixed this bug; please upgrade to the latest version.
Since we haven't received your feedback or confirmation, we are closing this issue to keep our tracker organized. If the issue persists after the upgrade, please leave a comment and we will reopen it.