tmail-backend
tmail-backend copied to clipboard
[PERF] Optimize PublicAsset create by using S3 copy object api
Why?
ref: https://github.com/linagora/tmail-backend/discussions/1061
Requirement
https://github.com/linagora/james-project/issues/5222
How
Introduce BlobStoreToolBox
interface
public interface BlobStoreToolBox {
Mono<BlobId> copyBlob(BlobId blobId, BucketName sourceBucket, BucketName targetBucket);
}
- The S3 implement looks like:
public class S3BlobStoreToolBox implements BlobStoreToolBox {
private final S3AsyncClient s3Client;
@Override
public Mono<BlobId> copyBlob(BlobId blobId, BucketName sourceBucket, BucketName targetBucket) {
return Mono.fromFuture(s3Client.copyObject(CopyObjectRequest.builder()
.sourceBucket(sourceBucket.asString())
.sourceKey(blobId.asString())
.destinationBucket(targetBucket.asString())
.destinationKey(blobId.asString())
.build()))
.publishOn(Schedulers.parallel())
.thenReturn(blobId);
}
}
- For another implement
public class NaiveBlobStoreToolbox implements BlobStoreToolBox {
private final BlobStore blobStore;
@Override
public Mono<BlobId> copyBlob(BlobId blobId, BucketName sourceBucket, BucketName targetBucket) {
return Mono.from(blobStore.readBytes(sourceBucket, blobId))
.flatMap(bytes -> Mono.from(blobStore.save(targetBucket, bytes, BlobStore.StoragePolicy.LOW_COST)));
}
}
Location: tmail-backend/blob/blob-extensions
Adapt PublicAssetSetMethod
Currently, we are using blobResolvers
to resolve creationRequest.blobId
. I propose that we use UploadRepository
directly to retrieve UploadMetaData
.
Impact scope:
- The blobId in the client request must have the prefix
uploads-
. - We can add a new API
Publisher<UploadMetaData> listUploads(Username user, UploadId id)
toUploadRepository
, or uselistUploads
directly and filterUploadMetaData
(in case the upload size for each user is not large). - Update the
PublicAssetCreationRequest
POJO by addingblobId
andbucketName
field, and removingcontent
field:
case class PublicAssetCreationRequest(size: Size,
contentType: ImageContentType,
identityIds: Seq[IdentityId] = Seq.empty,
blobId: BlobId,
bucketName: BucketName)
Inject BlobStoreToolBox
into PublicAssetRepository
and use the copyBlob API.
Guice binding
- using
S3BlobStoreToolBox
for S3blob store chooser - using
NaiveBlobStoreToolbox
for other remain