[Bug]: Self-hosted S3 Storage Configuration Issues (Blob & Avatar)

[bunomonteiro]

What happened?

I have self-hosted Affine locally following the official Docker Compose installation steps from the documentation (https://docs.affine.pro/self-host-affine/install/docker-compose-recommend).

I am attempting to configure S3 storage for both "blob" (general file uploads) and "avatar" storage. I've encountered issues with both:

1. Blob Storage (General Files) - Not Working:
   • When configuring S3 as the provider for blob storage, no objects are created in the specified bucket when I try to upload files within Affine. The storage appears to be non-functional for general file uploads.
2. Avatar Storage - Partially Working with Errors:
   • When configuring S3 as the provider for avatar storage, the upload process to the bucket does succeed, and I can see the avatar image file appearing in the bucket.
   • However, the avatar image is not displayed within the Affine UI after upload.
   • Upon checking the affine_server logs, I consistently see the following error when trying to display the avatar:

action_forbidden: Only available when avatar storage provider set to fs.
affine_server           |       at UserAvatarController.getAvatar (file:///app/dist/main.js:16867:19)
affine_server           |       at /app/node_modules/@nestjs/core/router/router-execution-context.js:38:29 {
affine_server           |   status: 403,
affine_server           |   type: 'action_forbidden',
affine_server           |   data: undefined,
affine_server           |   requestId: 'selfhosted:<...>'
affine_server           | }

To Reproduce:

1. Follow the Docker Compose installation guide for Affine.
2. Set up the bucket as usual.
3. Configure Affine's storage settings (via Admin Panel or config file) for both "blob" and "avatar" to use the aws-s3 provider with the following JSON structure:

{
  "region": "auto",
  "endpoint": "<S3_SERVER_ENDPOINT>",
  "credentials": {
    "accessKeyId": "<ACCESS_KEY>",
    "secretAccessKey": "<SECRET_KEY>"
  }
}

4. Attempt to upload a general file (e.g., an image in a page) to test blob storage.
5. Attempt to change a user's avatar to test avatar storage.
6. Observe the bucket for uploaded files and check the affine_server logs for errors

Expected behavior:

• Blob storage should successfully upload and retrieve files from the bucket.
• Avatar storage should successfully upload the avatar to the bucket and display it correctly in the Affine UI.

Questions:

• What is the correct configuration method to enable S3 storage for both blobs and avatars?
• Is avatar storage currently only supported when the provider is set to fs (filesystem), as suggested by the error message? If so, are there plans to support S3 for avatars?

Distribution version

Linux

App Version

AFFiNE 0.22.4

What browsers are you seeing the problem on if you're using web version?

Chrome

Are you self-hosting?

• [x] Yes

Self-hosting Version

AFFiNE 0.22.4

Relevant log output

Anything else?

No response

[affine-issue-bot[bot]]

Issue Status: 🆕 *Untriaged

*🆕 Untriaged

The team has not yet reviewed the issue. We usually do it within one business day. Docs: https://github.com/toeverything/AFFiNE/blob/canary/docs/issue-triaging.md

^{_{This is an automatic reply by the bot.}}

[a1ek5borodai]

1. You provided no logs describing errors with blob storage, so I can only guess about the cause. If you're using S3-compatible storage like MinIO that uses paths to target buckets, make sure you've added "forcePathStyle": true to your config. If you use TLS and self-signed certificates, make sure that your CA is trusted by parties.
2. You need to specify full path to your avatars bucket in avatar.publicPath. Like https://s3.example.com/affine-avatars/ (don't forget trailing slash). Also make sure that this bucket is publicly available for end users.

[vuongtrunghieu]

Hi, I setup AFFINE the first time today and faced very same issue. It's configured in the code, it seems we can only use "fs" storage provider for user avatars: https://github.com/toeverything/AFFiNE/blob/canary/packages%2Fbackend%2Fserver%2Fsrc%2Fcore%2Fuser%2Fcontroller.ts#L15-L19

Idk what was the reason.

[fengmk2]

Please refer to the configuration document and try again: https://docs.affine.pro/self-host-affine/administer/storage

[mogita]

Can confirm the exact issue happens on my setup too. Points to note:

• I have followed the documentation to spin up the docker compose services, all runs fine
• I have set both blobs and avatar storage with my wasabi account with the same JSON content, then checked that blobs uploading and displaying are working on the frontend
• For the avatar storage, I have updated the avatar.publicPath to the s3 URL
• The avatar requests still fail with the server log action_forbidden: Only available when avatar storage provider set to fs.

What @vuongtrunghieu has discovered looks like the cause. However, upon changing the avatar storage from aws-s3 to fs, the server throws another error about "cannot call startWith on undefined", seemingly another bug, thus the avatar settings are now stuck in limbo. Would love to contribute a fix but might need further clarification on the said code that only allows fs storage in accessing an avatar.