Tracking Issue: Complete object_store Integration Implementation

[Xuanwo]

Overview

This issue tracks the remaining work needed to fully implement the object_store trait in OpenDAL's integration layer. While the core functionality is working, several methods return NotSupported errors and some implementations could be enhanced.

Current Status

✅ Implemented

• put_opts with full PutMode support (Overwrite, Create, Update)
• put_multipart (basic implementation)
• get_opts with GetOptions support (range, conditionals, version)
• delete
• list, list_with_offset, list_with_delimiter
• MultipartUpload trait
• Error mapping between OpenDAL and object_store
• Amazon S3 builder support

❌ Not Implemented

The following methods currently return NotSupported errors:

1. put_multipart_opts (store.rs:165)

   • Required for advanced multipart upload options
   • Currently returns: "put_multipart_opts is not implemented so far"

2. copy (store.rs:248)

   • Server-side copy operation
   • Currently returns: "copy is not implemented so far"

3. rename (store.rs:255)

   • Server-side rename operation
   • Currently returns: "rename is not implemented so far"

4. copy_if_not_exists (store.rs:262)

   • Conditional server-side copy
   • Currently returns: "copy_if_not_exists is not implemented so far"

⚠️ Incomplete Implementations

1. PutResult metadata

   • Always returns e_tag: None, version: None
   • Should return actual ETag and version when available from backend

2. GetResult attributes

   • Always returns Default::default() for attributes
   • Should include object metadata/attributes from backend

3. Test coverage

   • Missing behavior tests for list operations
   • No tests for list_with_delimiter and list_with_offset

Tasks

High Priority

• [ ] Implement copy method using OpenDAL's copy operation
• [ ] Implement rename method using OpenDAL's rename operation
• [ ] Return proper ETag in PutResult when available
• [ ] Return proper version in PutResult when backend supports versioning

Medium Priority

• [ ] Implement put_multipart_opts with proper options handling
• [ ] Implement copy_if_not_exists with conditional logic
• [ ] Populate GetResult.attributes with object metadata

Low Priority

• [ ] Add comprehensive tests for list operations
• [ ] Add integration tests with different storage backends
• [ ] Document any limitations compared to native object_store implementations

Implementation Notes

1. Copy/Rename Operations: Check if OpenDAL's backend supports native copy/rename. If not, consider fallback to client-side implementation with appropriate warnings.

2. ETag/Version Support: Different backends have varying support for ETags and versioning. Implementation should gracefully handle backends that don't support these features.

3. Attributes Mapping: Need to define how OpenDAL's metadata maps to object_store's attributes.

References

• object_store documentation
• OpenDAL object_store integration

Acceptance Criteria

• All object_store trait methods return meaningful results instead of NotSupported
• Metadata (ETag, version, attributes) is properly populated when available
• Comprehensive test coverage for all implemented methods
• Documentation clearly states any limitations or backend-specific behavior

[dosubot[bot]]

Here's a concise summary of the current state and next steps for the object_store integration in OpenDAL:

Most core methods are implemented, but several (put_multipart_opts, copy, rename, copy_if_not_exists) still return NotSupported and need full implementations. Metadata handling is incomplete: PutResult does not return ETag or version info, and GetResult attributes are not populated from backend metadata. Test coverage for list operations is also lacking.

There are backend-specific complexities to consider. For example, multipart upload performance depends on tuning concurrency and chunk size, which are not enabled by default—this is being addressed in planned documentation and a performance guide (source). There is a known bug where object_store paths are percent-encoded twice, causing incorrect path handling; a fix is being discussed (source). Cloudflare R2 multipart uploads require all non-trailing parts to be the same size, so chunking must be handled carefully (source).

On the roadmap, improvements to object versioning and metadata handling are tracked in issues #2611 and #5425, which are directly relevant to the incomplete features here (source). Metadata retrieval during listing is also under discussion, as not all backends provide the same fields and retrieving metadata can impact performance (source).

For contributors: focus on implementing the missing methods, improving metadata population, and expanding test coverage. When implementing copy/rename, check for native backend support and fall back to client-side logic if needed. Handle ETag and version fields gracefully for backends that don't support them, and clarify attribute mapping between OpenDAL and object_store.

Documentation should clearly state any limitations or backend-specific behaviors, especially around metadata, versioning, and multipart uploads.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[criccomini]

Based on my reading here:

https://github.com/apache/opendal/pull/6468/files#diff-460d27db342dba48ecdea84d826580af2b63b2ab9c372c7d3c033a40f011d319L211-R257

It seems put_multipart_opts is now supported?