paimon
paimon copied to clipboard
[python] Use urlpath.URL to replace pathlib.Path to avoid scheme missing in pypaimon
Purpose
Fixes URI scheme loss issue in blob-as-descriptor mode by replacing pathlib.Path with urlpath.URL throughout the codebase. This ensures URI schemes (e.g., oss://, s3://, file://) are preserved when handling blob descriptors and file paths.
Key Changes
-
Replaced
pathlib.Pathwithurlpath.URL: Unified path handling to preserve URI schemes across all filesystem operations. -
Dependency: Added
urlpath==2.0.0as a required dependency.
Impact
- FileIO: All methods now accept
URLtype instead ofPath - FileSystemCatalog: Returns
URLobjects for database and table paths - DataFileMeta.set_file_path: Accepts
URLtype - SchemaManager: Uses
URLfor table paths - UriReader.get_file_path: Returns
URLinstead ofPath
Tests
test_blob_write_read_end_to_end_with_descriptor: Verifies URI scheme preservation in blob descriptors- Sample script
oss_blob_as_descriptor.py: Demonstrates OSS blob-as-descriptor functionality with scheme preservation