gcsfuse_mount_spec parsing and supporting mount_options from command line
This PR improves the command-line parsing for --gcsfuse_mount_spec to allow for multiple key-value pairs and robustly handle custom GCS FUSE mount_options.
Usage
- To specify custom mount_options: The entire argument must be wrapped in single quotes ('...') to protect the inner double quotes from the shell.
--gcsfuse_mount_spec='gcs_path=gs://my-bucket/data,mount_path=/data,mount_options="implicit-dirs,metadata-cache-ttl-secs=-1"'
- To use default mount_options: No special quoting is needed.
--gcsfuse_mount_spec=gcs_path=gs://my-bucket/data,mount_path=/data
Testing Unit Tests: Added parameterized unit tests to validate the new parsing logic for various formats (complex, basic, and empty mount_options).
End-to-End Test: Confirmed the full functionality by successfully launching a Fuji 7B training job that used a custom GCS FUSE mount defined with this flag.
The existing mount flags hardcoded in the code are not changed and I do see errors with those but haven't changed them. Instead passing the mount options I need in the command.
This pull request has been automatically marked as stale because it has been inactive for 60 days. It will be closed in 7 days if no further activity occurs. If you would like to continue working on this, please remove the stale label or leave a comment.