orbax icon indicating copy to clipboard operation
orbax copied to clipboard

HNS checkpoint folder moved to _todelete_subdirectory for future deletion to avoid slow listing

Open deepikarajani24 opened this issue 5 months ago • 0 comments

When a checkpoint folder is renamed, it's moved into a todelete subdirectory. This directory is created as a sibling to the main checkpoints/ folder.

This approach prevents these renamed folders from being listed during a checkpoint restore operation, which lists the contents of the checkpoints/ directory.

Note: This implementation uses a direct GCS API call as a temporary solution. This functionality will be migrated to tf.io once it becomes available there.

deepikarajani24 avatar Jul 23 '25 18:07 deepikarajani24