Bug: Passing an `S3PutObject` to `s3.put_files` treated as `tuple` of key path values.

Open narayanacharya6 opened this issue 2 years ago • 0 comments

Because S3PutObject is a named tuple, the else branch of this if statement is never hit, irrespective of user passing Tuple[str, PutValue] or S3PutObject.

https://github.com/Netflix/metaflow/blob/5908c4e1481c15a93ec5ef845cec630c4ef84919/metaflow/plugins/datatools/s3/s3.py#L1322

If you pass an S3PutObject that has path set and not value then the put_files method is unable to resolve the file to upload because in line

https://github.com/Netflix/metaflow/blob/5908c4e1481c15a93ec5ef845cec630c4ef84919/metaflow/plugins/datatools/s3/s3.py#L1324C21-L1324C39

path = key_path[1] resolve to path to None and it can no longer find the file.

Apr 19 '24 05:04 narayanacharya6