put() has inconsistent results if you run from Python Console vs in a file.

[AkshitaB]

On version gcsfs==2022.11.0

Consider the following code snippet:

import gcsfs
source = 'abc'  # this is a folder - abc/file.txt, abc/subfolder/file.txt
target = 'my_bucket/folder'
fs = gcsfs.GCSFileSystem(consistency="md5")
fs.touch(target + "/.placeholder", truncate=True)  # to create a folder first.
fs.put(source, target, recursive=True)

If I run this in IPython, I end up with my_bucket/folder/file.txt and my_bucket/folder/subfolder/file.txt. But if I put the exact same code in a python file and run that, I end up with my_bucket/folder/abc.

[martindurant]

Are you sure it is a difference between the execution environment, or might it just be that you get this the second time you run the code?

cc @ianthomas23 https://github.com/fsspec/filesystem_spec/pull/1148 may fix this

[ianthomas23]

It sounds like it could well be one of the behaviours that https://github.com/fsspec/filesystem_spec/pull/1148 is making consistent. It is not necessarily 'fixing' such behaviour as sometimes different behaviour is expected depending on what is in the target location as occurs with command cp for example.

@AkshitaB It would be helpful to repeat your test but with the fs.put repeated at the end, and check what is in the target after the first and second calls to fs.put. This could be done either in a python file or interactively.

[AkshitaB]

Thanks for looking into this!

@martindurant I ensured that the bucket was empty before each run, so I don't think it's an issue with the behavior being different the second time.

@ianthomas23 I tried repeating the put command in both scenarios, and it made no difference. With the Python console, I still end up with subfolders and files (copied again), and with the python file, I end up with the parent folder.