internetarchive icon indicating copy to clipboard operation
internetarchive copied to clipboard

"files" parameter is very flexible, but is underdocumented

Open mlissner opened this issue 7 years ago • 6 comments

I wanted to upload a file and set the remote filename, and I was happy to discover this bit of documentation:

https://internetarchive.readthedocs.io/en/latest/api.html#setting-remote-filenames

Cool! But wait, I'm passing a dict to the upload function? I thought that took either a file object or a list of paths? Better double check the docs. Hm, the docs for that say:

The filepaths or file-like objects to upload. This value can be an iterable or a single file-like object or string.

So, I guess this means it can also be a dict. I like the flexibility here, but we need to document it, and test the combinations:

  1. file_obj
  2. [file_obj, file_obj2]
  3. path
  4. [path1, path2]
  5. {'filename.txt': file_obj}
  6. {'filename.txt', path}
  7. [{'filename.txt': file_obj}, {'filenam2.txt', path}]
  8. [{'filename.txt': file_obj}, path, file_obj2]
  9. sets
  10. tuples

Wow, that gets messy. I have no idea what's supported here from reading the docs or even the code itself. I'd guess numbers 1, 2, 3, 4, 5, 6 should work at least. The rest.....?

mlissner avatar Aug 03 '18 05:08 mlissner

Thanks for the feedback, @mlissner. I agree. I'll work on better documenting this!

jjjake avatar Aug 03 '18 16:08 jjjake

@mlissner Let me know what you think of the updated docstring:

https://internetarchive.readthedocs.io/en/latest/internetarchive.html#internetarchive.Item.upload

The docs could use some work in general (e.g. surfacing these details in the Quickstart, etc.), but hopefully this updated docstring helps in the meantime.

Note that 7, 8, and 9 in your examples above are not currently supported. Let me know if you wish any of them were. For uploading mulitple files with a dict, simply use a dict instead of a list of dicts. For example::

>>> r = item.upload({'remote-name1.txt': 'foo.txt', 'remote-name2.txt': 'bar.txt'})

jjjake avatar Aug 03 '18 18:08 jjjake

That's super helpful, yes!

I think they still need some tweaking here, though:

https://internetarchive.readthedocs.io/en/latest/api.html#internetarchive.upload

That method says that the first argument is identifier, which isn't right, right? And then it again doesn't explain the files argument?

mlissner avatar Aug 03 '18 23:08 mlissner

That is confusing, but it is correct. Note that that's the docs for internetarchive.upload (i.e. internetarchive.api.upload) as opposed to internetarchive.Item.upload.

The function from the API (i.e. internetarchive.upload) does the item initialization for you, and then calls the Item.upload method which is why the first arg is identifier.

Does that make sense?

I'll think this over, and try to clarify this. Thanks again, @mlissner.

jjjake avatar Aug 06 '18 20:08 jjjake

Oh, right, yes, that makes sense. I wondered what was going on there. Anyway, I assume the files arg could be better documented here, right?

mlissner avatar Aug 06 '18 20:08 mlissner

Yes, this would be a good place to document that. I'll keep this issue open until I can get around to that.

jjjake avatar Aug 06 '18 22:08 jjjake