API should provide source file hashes for file-like objects

[doxygen-spammer]

Problem

It is difficult to automatically determine whether the package on ContentDB is up-to-date. Releases and screenshots are file-like objects that can be updated through the API, and easily downloaded. A release script needs to know whether a new version of these objects needs to be uploaded.

There are 3 types of information available for each object:

1. The downloaded file does not necessarily match the uploaded file. Screenshots might be transcoded, releases might be rearchived or have additional metadata filled in.
2. The object’s ID does not help, because it can only be compared against a persistent storage of IDs. This is inconvenient, and often impossible.
3. The title can be set to a deterministic value, and you can reasonably assume that a release titled “v1.0.0” is outdated when you are releasing “v2.0.0”. But it doesn’t help much for screenshots, since updating a screenshot usually does not change its purpose, and therefore the title stays the same.
4. (Only for releases) The commit hash is not unique if multiple packages share a repository. Additionally, adding commits to the repository does not necessarily change the release file.

Solutions

When a file-like object is created or updated, a hash of the uploaded file should be stored, and provided in the API. Whether this hash is Fletcher-32 or SHA-3xyz does not really matter, unless you are trying to attack your own releases. ;) I prefer SHA-1 for simplicity.

Suggested API documentation:

• GET /api/packages/<username>/<name>/releases/<id>/ (Read)
• POST /api/packages/<username>/<name>/releases/new/ (Create)
  • Requires authentication.
  • Body can be JSON or multipart form data. Zip uploads must be multipart form data.
  • title: human-readable name of the release.
  • hash: (Read-only) SHA-1 hash of original release file, for informational purposes.
  • For Git release creation:
    • method: must be git.
    • ref: (Optional) git reference, eg: master.
  • For zip upload release creation:
    • file: multipart file to upload, like <input type="file" name="file">.
    • commit: (Optional) Source Git commit hash, for informational purposes.
  • You can set min and max Minetest Versions using the content's .conf file.
• GET /api/packages/<username>/<name>/screenshots/ (List)
  • Returns array of screenshot dictionaries with keys:
    • id: screenshot ID
    • approved: true if approved and visible.
    • title: human-readable name for the screenshot, shown as a caption and alt text.
    • url: absolute URL to screenshot.
    • created_at: ISO time.
    • order: Number used in ordering.
    • is_cover_image: true for cover image.
    • hash: SHA-1 hash of original image file.

Alternatives

Update releases on ContentDB manually, investing time of impatient mod authors (me).

Additional context

Yes, I am finally working on a new tool that automates release uploads. It will be the third, and most complex-flexible, method to upload releases. (Unlike git update detection, it shall update description and screenshots too.)

I think the API is already quite good, I encounter only minor obstacles like this one. :)