Preserve file permissions in artifacts

[iand675]

Uploading and downloading artifacts doesn't properly preserve file modes. Downloading executables in particular would be very useful since when moving tasks out into concurrent steps it can break build scripts to lose the executable bit. Also not great when building docker images out of the executables.

https://buildkite.com/betterteamapp/dummy-pipeline/builds/7#2db28ceb-5fd4-45db-abe9-8dbb4f1623cd is a repro of the issue. It's a dummy repro, but the key is the pipeline:

steps:
  - command: "echo \"#/usr/bin/env bash\\necho wow\" > wow\nchmod +x wow\nbuildkite-agent artifact upload wow"

  - wait

  - command: "buildkite-agent artifact download \"wow\" .\nls -la\n./wow"

Expected: ./wow outputs wow. Actual: Build fails.

[lox]

Yup, I agree this would be a nice addition.

[nightah]

Is the recommendation at this stage still to add the executable bit back to artifacts?

[dominicbarnes]

I would like to see this addition as well. In my case, I'm building a binary in 1 step then passing it along to a different step for packaging and publishing. Right now I need to chmod +x on that artifact after downloading it, but ideally it would preserve the original permissions.

[keithduncan]

One way to do this would be to upload a tar archive which includes support for ownership and permissions data.

This can be done customer side but by way of tradeoff you won’t be able to piecewise select files from the archive when downloading, you’ll have to download the entire archive and then selectively extract.

[lox]

This would be really nice to have, it's fairly surprising it doesn't work to users.

If I was to add a PR that added whether the artifact was executable in https://github.com/buildkite/agent/blob/97f5207dd2f8818f905c4d990d010ca350ee7c7d/agent/artifact_uploader.go#L217 would that be something y'all would consider supporting? It might not even need to be stored in the database, it could be propagated to S3 as user metadata (x-amz-meta-executable or similar) and then retrieved on download.

[lox]

Actually, I take it back, it would need to be returned from the artifact search.

[lox]

I'd be happy to implement the agent side if someone would assist on the server side.

[pda]

Any thoughts on cross-platform file mode / permissions? I wonder how Windows would be supported here. Maybe they're less relevant for Windows, and can be ignored during upload/download by those agents.