internal/mod/modregistry: gitlab registry does not support modules artifact type

[rogpeppe]

What version of CUE are you using (cue version)?

$ cue version
v0.8.0-alpha.1

Does this issue reproduce with the latest stable release?

N/A (feature didn't exist in latest stable)

What did you do?

env TOKEN=xxxx
env GITLAB_USER=rogpeppe
env CUE_REGISTRY=registry.gitlab.com/rogpeppe/registrytest
env CUE_EXPERIMENT=modules
env DOCKER_CONFIG=$WORK
exec docker login registry.gitlab.com -u $GITLAB_USER --password $TOKEN
exec cue mod publish v1.0.0

-- cue.mod/module.cue --
module: "foo.com@v1"
-- m.cue --
package cuemodtest
x: 2

What did you expect to see?

A passing test

What did you see instead?

> env TOKEN=xxxx
> env GITLAB_USER=rogpeppe
> env CUE_REGISTRY=registry.gitlab.com/rogpeppe/registrytest
> env CUE_EXPERIMENT=modules
> env DOCKER_CONFIG=$WORK
> exec docker login registry.gitlab.com -u $GITLAB_USER --password $TOKEN
[stdout]
Login Succeeded
[stderr]
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in $WORK/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

> exec cue mod publish v1.0.0
[stderr]
cannot put module: cannot tag [email protected]: 400 Bad Request: manifest invalid: manifest invalid; detail: "unknown media type: application/vnd.cue.module.v1+json"
[exit status 1]
FAIL: /tmp/testscript4093019353/gitlab2.txtar/script.txtar:7: unexpected command failure

It turns out that gitlab does not support arbitrary media types. See https://gitlab.com/gitlab-org/container-registry/-/issues/973.

If the internal/modregistry code is changed to use a simple application/json type for the artifact type, the test above succeeds.

TODO

• [ ] Raise a PR with GitLab to add support for our media type
• [ ] Raise a PR with the central media type "registry" to record our media type

[mxey]

For the record, this does not apply to self-managed Gitlab installations, because they apparently do not use the new container registry (yet).

[rogpeppe]

These are the custom media types currently used by internal/mod/modregistry package:

• application/vnd.cue.module.v1+json - the artifact type for the top level module manifest
• application/vnd.cue.modulefile.v1 - the artifact type for the module file blob within the above

The gitlab registry only has an issue with the former: it seems to ignore unusual blob media types.

Some documentation on how to construct a custom media type: https://datatracker.ietf.org/doc/html/rfc6838#section-3.2 More naming requirements here: https://datatracker.ietf.org/doc/html/rfc6838#section-4.2

The list of currently assigned IANA media types is here: https://www.iana.org/assignments/media-types/media-types.xhtml

Some instructions for registering a new media type are here: https://www.w3.org/2020/01/registering-mediatypes

Here is a previous Gitlab PR that added support for a new media type (application/vnd.ansible.collection): https://gitlab.com/gitlab-org/container-registry/-/merge_requests/1063

[myitcv]

Per our discussion offline, here is a summary of the proposed plan:

• Raise a PR with GitLab to add support for two media types mentioned in https://github.com/cue-lang/cue/issues/2817#issuecomment-1953796090, specifically:

application/vnd.cue.module.v1+json
application/vnd.cue.modulefile.v1

• Use this issue to better understand the requirements for submitting something to IANA.

In the worst case, we get pushback/feedback from IANA on our choice of media type. In which case we can revisit. Hence we should try and flush out issues there ASAP.

But the most important thing for GitLab users is to get things unblocked.

[myitcv]

But the most important thing for GitLab users is to get things unblocked.

@rogpeppe raised https://gitlab.com/gitlab-org/container-registry/-/issues/1203, which was then resolved by the now-merged https://gitlab.com/gitlab-org/container-registry/-/merge_requests/1588. We will update in this issue as we learn about the availability of that fix via GitLab "products".

[rogpeppe]

According to a reply to the above-mentioned issue, the fix should be released with GitLab 16.10, due out 2024-03-21.