chore: split packages

[kristof-mattei]

Description

In #1407 the database packages were split, meaning the database choice is now a compilation choice, and thus we need individual packages.

In the PR the only package deployed is the one with DATABASE_BACKEND=spanner.

This PR changes that. Unfortunately the way Docker works it means we'll need to add 2 repos (as we can't rename old ones).

The proposal is:

Go from 2 repos (DOCKERHUB_REPO & UTILS_DOCKERHUB_REPO) to 3 (DOCKERHUB_REPO_SPANNER, DOCKERHUB_REPO_MYSQL & DOCKERHUB_REPO_UTILS).

The package for the Spanner DB would go to the DOCKERHUB_REPO_SPANNER, the one for MySQL DB to DOCKERHUB_REPO_MYSQL, and I did rename the variable for the utils repo from UTILS_DOCKERHUB_REPO to DOCKERHUB_REPO_UTILS to match the same naming structure.

That's it. It does not bring latest tags for the normal builds. That will go in another PR (should this one be accepted).

What needs to be done before merging this in?

Create the following repos in hub.docker.com (names are proposals):

• https://hub.docker.com/r/mozilla/syncstorage-rs-spanner
• https://hub.docker.com/r/mozilla/syncstorage-rs-mysql

Add the following Environment Variables into CircleCI:

• DOCKERHUB_REPO_SPANNER=https://hub.docker.com/r/mozilla/syncstorage-rs-spanner
• DOCKERHUB_REPO_MYSQL=https://hub.docker.com/r/mozilla/syncstorage-rs-mysql
• DOCKERHUB_REPO_UTILS=https://hub.docker.com/r/mozilla/sync-spanner-py-utils

Ensure the user $DOCKER_USER has access to those 3 repos ^ for pushing.

And after merging this in?

• Deprecate the old https://hub.docker.com/r/mozilla/syncstorage-rs
• Delete UTILS_DOCKERHUB_REPO env var.

Testing

1. Kick off a build in CircleCI with the changes above and see if it succeeds.
2. Fake tag the tip of the branch and push that to ensure it deploys the package in the respective repos.

Issue(s)

• https://github.com/mozilla-services/syncstorage-rs/issues/1482
• https://github.com/mozilla-services/syncstorage-rs/issues/1483
• https://github.com/mozilla-services/syncstorage-rs/issues/1495

Notes

1. I copied the following line, as I'm not sure it has meaning (some comment parser?): https://github.com/mozilla-services/syncstorage-rs/compare/master...kristof-mattei:syncstorage-rs:feature/split-docker-packages?expand=1#diff-78a8a19706dbd2a4425dd72bdab0502ed7a2cef16365ab7030a5a0588927bf47R454

2. My formatter made the YAML formatting more consistent. There is a view in GitHub to not display those whitespace changes. If this is not desired let me know & I'll patch them out.

[WAdama]

I would second that proposal!

[plaes]

@kristof-mattei The # touch: 1676417203 in the .circleci/config.yml file was probably added to re-trigger or retry circleci actions, so it could be removed.

[jackyzy823]

I'd sugguest to use tag (instead of different image) and add README, like:

1. "mozilla/syncstorage-rs:0.14.4" aka "mozilla/syncstorage-rs:0.14.4-spanner"
2. "mozilla/syncstorage-rs:0.14.4-mysql"
3. "mozilla/syncstorage-rs:0.14.4-tools"

[kristof-mattei]

I'd sugguest to use tag (instead of different image) and add README, like:

1. "mozilla/syncstorage-rs:0.14.4" aka "mozilla/syncstorage-rs:0.14.4-spanner"
2. "mozilla/syncstorage-rs:0.14.4-mysql"
3. "mozilla/syncstorage-rs:0.14.4-tools"

Unfortunately that doesn't fit Docker's treatment of tags. Not passing in a tag implies latest. So when you pull mozilla/syncstorage-rs which one do you get?

[tillmann]

Unfortunately that doesn't fit Docker's treatment of tags.

I don't think that is accurate. It is actually quite common to distinguish different configurations of an image via tags. It is even considered bad practice to not supply a tag ("rely on the latest-tag") when deploying a container.

E.g. the rust images: https://hub.docker.com//rust/ there are tags for various ubuntu-flavor as well as alpine-based images. latest just seems to point to the most common (stable?) one (in your case that would be "spanner" iiuc). (I guess there is plenty of more examples, just looked at wordpress: https://hub.docker.com//wordpress).

Please correct me if I'm wrong, I just stumbled upon this project with the intention of hosting an instance myself.

Another data-point: https://docs.docker.com/develop/dev-best-practices/

When building images, always tag them with useful tags which codify version information, intended destination (prod or test, for instance), stability, or other information that's useful when deploying the application in different environments. Don't rely on the automatically-created latest tag.

[pasbec]

Hi, First of all thanks for providing this software!

Just wanted to check before spending more time on this: Is this pull request still valid and will updated SQL docker images be provided in the proposed or some other way?

I'm currently planing to run my own conatiner but having to stick with version 0.13.7 forever wouldn't be a good idea.

[kristof-mattei]

Unfortunately that doesn't fit Docker's treatment of tags.

I don't think that is accurate. It is actually quite common to distinguish different configurations of an image via tags. It is even considered bad practice to not supply a tag ("rely on the latest-tag") when deploying a container.

E.g. the rust images: https://hub.docker.com//rust/ there are tags for various ubuntu-flavor as well as alpine-based images. latest just seems to point to the most common (stable?) one (in your case that would be "spanner" iiuc). (I guess there is plenty of more examples, just looked at wordpress: https://hub.docker.com//wordpress).

Please correct me if I'm wrong, I just stumbled upon this project with the intention of hosting an instance myself.

Another data-point: https://docs.docker.com/develop/dev-best-practices/

When building images, always tag them with useful tags which codify version information, intended destination (prod or test, for instance), stability, or other information that's useful when deploying the application in different environments. Don't rely on the automatically-created latest tag.

Late reply but I sit corrected. Let me find some time to see how this impacts this PR. Will need to delete those new repo references.

Thanks.

[chkpnt]

In the meantime, does anyone know of or recommend a 3rd party image compiled with mysql support?

[almereyda]

OT: Like the @porelli image? https://github.com/mozilla-services/syncstorage-rs/issues/1482#issuecomment-2211299990

https://github.com/porelli/firefox-sync