Bake in latest CDK versions at build time (in addition to on boot + cron)

[erohmensing]

We seeded the declarative_manifest_image_version table with latest versions for majors 0 and 1 (those that exist right now). This means that with no access to DockerHub, users will be able to save connectors that were built in the connector builder with CDK versions on majors 0 or 1.

When we release version 2.0, and update the builder to use the 2.0 version, there needs to be an entry for majorVersion 2 in the database in order to be able to save connector builder connectors on this version.

Normally this would happen on the bootloader start up, but if there's no access to DockerHub, that can't resolve. Ways we could approach this:

1. Decide we don't care: If a user can't access dockerhub to get new versions, they can't access dockerhub to run their versions.

• Downside: connector builder itself doesn't pull and run a source declarative manifest image, it has the baked in python package. Therefore, you can build and test a connector without dockerhub access (assuming your api is available to hit)... you just can't save it. This seems wrong - maybe you are building a connector and not intending to run it in that same cluster?

2. Get the latest versions at build time, similar to how we pull and store the connector registry at build time and then are able to load it up with a LocalDefinitionsProvider instead of a RemoteDefinitionsProvider

• Downside: most of the logic of the cron is the parsing of the response itself. We'd like to use the service code to run the build step, but it's also part of the build dependency of that service, so...

3. Instead of getting the latest version when we deploy, we can bake in the version that the connector builder is using. As of now, the connector builder server CDK version is manually updated. Note that this issue doesn't become a problem when CDK 2.0.0 is published, just when the connector builder server is updated to 2.0.0. We could look for a way to add that new entry as part of this process.

• Downsides: I think this would mean a database migration every time, which is gnarly and not fun.

I don't really like any of these options, but I think the easiest one would be a simplified LocalDefinitionsProvider-style DeclarativeManifestImageVersionsProvider which just returns a list that we keep. Part of shipping 2.0.0 to connector builder server is adding an entry for 2 in that list.