docker icon indicating copy to clipboard operation
docker copied to clipboard

[FEA] add tests on image characteristics

Open jameslamb opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe.

Images published from this repo are intended for use in a wide variety of contexts. For example, they're used in several different cloud providers' bring-your-own container solutions (see https://docs.rapids.ai/deployment/stable/).

Those environments each have slightly different requirements for images you bring to them... things like default user, exposed ports, expected filepaths, etc.

When changes like #449 are made to modify the images here, there are currently not strong automatic protections against accidental regressions.

This issue proposes adding such protections, to improve release confidence and hopefully catch some types of issues during development instead of further downstream.

Describe the solution you'd like

I'm proposing the following:

  • on every commit to PRs or main here, test all built images with container-canary (https://github.com/NVIDIA/container-canary), with the following tests:
    • compliance with various target deployment environments (with tests sourced from https://github.com/NVIDIA/container-canary/tree/main/examples)
    • other RAPIDS-specific tests to prevent accidental regressions (with tests checked into source control here)
  • on every commit to PRs, test files in this repo with some static analyzers. For example:
    • hadolint for Dockerfiles (link)
    • shellcheck for shell scripts (link)
    • ruff for Python code (link)

Describe alternatives you've considered

N/A

Additional context

This proposal comes out of mine and @jacobtomlinson 's discussions around this: https://github.com/rapidsai/deployment/issues/277

And would help with stuff like this:

  • https://github.com/rapidsai/docker/issues/582

I'd be happy to come do the work, just looking for a 👍🏻 / 👎🏻 on the general idea (cc @raydouglass @ajschmidt8 @AyodeAwe )

jameslamb avatar May 01 '24 14:05 jameslamb

Based on my experiments in #670 so far, I think this will require some updates to container-canary:

  • [x] https://github.com/NVIDIA/container-canary/issues/61
  • [ ] https://github.com/NVIDIA/container-canary/issues/62

I'd like to try contributing those, at some point.

jameslamb avatar Jul 08 '24 16:07 jameslamb

Some good discussion on possible container-canary tests here:

  • https://github.com/rapidsai/docker/issues/712#issuecomment-2386370049
  • https://github.com/rapidsai/docker/issues/712#issuecomment-2388384139

jameslamb avatar Oct 02 '24 18:10 jameslamb