testcontainers-go icon indicating copy to clipboard operation
testcontainers-go copied to clipboard

feat: more robust Reusable containers experience

Open mdelapenya opened this issue 1 year ago • 5 comments

What does this PR do?

This PR modifies the way the Reuse mode is implemented in the project, taking tc-java as the implementation reference. For that:

  • instead of passing Reuse plus the container name, we don't need to pass a container name anymore, only the Reuse field. This is a breaking change in terms of semantics.
  • we are calculating a hash of the container request just before the creation of the container.
  • we use github.com/mitchellh/hashstructure/v2 for the hash of the container request, with the following exceptions:
    • functions in the struct are ignored (e.g. config modifiers). Please search for hash:"ignore" to identify them all.
    • if the Files field contains a reference to a directory in the host, it won't participate in the calculation of the hash, as we don't want to traverse the directory getting the hash for the directory tree.
    • for the real files in the Files field, we'll read their bytes and use them get a hash, which will be added to the struct hash.
  • for containers marked for reuse, two labels will be added:
    • org.testcontainers.hash: the hash of the container request.
    • org.testcontainers.copied_files.hash: the hash of the files copied to the container using the Files field in the container request.
  • those two labels will be used to check if a container exists in the docker daemon: the code will do a backoff search up-to 5 seconds (not configurable, maybe desired?), and if there is no container, it will create a new one. Else, the container will be reused. This backoff search is needed because we want to support the use case where multiple and different test processes start the same container, so they can "reuse" it.
  • the SessionID label of the container will be removed from the container request before the container is created, to not tell Ryuk this container must be terminated alongside the test session. Therefore, the container persists across test sessions.
  • the ReuseOrCreateContainer from the provider struct has been deprecated for removal in the upcoming releases.
  • the docs for Reuse has been updated, trying to make it clear the expectations around Reuse.
  • the tests have been updated too, to reflect the new state for Reuse.
  • modules are receving a functional option for marking module containers as reusable.

Why is it important?

Create a more consistent experience around reuse, based on the container state and not forcing the user to add a container name, that can be reused by mistake in a totally different container request.

We expect this change helps the community, but at the same time warn about its usage in parallel executions, as it could be the case that two concurrent test sessions get to the container creation at the same time, which could lead to the creation of two containers with the same request.

Related issues

  • Closes #2726
  • Closes #2470
  • Closes #2654
  • Closes #2445
  • Closes #568

Docs

Render URLs:

  • Reuse docs: https://deploy-preview-2768--testcontainers-go.netlify.app/features/creating_container/#reusable-container
  • Reuse option for modules: https://deploy-preview-2768--testcontainers-go.netlify.app/modules/chroma/#withreuse

Follow-ups

Please try out this branch, and read the docs so that they can be improved if needed :pray:

mdelapenya avatar Sep 04 '24 16:09 mdelapenya