Error message when building multiple Docker images, intermediate image not removed

[nigredo-tori]

Expected behaviour

When doing Docker / publishLocal for multiple subprojects, all the intermediate images are removed.

Actual behaviour

Sometimes, depending on how Docker / publishLocal tasks "align", both docker prune calls happen at the same time:

[info] Successfully built ceded90ac63b
[info] Successfully built 55bf8e630c59
[info] Successfully tagged foo:latest
[info] Removing intermediate image(s) (labeled "snp-multi-stage-id=c296dd85-0422-4820-a2ce-a2fbaa6359af") 
[info] Successfully tagged bar:latest
[info] Removing intermediate image(s) (labeled "snp-multi-stage-id=f4b3c9c2-d84c-4108-a486-904219e0c533") 
[error] Error response from daemon: a prune operation is already running
[error] Something went wrong while removing multi-stage intermediate image(s)

When that happens, one of the intermediate images is not cleaned up:

$ docker images --filter=label=snp-multi-stage-id=f4b3c9c2-d84c-4108-a486-904219e0c533
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
<none>              <none>              b632802e7021        About a minute ago   729MB

Information

• sbt-native-packager 1.7.5
• SBT 1.3.12
• Ubuntu 20.04
• Docker 19.03.13

Analysis

An easy solution would be to add a tag to the Docker / publishLocal task, eliminating parallelism for the Docker-related part of the build. If we wish to preserve parallelism, there's probably a way we can split Docker / publishLocal into multiple Task stages, and only tag the one containing the docker prune call - but that would be more difficult.

Workaround

Instead of aggregating or .joining Docker / publishLocal tasks, execute them sequentially:

Docker / publishLocal := Def.sequential(
  foo / Docker / publishLocal,
  bar / Docker / publishLocal
).value

[muuki88]

Thanks for the detailed analysis and provided workarounds :heart_eyes:

I would favor the tagging solution. This would also require sbt-native-packager to configure the parallelism for the docker tag, right?

val DockerTag = Tags.Tag("native-packager-docker")

Global / concurrentRestrictions += Tags.limit(DockerTag, 1)

Docker / publishLocal := ((Docker / publishLocal) tag(DockerTag)).value

[nigredo-tori]

Yes, that's it. In the last line the tag will have to be put on publishLocalTask, since tags are tied to Tasks rather than TaskKeys.