[jib-cli]: Reusable layers

[tomassatka]

Hi Team

We have started to use jib-cli and find the tool very useful. Don't know if you have any slack channel to discuss but will try to ask here.

I am wondering if jib is or is planned to support reusable layers. The idea would be to define a security layer and then share it among multiple builds within the enterprise. Like a common security layer for example... I think it is applicable on buildpacks so wondering if you plan to also introduce such feature.

thx Tomas

[diegomarquezp]

Hi @tomassatka, is the common security layer your actual use case? If so, would you mind sharing more details of your project (if you can) in terms of your setup, structure, etc., (or a similar project to give details of the situation)?

[tomassatka]

Hi @diegomarquezp

Security layer was just one example of such use case.

So in that particular case it could be cacerts emitted by company that is included to all apps. Another use cases could be agents that you can always add, for example opentelemetry for tracing, jprofiler. The idea would be a collection of pluggable modules that could be centrally managed.

I can draw some draft of how I would imagine it

apiVersion: jib/v1alpha1
kind: BuildFile

from: 
  image: foo/base-java:0.2.0

layers:
  entries:
    - name: tracing
      files:
        - src: ref://remote_layer@sha
          dest: "/opentelemetry"
    - name: security
      files:
        - src: ref://remote_layer@sha
          dest: "$JAVA_HOME/lib/security/cacerts"
    - name: envConf
      files: 
        - src: "env.sh"
          dest: "/env.sh"
    - name: application
      files:
        - src: "target/app.jar"
          dest: "/app/app.jar"

cmd: ["/bin/bash", "-c", "java $JAVA_OPTS -jar /app/app.jar"]

Is that something that you consider for future?

[emmileaf]

Hi @tomassatka! Similar to @diegomarquezp, I’m curious to understand the underlying blocker / pain point you are trying to work around with this feature request.

Since layers can be specified in the jib-cli build file to point to local files as needed, what is the proposal for pulling a shared layer hosted on remote mainly to solve - simpler configuration, build time reduction across multiple builds, or different blocker that’s preventing use of jib-cli as-is?

There may be different alternatives that can be explored, depending on your use scenario. For example, to configure shared properties across builds, Jib does support creating a custom image for use as base image. If you are interested in having customized configuration of layers that Jib does not currently support, writing your own third party extension could also be an option.

[chanseokoh]

writing your own third party extension could also be an option.

Unfortunately, the extension is only for the Jib Maven/Gradle build plugins.

#1877 is somewhat related, but in this case, the context is Jib CLI. Theoretically, because a layer is just a tar archive of files, a hacky workaround is a script to download and untar a layer tar, and then copy and place necessary files into some directory for Jib to pick up.

[tomassatka]

Hi @emmileaf

Happy to explain my intention on some simplified example.

Let's assume I would like to introduce an option to use tracing in our enterprise. To do that the applications devs would need to add jar & respective config file to point to server collecting the data. By old way you would create some page and tell the teams to fetch the jar and config and add them to each team git repo. Contrary to that would be to say to use certain layer that has been pre-build and only refer to it in jib.yaml.

One way would be to say that all agents and static layers shall be part of a base image but I would see base image more like an plain alpine/ubuntu to which binaries are added by asdf/rtx. But even managing central base images for company is not manageable due to numbers of permutations required by each team. So we end up of providing a way how teams can setup base images due to their needs.

As you asked what would be the motivation, the driver would be a simpler configuration where the jib would serve as a tool to compose multiple static layers and on top place an application layer.

I was just inspired by buildpack groups..

[tomassatka]

hi @chanseokoh In theory the layer could be something like an buildkit registry cache

[lqiu96]

Hi @tomassatka, thanks for your interest on improving Jib. Unfortunately, this is not something that we have planned or are currently planning on adding.

I'll leave this issue open so other members of the community can comment and see if this is a feature that is widely needed or has a lot of interest. For now, if you really do need this, you could always work on implementing something like this in your own fork of Jib. Thanks!