camel-k icon indicating copy to clipboard operation
camel-k copied to clipboard

[WIP] chore(build): Use Camel K runtime Maven structural logging module

Open christophd opened this issue 2 years ago • 1 comments

  • Load and extract structural logging artifacts from Camel K runtime module
  • Use logback configuration provided by this runtime module
  • Add logging artifacts during Dockerfile image build

IMPORTANT: This PR must wait for Camel K runtime module to be merged and released to Maven central (https://github.com/apache/camel-k-runtime/pull/833) otherwise build scripts will fail because of the missing module.

As a workaround users can explicitly build the Camel K runtime modules locally and reference the SNAPSHOT version as follows:

make RUNTIME_VERSION=1.14.0-SNAPSHOT images

or use the local Camel K runtime repository

make CAMEL_K_RUNTIME_DIR=$(pwd)/../camel-k-runtime images

This will build the Camel K image with the SNAPSHOT runtime modules and its Maven structural logging settings.

Release Note

Chore(build): Use Camel K runtime Maven structural logging module

christophd avatar Jun 20 '22 07:06 christophd

FYI @squakez

christophd avatar Jun 20 '22 07:06 christophd

@christophd can we rebase this? we are now using the next staging release for camel-k-runtime.

oscerd avatar Aug 23 '22 04:08 oscerd

@oscerd sure will do right now

christophd avatar Aug 23 '22 07:08 christophd

@oscerd I had to add another commit to account for the Camel K staging repository when loading the Camel K runtime Maven logging overlay artifacts

Tried to also retain the functionality to specify a local Camel K runtime codebase to use this instead of loading from the internet.

@squakez @tadayosi can you please also have another look at the changes made?

christophd avatar Aug 23 '22 11:08 christophd

How do we deal with this if we'll have to release artifacts together? Once we release the runtime the staging repo will go away.. while this is managed for camel k release process, I'm not sure this will be fine for this feature. It could make it unusable

oscerd avatar Aug 23 '22 11:08 oscerd

not sure but I think we need Camel K runtime to be fully released to maven central before kicking off a release for Camel K right? So we can assume the Camel K runtime bits to always be available on central?

The staging repo support is only for temporary builds like in this PR and should be given as an argument to the Make file call explicitly?

christophd avatar Aug 23 '22 11:08 christophd

No, the release is done in one shot with the runtime still in staging. If the code remain there during the release we'll have a reference to a repository that doesn't exist. So we should remove it before releasing.

oscerd avatar Aug 23 '22 11:08 oscerd

So my suggestion is to remove the staging reference for this specific feature before the release start.

oscerd avatar Aug 23 '22 12:08 oscerd

I think we should pick the artifacts from staging, when the reference is present, otherwise it must take it from central. Or am I missing something?

squakez avatar Aug 23 '22 12:08 squakez

During the release, in the beginning there will be some steps using the staging repo, but before the end of the release once the catalog has been generated and the resources generated, the reference to the staging repository will be removed, so in the end there won't any reference to the staging repository and the final artifacts/image etc. The release will take for granted the promotion of the artifacts coming from staging repository to central. In this case, we'll still have reference to the staging repo in the final artifacts and we shouldn't. Hope this makes thing a bit more clear. We are not in the situation in which: k-runtime will be released to central before starting the release of camel-k, the release vote will be one for all the components (camel-kamelets, camel-k-runtime and camel-k): for camel-kamelets we are pointing a tag, so no problem, for k-runtime we are managaing the problem through Makefile subsequent steps, for this overlay feature we are not managing the switching from staging to central at all.

oscerd avatar Aug 23 '22 12:08 oscerd

@oscerd so it basically follows the steps described in here, right?: https://github.com/apache/camel-k/blob/main/release.adoc

christophd avatar Aug 23 '22 12:08 christophd

Yes

oscerd avatar Aug 23 '22 12:08 oscerd

@oscerd so in this process when calling make release-staging the Camel K runtime staging repository is still available, right? I think this is all we need to reference the Maven overlay artifacts in order to add these to the resulting Camel K Docker image that gets pushed to the staging account.

Would that be ok?

christophd avatar Aug 23 '22 14:08 christophd

The staging repository si be available until the release vote ends, so it will be there until the release passed. But the reference will be removed in next steps of releasing

oscerd avatar Aug 23 '22 14:08 oscerd

From my understanding the maven-overlay is only used to build the Docker image. Once this is done via make release-staging and the reference to staging gets removed the release process should not build the image once again. I think we are fine with that are we?

christophd avatar Aug 23 '22 17:08 christophd

I think so.

oscerd avatar Aug 24 '22 04:08 oscerd

@oscerd Not this time but IMO in the future we should separate the releasing process of Camel K runtime from the rest of Camel K and release the runtime more frequently because of #3481 to let the runtime catch up with latest LTS releases of Camel / Camel Quarkus. We can vote each release separately.

If I'm not mistaken architecturally Camel K and Camel K runtime are designed to be decoupled so I think it's even encouraged to do so.

tadayosi avatar Aug 24 '22 05:08 tadayosi

We can do that, but instead of taking 72 hours to release, it would take up to one week if everything goes well, if not, maybe more. I don't have particular point of view on this kind of stuff, but camel-k-runtime doesn't have any reason to live without camel-k, so, releasing and voting it as standalone project doesn't make sense.

oscerd avatar Aug 24 '22 05:08 oscerd

Please note that the staging repo is now https://repository.apache.org/content/repositories/orgapachecamel-1460/ and not https://repository.apache.org/content/repositories/orgapachecamel-1459/

oscerd avatar Aug 24 '22 05:08 oscerd

but camel-k-runtime doesn't have any reason to live without camel-k

Yes, right, but my point is that if we release camel-k-runtime separately users could serve themselves for upgrading the runtime to a more secure one if they are concerned with any CVEs. Right now they have to wait for a main Camel K release to receive the latest patches for the runtime.

tadayosi avatar Aug 24 '22 05:08 tadayosi

In the last release of camel-k-runtime we just released because of camel-k, but yeah, if there is security concern or CVE to fix we could release it alone.

oscerd avatar Aug 24 '22 05:08 oscerd

removed the WIP marker as this is ready to be merged IMO

christophd avatar Aug 24 '22 11:08 christophd