Values from file-based ConfigMaps are not available as properties with plain-quarkus

[cfitzw]

What happened?

When mounting a file-based configmap, using the plain-quarkus runtime, the associated values from the ConfigMap are not made available as properties.

This is only true when using the plain-quarkus runtime and works as expected with the default runtime as of v2.6.

Steps to reproduce

1. Create a file-based ConfigMap
2. Pull in the ConfigMap into the integration
3. Attempt to reference a property:

# file: app.properties
my-variable=result

kubectl create configmap my-configmap --from-file=app.properties

# camel-k: config=configmap:my-configmap

- from:
    uri: "timer:tick?period=1000"
    steps:
      - setBody:
          simple: "{{my-variable}}"
      - to: "log:info"

Relevant log output

Camel K version

v2.6

[cfitzw]

This may or may not be related, but if using the standard .properties ${} placeholder syntax, camel k is unable to resolve these... meaning using {{base.fqdn}} in an integration fails.

# file: app.properties
base.env=dev
base.domain=domain.com
base.fqdn=cluster-${base.env}.${base.domain}

[squakez]

About the generic question, I've provided an answer in #camel-k > ✔ Camel K Quarkus ConfigMap Properties @ 💬

[squakez]

This may or may not be related, but if using the standard .properties ${} placeholder syntax, camel k is unable to resolve these... meaning using {{base.fqdn}} in an integration fails.

file: app.properties

base.env=dev base.domain=domain.com base.fqdn=cluster-${base.env}.${base.domain}

Specifically to this one, this is because Camel K in this case is using a "pre-made" generic build, so the app.properties file variable are not recalculated as it would happen when you build it from source. In version 2.7.0 we're introducing a build from source (via git project) which should resolve this situation. In any case, you'd be needing to build it from source anyway in order to calculate (and embed into the jar) the properties file.

[github-actions[bot]]

This issue has been automatically marked as stale due to 90 days of inactivity. It will be closed if no further activity occurs within 15 days. If you think that’s incorrect or the issue should never stale, please simply write any comment. Thanks for your contributions!

[eshepelyuk]

Hello

is there any progress on this issue, just tried with the latest operator 2.8 and plain-quarkus runtime 3.27 and the bug still exists.

[ogomezdi]

Hi Just to mention that in my case I've faced no problem with configmap but with secrets as mentioned #camel-k > Runtime conf base on secrets taken from files not working

[squakez]

Hello, I had a look again and I think I got now what the problem is. The plain-quarkus runtime is not (yet) able to scan a properties file coming from the configmap/secret. From core version 4.15.0/4.14.2 onward (and related quarkus version) this should be no longer happening. Ideally you should turn your secrets or configmap in a plain structure, in which case, the scan of properties work.

Following the example in the issue, while this is fixed on the core, you can have a configmap/secret defined as:

kubectl create configmap my-configmap --from-literal prop1=1 --from-literal prop2=2 ...

This approach is probably more natural on a cloud environment, reason why it was not originally developed on the core. However, I understand it can be still a useful feature. My suggestion would be however to change the way you're storing configmap/secrets to be more cloud oriented.

[eshepelyuk]

Sry, @squakez, what is core version ? it's a version of what ?

[squakez]

It's the camel framework version. Camel K uses a runtime (in case of Quarkus, camel-quarkus) which depends on a given camel core version see [1]. Likely next Camel Quarkus 3.29.0 and 3.27.x should contain the patch making this issue to disappear.

[1] https://camel.apache.org/camel-quarkus/next/index.html#_compatibility_matrix

[eshepelyuk]

It's the camel framework version. Camel K uses a runtime (in case of Quarkus, camel-quarkus) which depends on a given camel core version see [1]. Likely next Camel Quarkus 3.29.0 and 3.27.x should contain the patch making this issue to disappear.

[1] https://camel.apache.org/camel-quarkus/next/index.html#_compatibility_matrix

thank you

[eshepelyuk]

@squakez leaving it here for others.

I was able to confirm your assumption / suggestion regarding ConfigMap/Secret creation. When properties are declared as separate keys on ConfigMap/Secret - plain-quarkus runtime picks up those properties.

One difficulty though is configuring quarkus logs. I've been using property file like this

quarkus.log.category."ua.eshepelyuk".level = DEBUG
quarkus.log.category."org.apache.hc.client5.http.wire".level = DEBUG

I've tried to add those properties a separate keys to ConfigMap but it produced following K8s error

data[quarkus.log.category."ua.eshepelyuk".level]: Invalid value: "quarkus.log.category.\"ua.eshepelyuk\".level": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+')

[squakez]

Yeah, those quotes are bothering. I wonder if you can try removing them, but likely the runtime won't recognize such configuration. The alternative is to use instead env vars such as QUARKUS_LOG_CATEGORY__UA_ESHEPELYUK__LEVEL which is working for sure.

[eshepelyuk]

Yeah, those quotes are bothering. I wonder if you can try removing them, but likely the runtime won't recognize such configuration. The alternative is to use instead env vars such as QUARKUS_LOG_CATEGORY__UA_ESHEPELYUK__LEVEL which is working for sure.

Another option is using spec.traits.camel.properties

[ogomezdi]

And what if we try to use directly the option provided by quarkus Kubernetes Config, I've tried to use it adding the dependence and the property pointing to de configmaps and secrets, but not working in Camel-K. Regarding creating the cm and secrets with the option "--from-literal" under my point of view it's not useful when dealing with property files have many entries, despite the problems with properties with quoted names.

[squakez]

The manual creation of the secret was just an example. Of course, the config settings proposed makes sense when you're already dealing with a cloud native way in parameters setting. Those Kubernetes config probably requires privileges settings (see https://quarkus.io/guides/kubernetes-config#kubernetes-permissions) in order to work. IMO, if you want to be Kubernetes cloud compliant, you need to have a way to inject configmaps and secret as plain key-values stores. This would help you with rotation, reloading and vault management as well.

[squakez]

Available in Camel Quarkus 3.29.0