Can't find documentation pointing towards changing the kafka settings

[julienperignon]

Hi guys,

I'm looking at overriding the max.request.size parameter via environment variables on the docker image but I can't find what should be the name for it.

Cheers

[EricWittmann]

I assume you are using the Streams storage? If yes, then I think you should be able to set the following env var:

REGISTRY_STREAMS_COMMON_MAX_REQUEST_SIZE

I haven't tried it though. The Streams storage has a couple of different property prefixes that it uses to configure various Kafka related things (producer, consumer, etc). The two prefixes for the producer are:

registry.streams.common and registry.streams.storage-producer

So basically you can, for example, set an application property like this:

registry.streams.common.max.request.size or registry.streams.storage-producer.max.request.size (either would work)

That will strip out the prefix of e.g. registry.streams.storage-producer and just pass the last part max.request.size to the producer when it's created. I believe Quarkus automatically maps ENV vars to application properties by lowercasing and converting _ to . - hence my suggestion above.

If that doesn't work please post back. :)

[julienperignon]

Hey @EricWittmann

I had tried that, unfortunately. Alongside a whole other bunch of other configuration key combinations passed in from environment variables.
I can't seem to be able to get rid of the default value which is : max.request.size = 1048576.

Cheers

[EricWittmann]

@jsenko and @famartinrh - Can you guys find a little bit of time to try and reproduce this? Maybe we can at least confirm some configuration that works.

[famarting]

the only way I've been able to make it work is using JAVA_OPTIONS @julienperignon

        - name: JAVA_OPTIONS
          value: >-
           -Dquarkus.http.host=0.0.0.0 
           -Djava.util.logging.manager=org.jboss.logmanager.LogManager
           -Dregistry.streams.common.max.request.size=20000

note I'm adding those two first properties, because otherwise that would be overriden and the log output would be messed up...

I think because we are using io.apicurio.registry.utils.PropertiesUtil.properties we are not able to set any random property with it's equivalent environment variable. The PropertiesUtil does not look like it allows for that functionality... it would be nice to find a solution. wdyt? @EricWittmann @alesj

To my understanding we have this functionalities:

• if the property is defined in application.properties i.e quarkus.http.port you can override it with it's equivalent environment variable
• if the property is defined in application.properties like registry.streams.common.bootstrap.servers and the property is being processed by our PropertiesUtil. , It's possible to override it with the equivalent env var
• if the property is not defined in the application.properties such as registry.streams.common.max.request.size and that property is being processed by our PropertiesUtil, it's not possible to override it with it's equivalent environment variable
• if the property is not defined in the application.properties but it's not processed by our PropertiesUtil, I don't know what happens in that case. I haven't been able to find an example of this.

[EricWittmann]

Here is comment from another GitHub issue that is relevant here I think:

https://github.com/Apicurio/apicurio-registry/issues/889#issuecomment-763109706