cp-docker-images icon indicating copy to clipboard operation
cp-docker-images copied to clipboard

Setting SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR

Open zoltan-fedor opened this issue 6 years ago • 4 comments

Settings SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR doesn't work:

Dockerfile:

  schema_registry:
    image: confluentinc/cp-schema-registry:4.0.0-3
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR: 3

Container log: WARN The configuration 'topic.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)

The kafkastore.topic.replication.factor is an existing config value, see https://docs.confluent.io/current/schema-registry/docs/config.html

The same works for some other config values, like SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS

zoltan-fedor avatar Mar 20 '18 20:03 zoltan-fedor

I can verify this, I get exactly the same, and then:

WARN Creating the schema topic _schemas using a replication factor of 1, which is less than the desired one of 3. If this is a production environment, it's crucial to add more brokers and increase the replication factor of the topic. (io.confluent.kafka.schemaregistry.storage.KafkaStore)

@zoltan-fedor did you figure out what it was?

mamoit avatar Jan 24 '19 14:01 mamoit

same here if I set SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR to 3, The configuration 'topic.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) for 5.2.1

elvis-cai avatar Jun 06 '19 04:06 elvis-cai

This works for me in CP 5.3.1. It might have worked in earlier versions as well.

From my docker-compose file...

# kafkastore.topic.replication.factor
      #   The desired replication factor of the schema topic. The actual replication factor will be the smaller of this
      #   value and the number of live Kafka brokers.
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR: 2

The result is that the topic has 2 replicas.

I think this can be closed.

robcowart avatar Sep 28 '19 21:09 robcowart

Alright, well, glad it worked for you.

Here's the source of truth, which I believe the documentation is generated based on

https://github.com/confluentinc/schema-registry/blob/master/core/src/main/java/io/confluent/kafka/schemaregistry/rest/SchemaRegistryConfig.java#L87

OneCricketeer avatar May 29 '21 01:05 OneCricketeer