Replication factor to 1 but still have 3

[ArturoS159]

Hey ;) I try to have only one broker. I need set offsets.topic.replication.factor to 1 default is 3. Here is my values.yaml

cp-zookeeper:
    servers: 1
    prometheus:
        jmx:
            enabled: false

cp-kafka:
    brokers: 1
    configurationOverrides:
        offsets.topic.replication.factor: 1
        default.replication.factor: 1
        min.insync.replicas: 1
    prometheus:
        jmx:
            enabled: false
    
cp-schema-registry:
    enabled: true
    prometheus:
        jmx:
            enabled: false

cp-kafka-rest:
    enabled: false

cp-kafka-connect:
    enabled: false

cp-ksql-server:
    enabled: false
      
cp-control-center:
    enabled: false

And still have 3 factors and error: [2020-11-29 17:08:04,178] ERROR [KafkaApi-0] Number of alive brokers '1' does not meet the required replication factor '3' for the transactions state topic (configured via 'transaction.state.log.replication.factor'). This error can be ignored if the cluster is starting up and not all brokers are up yet. (kafka.server.KafkaApis)

Command: helm install kafka -n kafka -f .\values.yaml confluentinc/cp-helm-charts

What Am I doing wrong ?

[archy-bold]

Looks like you also need to set the configuration for confluent metrics:

  configurationOverrides:
    "offsets.topic.replication.factor": 1
    "default.replication.factor": 1
    "confluent.metrics.reporter.topic.replicas": 1

[OneCricketeer]

As the error says, transaction.state.log.replication.factor needs to be set to 1, as well

[survivant]

I have the same problem. https://github.com/confluentinc/cp-helm-charts/issues/509

there are params that are not passed to the applications

[OneCricketeer]

@survivant I'm not seeing any error about replication factors in your post

[survivant]

the logs wasn't complete

...
[2021-04-07 19:07:42,935] WARN The replication factor of the schema topic _schemas is less than the desired one of 3. If this is a production environment, it's crucial to add more brokers and increase the replication factor of the topic. (io.confluent.kafka.schemaregistry.storage.KafkaStore)
[2021-04-07 19:07:43,230] INFO ProducerConfig values:
...

my point was more that I have specify 1 replica, 1 server... in all the settings that I found in the chart's values.yaml and at the end, the applications are still looking for 3 replicas (probably a default value).

there are settings that are not passed using Helm charts

[OneCricketeer]

@survivant You need to pass through a configurationOverrides for kafkastore.topic.replication.factor or env-var for SCHEMA_REGISTRY_KAFKA_STORE_REPLICATION_FACTOR

https://github.com/confluentinc/cp-helm-charts/blob/master/charts/cp-schema-registry/templates/deployment.yaml#L88-L94

The default, indeed, is 3 - https://docs.confluent.io/platform/current/schema-registry/installation/config.html#kafkastore-topic-replication-factor

[zulrang]

@survivant You need to pass through a configurationOverrides for kafkastore.topic.replication.factor or env-var for SCHEMA_REGISTRY_KAFKA_STORE_REPLICATION_FACTOR

https://github.com/confluentinc/cp-helm-charts/blob/master/charts/cp-schema-registry/templates/deployment.yaml#L88-L94

The default, indeed, is 3 - https://docs.confluent.io/platform/current/schema-registry/installation/config.html#kafkastore-topic-replication-factor

This is incorrect. It should be just replication.factor (or SCHEMA_REGISTRY_REPLICATION_FACTOR) as per the linked source code. Confirmed that it works.

[OneCricketeer]

replication.factor alone is not a valid config

Related issue https://github.com/confluentinc/cp-docker-images/issues/448

[zulrang]

Disregard. I may have gotten this confused with the Control Center properties. After reverting the change, it worked as above (works both ways?)

[marytlf]

In my case I had to set these variables in my values file:

kafka:
  replicas: 1
  config:
    offsets.topic.replication.factor: 1
    transaction.state.log.replication.factor: 1
    transaction.state.log.min.isr: 1