kafka-configurator icon indicating copy to clipboard operation
kafka-configurator copied to clipboard

Update circe(-yaml) when max aliases is configurable

Open r-glyde opened this issue 4 years ago • 1 comments

We use yaml anchors and aliases to reduce duplication for some of our topic configurations. Following #51 this is now broken in a project that is creating lots of topics due to an update in snakeyaml that sets a maximum number of allowed aliases. We see the error below:

ERROR com.sky.kafka.configurator.Main$ - Number of aliases for non-scalar nodes exceeds the specified max=50

Example configuration:

test-topic: &DEFAULT
  partitions: 9
  replication: 3
  config:
    cleanup.policy: compact

topic-one: *DEFAULT
topic-two: *DEFAULT
topic-three: *DEFAULT
# Repeats > 50 times

This is configurable in snakeyaml but circe-yaml (which translates between the snakeyaml AST and circe's JSON AST) does not currently give you this option - see circe/circe-yaml#136.

For now we should probably revert the circe update and wait for the linked issue to be resolved.

r-glyde avatar Mar 22 '21 17:03 r-glyde

0.14.0 includes #53 which reverted the change in circe version and so means there is no limit on the maximum number of aliases again.

I think we can leave this issue open to track being able to update to a future version of circe-yaml (when configuring the maximum should be possible).

r-glyde avatar Mar 24 '21 13:03 r-glyde