jikkou

jikkou copied to clipboard

= Jikkou : Automate the management of your Apache Kafka using GitOps approach!

image:https://img.shields.io/github/license/streamthoughts/jikkou[] image:https://img.shields.io/github/issues/streamthoughts/jikkou[] image:https://img.shields.io/github/forks/streamthoughts/jikkou[] image:https://img.shields.io/github/stars/streamthoughts/jikkou[] image:https://github.com/streamthoughts/jikkou/actions/workflows/maven-build.yml/badge.svg[]

toc::[]

See: https://streamthoughts.github.io/jikkou/[official documentation]

== Introduction

https://github.com/streamthoughts/jikkou[Jikkou] (jikkō / 実行) is an open-source tool to help you automate the management of the configurations that live on your https://kafka.apache.org/documentation/[Apache Kafka] clusters. It was developed by Kafka ❤️ to make daily operations on an Apache Kafka cluster simpler for both developers and administrators.

It can efficiently manage configuration changes for Topics, ACLs, Quotas and more with the use of resource specification files. It is also applicable to quickly replicate the configuration of a production cluster to another with a few command lines or to initialize a new cluster for testing purpose.

Note: Jikkou is the project formerly known as Kafka Specs. Old versions of this project can still be found under that name.

=== The main usage scenarios

• Create new resource entities on an Apache Kafka cluster (i.e., Topics, ACLs, and Quotas).
• Update the configurations of existing resource entities.
• Delete resource entities which are not anymore managed.
• Describe all the configuration defined for Brokers.
• Describe all the configuration defined for Topics, ACLs, and Quotas.

=== Core features that make it awesome

• Simple command line interface (CLI) for end user.
• Simple Java API on top of the Kafka's https://kafka.apache.org/30/javadoc/org/apache/kafka/clients/admin/Admin.html[Java AdminClient].
• Completely stateless and thus does not store any state (Basically: Your kafka cluster is the state of Jikkou).
• Pluggable validation rules to ensure that resources meet your requirement before being created or updated ona target cluster.
• Pluggable resource manager to extend Jikkou with cloud managed services for Apache Kafka which are supported out-of-the-box.
• Simple templating mechanism using https://jinja.palletsprojects.com/en/3.0.x/[Jinja] notation.

== Quick Installation

Jikkou is available:

• As a zip/tar.gz package from https://github.com/streamthoughts/jikkou/releases/tag/v0.12.2[GitHub Releases]
• As a fatJar available from https://repo.maven.apache.org/maven2/io/streamthoughts/jikkou/0.12.2/[Maven Central]
• As a docker image available from https://hub.docker.com/r/streamthoughts/jikkou[Docker Hub].
• As a Debian package from https://github.com/streamthoughts/jikkou/releases/tag/v0.12.2[GitHub Releases]

wget https://github.com/streamthoughts/jikkou/releases/download/v0.12.2/jikkou.deb
sudo dpkg -i jikkou.deb

NOTE: Jikkou will install itself in the directory : /opt/jikkou

It is recommended to install the bash/zsh completion script jikkou_completion:

wget https://raw.githubusercontent.com/streamthoughts/jikkou/master/jikkou_completion . jikkou_completion

or alternatively, run the following command for generation the completion script.

source <(jikkou generate-completion)

== Overview

It allows you to define the desired current state of your Topics, ACLs, or Quotas, using one or more YAML resource specification files :

kafka-topics.yml::: [source,yaml]

apiVersion: 1 spec: topics:

• name: 'my-first-topic-with-jikkou' partitions: 12 replication_factor: 3 configs: min.insync.replicas: 2

It will then take care of computing and applying the necessary changes directly to your cluster.

[source, bash]

$ jikkou --bootstrap-servers localhost:9092 topics apply
--file kafka-topics.yml
--yes

[source]

TASK [CREATE] Create a new topic my-first-topic-with-jikkou (partitions=12, replicas=3) - CHANGED **********************

[source, json]

{ "changed" : true, "end" : 1634071489773, "resource" : { "name" : "my-first-topic-with-jikkou", "operation" : "ADD", "partitions" : { "after" : 12, "operation" : "ADD" }, "replication_factor" : { "after" : 3, "operation" : "ADD" }, "configs" : { "min.insync.replicas" : { "after" : "2", "operation" : "ADD" } } }, "failed" : false, "status" : "CHANGED" }

[source]

EXECUTION in 2s 661ms (DRY_RUN) ok : 0, created : 1, altered : 0, deleted : 0 failed : 0

Most of the time, you will use Jikkou part of your CI/CD pipeline allowing you to version and ship your Apache Kafka resources directly from Git (more on https://about.gitlab.com/topics/gitops/[GitOps])

Jikkou can be used with self-hosted Kafka, managed Kafka and Confluent Cloud.

NOTE: you can use the environment variable JIKKOU_DEFAULT_KAFKA_BOOTSTRAP_SERVERS for connection to the Kafka cluster (instead of the CLI arg --bootstrap-servers)

== Documentation

Check the official https://streamthoughts.github.io/jikkou/[documentation] for further https://streamthoughts.github.io/jikkou/docs/introducion/_installation/[installation] and usage https://streamthoughts.github.io/jikkou/docs/user-guide/[instructions].

== 🏭 Developers

You need to have http://www.oracle.com/technetwork/java/javase/downloads/index.html[Java] and https://www.docker.com/[Docker] installed.

=== Dependencies

Jikkou modules are built with:

• Java 17
• https://docs.confluent.io/platform/current/clients/index.html[Apache Kafka Client] (3.0.x)

=== Build project

This project includes https://maven.apache.org/wrapper/[Maven Wrapper].

For building distribution files.

[source,bash]

$ ./mvnw clean package -Pdist

=== Build Docker Images (locally)

[source,bash]

$ make

=== Formats

This project uses the Maven plugin https://github.com/diffplug/spotless/tree/master/plugin-maven[Spotless] to format all Java classes and to apply some code quality checks.

=== Bugs

This project uses the Maven plugin https://spotbugs.github.io/[SpotBugs] and https://find-sec-bugs.github.io/[FindSecBugs] to run some static analysis to look for bugs in Java code.

Reported bugs can be analysed using SpotBugs GUI:

[source,bash]

$ ./mvnw spotbugs:gui

== 💡 Contributions

Any feedback, bug reports and PRs are greatly appreciated!

• Source Code: https://github.com/streamthoughts/jikkou
• Issue Tracker: https://github.com/streamthoughts/jikkou/issues

== 🙏 Show your support

You think this project can help you or your team to manage your Apache Kafka Cluster ? Please ⭐ this repository to support us!

== FAQ

== Licence

Copyright 2021 StreamThoughts.

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.See the NOTICE file distributed with this work for additional information regarding copyright ownership.The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License