egeria-docs [REPOSITORY] New Repository Strimzi Topic Integration connector

trafficstars

Name

egeria-connector-integration-topic-strimzi

Owner

davidradl

Deliverable

Provides an integration connector to bring in Kafka Topics by monitoring Strimzi custom resource definitions

Build, test and CI-CD process

answering the questions in order Same as for the crux Gradle Java I suspect there already is one Same as for Egeria core Nothing out of the ordinary Not needed

Dependencies

The is the gradle.build file we have used for testing `/*

SPDX-License-Identifier: Apache-2.0
Copyright Contributors to the ODPi Egeria project. */

repositories { mavenCentral() } group = 'org.odpi.egeria' version = 1 configurations { extralibs } apply plugin: 'java-library' ext { egeriaversion = '3.2' logbackVersion = '1.2.6' jacksonVersion = '2.13.0' junitjupiterVersion = '5.8.1' } dependencies { implementation "org.slf4j:slf4j-api:${logbackVersion}" implementation "com.fasterxml.jackson.core:jackson-databind:${jacksonVersion}" implementation "com.fasterxml.jackson.core:jackson-annotations:${jacksonVersion}" implementation "com.fasterxml.jackson.core:jackson-core:${jacksonVersion}" implementation "org.junit.jupiter:junit-jupiter:${junitjupiterVersion}" implementation "org.odpi.egeria:open-connector-framework:${egeriaversion}" implementation "org.odpi.egeria:audit-log-framework:${egeriaversion}" implementation "org.odpi.egeria:repository-services:${egeriaversion}" implementation "org.odpi.egeria:repository-services-apis:${egeriaversion}" implementation "org.odpi.egeria:topic-integrator-api:${egeriaversion}" implementation "org.odpi.egeria:kafka-open-metadata-topic-connector:${egeriaversion}" extralibs 'org.apache.httpcomponents:httpclient:4.5.13' configurations.compile.extendsFrom(configurations.extralibs) }

test { useJUnitPlatform() dependencies { testCompile("org.junit.jupiter:junit-jupiter-api:5.7.0") testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:5.7.0" testImplementation "org.junit.jupiter:junit-jupiter-params:5.7.0" testImplementation("org.junit.jupiter:junit-jupiter-api:5.7.0") } testLogging { events "passed", "skipped", "failed" } reports { html.enabled = true } filter { includeTestsMatching "*Test" } }

// Maven Central (technically sonatype oss) requires we distribute source and javadoc java { sourceCompatibility = JavaVersion.VERSION_11 targetCompatibility = JavaVersion.VERSION_11 } // More Java language settings tasks.withType(JavaCompile) { options.encoding = 'UTF-8' sourceCompatibility = "11" targetCompatibility = "11" options.incremental = true options.fork = true options.failOnError = true options.compilerArgs << '-Xlint:unchecked' << '-Xlint:deprecation' }

jar { from { configurations.extralibs.filter{it.exists()} .collect{it.isDirectory() ? it : zipTree(it)} } }`

Justification

There is no obvious repo to put this in

Assumptions

Yes all true.

Additional Information

https://github.com/odpi/egeria/issues/6021

Work Plan

Before creating the repo

[x] review overall request & get clarifications
[x] get approval on developer/TSC call from maintainers

Creating the repo

[x] Create the repo under the odpi organization (default gitignore, license, readme)
[x] Setup branch protection rules
[x] Set pull request options (allow merge, squash, rebase, suggest updating, allow automerge, do not delete head)
[x] Update security settings in repo (policy, advisories, alerts)
[x] set up permissions

First steps

[x] Initial code-drop (author)

Getting CI/CD started & refining settings

[x] Add initial build script for PR (including gradle wrapper if required)
[x] Add initial build script for merge

Further Refinement

[x] Add link in egeria docs to new repo describing purpose etc
[x] Add required credentials for publishing to container repos, maven central etc
[x] Add artifact signing if needed
[x] Add dependabot config
[x] Add CodeQL
[x] Add to LFXSecurity
[x] Add to LFAnalytics
[x] Add check for stale defects
[x] Add standard issue tags

Release

[x] Add release pipeline

Mar 17 '22 17:03 davidradl

Thanks for the info.

Note that this work was already pre-approved at a previous developer/maintainer call.

Let's go ahead with that name, we can revise if required when we discuss additional connectors.

Mar 17 '22 19:03 planetf1

Have started, but best to leave repo as-is until the initial setup is finished (may be tomorrow am)

Mar 17 '22 19:03 planetf1

@davidradl I think we have enough for you to add the first code drop (or more). Once there's some code in, I'll look at the buid side - at least initial + code scans. Do you intend to migrate from an existing repository ? If so it's possible to migrate the history if needed?

Mar 18 '22 08:03 planetf1

Hi Nigel, we do not want to migrate the history as it contains information that should not be public. I will look at putting the initial code in

Mar 18 '22 10:03 davidradl

Ok. Once you have something that could potentially build, and it's cheked in, I'll look at the workflows, gradle wrapper, - or you can give them a go if you prefer. Inevitable it will involve a few stages. We'll also need to setup artifact creation/signing (as was done in the postgres repo)

Mar 18 '22 10:03 planetf1

Quick update - I've ticked off the completed tasks. Builds are running for pr/merge with checks, though I need to fix an issue with publishing the artifacts to maven central. Also still need to verify the built jar (dependencies) and a few doc/external updates to do.

Mar 21 '22 19:03 planetf1

Since we now have the new repo - let's move this issue over there to track the initial setup tasks..

Mar 21 '22 19:03 planetf1

Artifacts now being built, signed, & distributed to maven central - or rather the snapshot repo Screenshot 2022-03-22 at 12 38 21

There's been a fair amount of gradle refactoring, dependencies, additions around source/javadoc/signing/publishing (including more metadata), and the name of the artifact itself.

I've not verified the dependencies in the uber jar yet, and there's a bit more possible cleanup - but it's a good start

There is a release pipeline in place, but untested

Have to finish off some of the docs/lfx security updates, but please feel free to take a look @davidradl

Mar 22 '22 12:03 planetf1

If using maven/gradle you can retrieve this artifact as long as the snapshot repo in in your repo list.

Via http you can retrieve via wget

wget https://oss.sonatype.org/service/local/artifact/maven/redirect\?r\=snapshots\&g\=org.odpi.egeria\&a\=egeria-connector-integration-topic-strimzi\&v\=1.0-SNAPSHOT\&c\=jar-with-dependencies -O egeria-connector-integration-topic-strimzi.jar-with-dependencies.jar

You can also get the artifact by going to the build under 'Actions' (recommend 'merge') and seeing the attachment

No docker build (egeria+connector) for now - I think it's unnecessary for now, but easy to do if required

Mar 22 '22 12:03 planetf1

@planetf1 can we close - or is there more you want todo on this ?

Mar 23 '22 11:03 davidradl

@davidradl there are still a few actions - see tasklist above

Mar 23 '22 11:03 planetf1

Additional task

[ ] Add to connector catalog

Mar 24 '22 08:03 planetf1