debezium icon indicating copy to clipboard operation
debezium copied to clipboard

DBZ-7786 Add MariaDB connector documentation

Open Naros opened this issue 1 year ago • 3 comments

https://issues.redhat.com/browses/DBZ-7786

@roldanbob, as we discussed, I duplicated the MySQL documentation and commented on the parts that were shared. The one part I did not comment on was the properties. I think it's reasonable for us to compare the properties between the two connectors. The shared ones can be moved to a shared include file, and those that do not remain specific to the appropriate connector. Hopefully, that's okay.

Naros avatar May 29 '24 22:05 Naros

@Naros Quick update so that you don't think this fell into a black hole. I started work on this, but after talking to others in the docs group, I learned about similar work that was done on another project. So, for now, rather than reinvent the wheel, I'm putting pausing work, pending a meeting early next week to discuss an alternate approach. If the other approach doesn't dovetail well with the structure of the upstream repo, I'll resume work on the current track.

roldanbob avatar Jun 06 '24 15:06 roldanbob

Hi @Naros, thanks for your contribution. Please prefix the commit message(s) with the DBZ-xxx JIRA issue key.

github-actions[bot] avatar Jun 26 '24 23:06 github-actions[bot]

@Naros I've tested using shared content in the MariaDB docs, and it's looking good -- at least in the upstream. Next step is to validate that things work in the downstream build. If I'm able to build successfully, I'll propagate the changes to MySQL. One thing that I'm a little bit unhappy about is that after implementing the properties as description lists, it's a little dissonant to go from looking at the Required and Advanced properties that are in that format to viewing the remaining properties (schema history, Kafka signals, sink notifications, and so forth), which are still formatted as tables. Not ideal in terms of consistency. But as a POC, I think the current work is a good incremental step, and it provides a roadmap for how we might move those other properties sets to shared lists.

roldanbob avatar Jun 27 '24 02:06 roldanbob

The latest changes address ID collisions and linking problems that caused the downstream build to fail. The MariaDB connector doc now uses shared content, including shared lists of required and advanced configuration properties, and builds upstream and downstream. The next step is to replace content in the MySQL docs with the shared content.

roldanbob avatar Jul 03 '24 20:07 roldanbob

I think we're good on this. I updated the MySQL doc so that it shares content the same as the MariaDB docs, and tested it in upstream and downstream builds. The downstream build threw some errors at first, but I refreshed the branch and just got a clean build. 🤞🏼 Let's let this marinate until I return on the 8th, and if I can reproduce that success, we'll be ready to merge this. Note that I did make a few content changes along the way. Nothing that requires major technical review, but if you have a chance to give it a once over, that might be good. Thanks.

roldanbob avatar Jul 04 '24 03:07 roldanbob

Setting up the two connectors to share content was pretty straightforward for the community version, but turned out to be way more complex for the downstream. I hadn't anticipated that xrefs hosted in the shared content are not parsed if the link target has a ModuleID defined for it. As a result, I've removed ModuleID declarations in the source for several topics.

roldanbob avatar Jul 08 '24 18:07 roldanbob

Adds documentation for the new MariaDB connector. The connector is based on the existing MySQL connector and much of the content is identical. For long-term efficiency, in the upstream repository, the shared content is maintained in 3 files:

  • /partials/modules/all-connectors/shared-mariadb-mysql.adoc
  • /partials/modules/all-connectors/ref-mariadb-mysql-rqd-connector-cfg-props.adoc
  • /partials/modules/all-connectors/ref-mariadb-mysql-adv-connector-cfg-props.adoc

The content from these files is incorporated into mysql.adoc and mariadb.adoc through the use of include directives. The latter two files contain the Required and Advanced properties, organized as definition lists. The other file is organized into tagged regions. In the main connector files, each include directive specifies a tag attribute in the shared file that corresponds to the region of the file that contains the relevant text. Within these regions, in addition to the standard conditionalization that differentiates between community and product content, some further conditionalization designates content that applies to only one of the connectors (MARIADB or MYSQL).

In testing, I discovered that the fetch script that pulls content into the downstream repository is unable to process xref links that target the ModuleIDs that are defined for some topics in the sources to provide enhanced descriptions of the topic content. The tooling successfully converts the original IDs for these link targets to the designated ModuleIDs, but it does not convert the IDs in the links, resulting in Unknown ID errors in the asciidoc build, which prevent some content from building. To eliminate these errors, I removed ModuleID declarations as needed.

Tested in local Antora and downstream builds.

This should be ready to merge now.

roldanbob avatar Jul 08 '24 18:07 roldanbob

Hi @roldanbob, I found a few places where the connector-name was not being set properly, otherwise LGTM.

Oh, thanks for catching that. I'm definitely a bit bleary-eyed after testing this pretty much non-stop for the past week or so. Sorry for the multiple commits, but the option to batch them didn't seem to be available.

roldanbob avatar Jul 08 '24 20:07 roldanbob

Applied to main (=3.0) and branch 2.7.

Naros avatar Jul 09 '24 12:07 Naros