microcks Why do microcks compose its own mongoDB/keycloak/kafka chart instead of use public chart from bitnami?

Reason/Context

I've been trying to modernise the Helm chart for Microcks because I have noticed that the current structures are incompatible with the mainstream AWS CI flow. I believe there's an opportunity to improve this and I'd be happy to help by raising a pull request if the team is interested in renewing the Helm chart.

I've made some serious changes to those self-composed charts (pg/mongo/keycloak/kafka) and turned to utilize these with the bitnami chart.

I've been wondering if it might be beneficial for us to consider using the public chart from bitnami instead. It seems like it could streamline our processes and provide more alignment with industry standards.

Description

none

Implementation ideas

No response

Jun 12 '24 09:06 Eroyi

👋 @Eroyi

Welcome to the Microcks community! 💖

Thanks and congrats 🎉 for opening your first issue here! Be sure to follow the issue template or please update it accordingly.

📢 If you're using Microcks in your organization, please add your company name to this list. 🙏 It really helps the project to gain momentum and credibility. It's a small contribution back to the project with a big impact.

If you need to know why and how to add yourself to the list, please read the blog post "Join the Microcks Adopters list and Empower the vibrant open source Community 🙌"

Hope you have a great time there!

🌟 ~~~~~~~~~ 🌟

📢 If you like Microcks, please ⭐ star ⭐ our repo to support it!

🙏 It really helps the project to gain momentum and credibility. It's a small contribution back to the project with a big impact.

Jun 12 '24 09:06 github-actions[bot]

Hi @Eroyi Thanks for joining!

The Microcks Helm Chart was initialized a long time ago with no refactoring at all... It is very far from following best practices, and a draft PR was started some months ago. See #1079.

Initially, the goal was to provide minimal support for external dependencies (Keycloak, Mongo) so that users could start very easily from scratch. Dependencies setup that way are far from being production-ready deployments and we recommend not installing them via Microcks chart (putting keycloak.install=false and mongodb.install=false) but rather using other ones coming from vendors or Bitname for any serious and high-scale setup.

While I'm not an expert in Helm, I did encounter some challenges when exploring Bitnami charts in the past. It was not straightforward to set up these components for simple or small-scale deployments. This is why we persisted in providing our own minimalist solution, which is ideal for quick startup scenarios.

That said, we're still open to improvements and lower the burden of maintaining this. Maybe a demonstration/PoC on what's possible to do now in the spirit of the initial goal can help us figure it out? Have you checked the Draft PR? Maybe we could start some kind of Design discussion on that topic if you wish to contribute it? WDYT?

Jun 12 '24 10:06 lbroudoux

Hi @Eroyi Thanks for joining!

The Microcks Helm Chart was initialized a long time ago with no refactoring at all... It is very far from following best practices, and a draft PR was started some months ago. See #1079.

Initially, the goal was to provide minimal support for external dependencies (Keycloak, Mongo) so that users could start very easily from scratch. Dependencies setup that way are far from being production-ready deployments and we recommend not installing them via Microcks chart (putting keycloak.install=false and mongodb.install=false) but rather using other ones coming from vendors or Bitname for any serious and high-scale setup.

While I'm not an expert in Helm, I did encounter some challenges when exploring Bitnami charts in the past. It was not straightforward to set up these components for simple or small-scale deployments. This is why we persisted in providing our own minimalist solution, which is ideal for quick startup scenarios.

That said, we're still open to improvements and lower the burden of maintaining this. Maybe a demonstration/PoC on what's possible to do now in the spirit of the initial goal can help us figure it out? Have you checked the Draft PR? Maybe we could start some kind of Design discussion on that topic if you wish to contribute it? WDYT?

Yeah I would love to contribute! I haven't got the time to scroll through the #1079 , but these are what I've managed to achieve now, and whats left to do:

[x] Separate async, postman and microcks from the original omni deployment yaml file.
[x] Trim down and split the service.yaml file for each component.
[x] Replace MongoDB with the latest Bitnami Chart.
[x] Replace Keycloak(PostgreSQL) with the latest Bitnami Chart to solve the permission issue.
[x] Replace Kafka(Zookeeper) with the latest Bitnami Chart, using KRaft to get rid of the Zookeeper.
[x] Import microcks-realms.json in configMap with Bitnami Keycloak Chart.
[x] Allowed users to customise the password for user, manager and admin accounts.
[x] Fixed the annotations section to adapt the AWS Network CSI.
[x] Fixed microcks uri not having a port number attached when passed into the microcks-realms.json file to be used as redirectUris
[x] Provided an entry in Values.yaml to custom the HTTP protocol, allowing microcks using http:// to connect with keycloak and http prefix in redirectUris.
[ ] Re-organised the Values.yaml for better integration (60%)
[ ] Optimise the _helper.tpl
[ ] Regression test for all the integration and changes (Fixes mentioned above have been tested in our environment).

Initially, the goal was to provide minimal support for external dependencies (Keycloak, Mongo) so that users could start very easily from scratch

I fully agree about this, therefore I've placed entries in the main Values.yaml that allowed users to disable the built-in dependencies and reuse their existing components. And if they are more eligible to use the built-in one for a quick start, all Bitnami charts have been optimized to minimise the performance requirement and can be custom in Values.yaml

Jun 13 '24 04:06 Eroyi

Awesome! Looks nice! Could you open a draft PR? It will be easier to review it step by step. Thanks!

Jun 13 '24 07:06 lbroudoux

Awesome! Looks nice! Could you open a draft PR? It will be easier to review it step by step. Thanks!

Sure! Here is the PR (#1200 ), but still WIP

Hopefully it will be done within the next week 🫠

Jun 13 '24 09:06 Eroyi

@lbroudoux I'm not familiar with async when it works with schema-registry, I'll put the schema-registry charts on, and make some proper tunning in values.yaml, but no guarantee it will work as expected.

Jun 20 '24 08:06 Eroyi

This issue has been automatically marked as stale because it has not had recent activity :sleeping:

It will be closed in 30 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. Microcks is a Cloud Native Computing Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience :heart:

Jul 21 '24 00:07 github-actions[bot]