Consul as default

[deepu105]

trafficstars

Switch to use Consul as default service discovery mechanism

Part of #19109

---

Please make sure the below checklist is followed for Pull Requests.

• [ ] All continuous integration tests are green
• [ ] Tests are added where necessary
• [ ] The JDL part is updated if necessary
• [ ] jhipster-online is updated if necessary
• [ ] Documentation is added/updated where necessary
• [ ] Coding Rules & Commit Guidelines as per our CONTRIBUTING.md document are followed

When you are still working on the PR, consider converting it to Draft (below reviewers) and adding skip-ci label, you can still see CI build result at your branch.

[deepu105]

@pascalgrimaud @mraible I have done some tests but would be nice to have a second pair of eyes to make sure I haven't missed anything

[pascalgrimaud]

This PR won't fix the ticket as you need to check these items too:

• JHipster Online: https://github.com/jhipster/jhipster-online -> https://start.jhipster.tech/generate-application
• Sample JDL: https://github.com/jhipster/jdl-samples
• Sample Apps: https://github.com/jhipster/jhipster-sample-app-gateway and https://github.com/jhipster/jhipster-sample-app-microservice
• JHipster Base: https://github.com/jhipster/jhipster-base
• Probably IDE too: https://github.com/jhipster/jhipster-ide
• The documentation: https://github.com/jhipster/jhipster.github.io

[deepu105]

@pascalgrimaud thanks. Yes i'll update the docs and JH online. JDL samples are fine as we still have registry support and maybe I'll update some of them to use consul. I'll check the other apps

[deepu105]

Folks does anyone have an objection with merging this? @pascalgrimaud @mraible @Tcharl @gmarziou @atomfrede ?

[mraible]

I believe this is a breaking change for v8 and I'm not sure if there are other v8-related issues that've been merged. Personally, I'd like another release of 7.9.x to fix microservice and microfrontend issues I've found. I'll use this release as the basis for my JHipster book and fall conference talks.

[DanielFran]

@deepu105 Agree with @mraible, we have few fixes merged, would be better to do a latest v7 version. Still no breaking change merged...

[mshima]

v7.9.0 was unusually big with a lot of bugs, I agree to release another version, maybe with spring boot v2.7.3 due in 3 days.

[atomfrede]

Sound good to have another 7.9.x release and afterwards start the 8.x cycle. If boot 2.7.3 is around the corner thats perfect timing

[gmarziou]

@deepu105 what will happen to a user who has a project in v7 using Eureka and who runs jhipster upgrade to move to v8?

[deepu105]

Ok that sounds good. No hurry, I just didn't want to keep handling merge conflicts.

@gmarziou nothing will happen to existing projects since this only updates consul config for k8s and makes it the default choice in CLI and JDL. So only fresh projects where user chooses consul or doesn't specify a choice in JDL will get the new consul k8s setup

[deepu105]

@DanielFran will you do the release? The 7.x maintenance branch needs to be recreated then. If you want me to do it let me know.

[mshima]

There are 4 png files at test-integration/samples/jdl-entities. We should add .gitignore, but the bug is at jdl plugin that generates png files without open the jdl.

[deepu105]

@pascalgrimaud @DanielFran @mshima @mraible do you guys thinkl there will be any more releases needed from 7.x? else I would like to merge this and proceed with other upfdates for docs and so on

[DanielFran]

@deepu105 I do not believe we will do any more v7 release

[pascalgrimaud]

Same opinion than @DanielFran so let's go ahead and merge this @deepu105 I'm updating the old branch v7.x_maintenance

[deepu105]

Cool. I'll fix the conflicts and merge this tomorrow

[mraible]

@deepu105 Conflicts need to be fixed.

[deepu105]

i'm planning to fix conflicts and merge it beginning of march when I have some time off

[mraible]

The beginning of March seems sooo far away! Is there anything I can do to help get this resolved sooner?

[deepu105]

I need to fix conflicts and test if everything works well. I think the testing is what takes the most time. Let me see if I can start this week on Okta time :P

[deepu105]

I fixed the conflicts, but given the amount of change that happened between, I expect a lot of test failures and stuff, so I'll monitor and fix them along the way. Once the builds are green, @mraible would love some help to test some usual configurations

[deepu105]

The tests should be fixed. @mraible can you try and give feedback?

[mraible]

I tried this branch with reactive-mf.jdl.

I started by downloading it:

jhipster download reactive-mf.jdl

Then, I removed all the lines that have serviceDiscoveryType consul.

I created all the apps using:

jhipster jdl reactive-mf.jdl

I was able to get everything working, but running npm run e2e took several attempts before it passed in the blog app. This might be related to Neo4j. I noticed it has a warning when you fetch the blog list.

2023-02-14T08:41:20.477-07:00 DEBUG 55855 --- [ctor-http-nio-7] c.o.d.blog.web.rest.BlogResource         : Exit: getAllBlogs() with result = Flux.collectList ⇢ at com.okta.developer.blog.web.rest.BlogResource.getAllBlogs(BlogResource.java:199)
2023-02-14T08:41:20.537-07:00  WARN 55855 --- [o4jDriverIO-3-2] org.springframework.data.neo4j.cypher    : Neo.ClientNotification.Statement.UnknownPropertyKeyWarning: The provided property key is not in the database__MATCH (blog:`Blog`) RETURN blog{.handle, .id, .name, __nodeLabels__: labels(blog), __internalNeo4jId__: id(blog), __elementId__: toString(id(blog)), Blog_HAS__jhi_user: [(blog)<-[:`HAS_`]-(blog_user:`jhi_user`) | blog_user{.activated, .created_by, .created_date, .email, .first_name, .image_url, .lang_key, .last_modified_by, .last_modified_date, .last_name, .login, .user_id, __nodeLabels__: labels(blog_user), __internalNeo4jId__: id(blog_user), __elementId__: toString(id(blog_user)), jhi_user_HAS_AUTHORITY_jhi_authority: [(blog_user)-[:`HAS_AUTHORITY`]->(blog_user_authorities:`jhi_authority`) | blog_user_authorities{.name, __nodeLabels__: labels(blog_user_authorities), __internalNeo4jId__: id(blog_user_authorities), __elementId__: toString(id(blog_user_authorities))}]}]}__                                                                                                                                                                                                                                                                                             

^_One of the property names in your query is not available in the database, make sure you didn't misspell it or that the label is available when you run this statement in your application (the missing property name is: image_url)

I noticed some other warnings in my terminal after running e2e tests.

2023-02-14T08:50:21.212-07:00 DEBUG 55855 --- [ctor-http-nio-5] c.o.d.blog.web.rest.PostResource         : Exit: getAllPosts() with result = Mono.map ⇢ at com.okta.developer.blog.web.rest.PostResource.getAllPosts(PostResource.java:198)
2023-02-14T08:50:21.218-07:00  WARN 55855 --- [4jDriverIO-3-14] org.springframework.data.neo4j.cypher    : Neo.ClientNotification.Statement.UnknownRelationshipTypeWarning: The provided relationship type is not in the database.__MATCH (post:`Post`) OPTIONAL MATCH (post)<-[__sr__:`HAS_POST`]-(__srn__:`Tag`) WITH collect(id(post)) AS __sn__, collect(id(__srn__)) AS __srn__, collect(id(__sr__)) AS __sr__ RETURN __sn__, __srn__, __sr____                                                   

^_One of the relationship types in your query is not available in the database, make sure you didn't misspell it or that the label is available when you run this statement in your application (the missing relationship type is: HAS_POST)

Next, I ran npm run java:docker:arm64 in each project to build a Docker image.

I started them all using docker compose up in the docker-compose directory.

It looks like it's still pulling in JHipster Registry rather than Consul.

[+] Running 2/10
 ⠼ jhipster-registry Pulling                                                                                      10.5s
   ⠿ 7a9f619ee5e9 Pull complete                                                                                    7.3s
   ⠿ f1e9e85c7aa3 Pull complete                                                                                    8.0s
   ⠼ 8db301fee8d9 Downloading [=======================>                           ]  21.47MB/...                   8.5s
   ⠼ d551363a4323 Download complete                                                                                8.5s
   ⠼ 4ca92398d8dd Waiting                                                                                          8.5s
   ⠼ 18ab03e99e3f Downloading [===>                                               ]  123.7kB/...                   8.5s
   ⠼ eab0653ff666 Waiting                                                                                          8.5s
   ⠼ b7ba810be806 Waiting                                                                                          8.5s
   ⠼ f0a0176afa04 Waiting

If I look at docker-compose.yml, it contains the proper consul references for the applications, but jhipster-registry is listed as a container, not consul.

If I look in the kubernetes folder, the same issue exists. There's a registry-k8s directory, not a consul-k8s directory.

[deepu105]

Thank you @mraible , I might have overlooked the folder name and container name, i'll fix them

[mraible]

FWIW, I do see two things that are kinda concerning when I create apps with jhipster jdl reactive-mf.jdl and start them all with docker compose up:

docker-compose-gateway-postgresql-1    | 2023-02-19 23:39:14.825 UTC [80] FATAL:  role "root" does no

docker-compose-keycloak-1              | 2023-02-19 23:39:33,864 WARN  [org.keycloak.protocol.oidc.utils.AcrUtils] 
(executor-thread-0) Invalid realm configuration (ACR-LOA map)

[mraible]

I'm not sure why Jest tests are failing. Any ideas @deepu105?

[deepu105]

Thanks @mraible @mshima. so @mraible I guess only things left to fix are folder names and docker app names

[deepu105]

@mraible seems like you already fixed the container name issues (the kubernets folder is still;l registry-k8s, as registry is just generic enough to cover consul which is a registry as well IMO and avoids a bunch of templating everywhere), so I guess we can merge this, or do you have some outstanding concern?

[mraible]

I have no outstanding concerns, except for the failing tests.