Access the data from ZeeQS's GraphQL API

[saig0]

Currently, the data are imported from Hazelcast, aggregated, and stored in a database. The frontend is rendered on the server and access the database.

ZeeQS is collecting the data in a similar way but providing them by a GraphQL API. We can replace the data part here and just use the GraphQL API instead.

This also allows us to build a better UI that doesn't need to be rendered on the server. The UI could be more dynamic and access the GraphQL API directly in the client.

[aivinog1]

@saig0 Hi! I think I can take this. But I have a question: should we maintain both (DB and GraphQL) sources of the data in some intermediate period to ensure backward compatibility?

[saig0]

Go for it :rocket: But be aware that this is a bigger change. We can replace the whole DB storage by querying the GraphQL API of ZeeQS. We don't need to support both options.

It may make sense to replace the UI at the same time. Otherwise, the backend needs to read from GraphQL and the UI is updated later.

I wrote some code in another project to read the GraphQL API. Feel free to have a look: https://github.com/zeebe-io/bpmn-spec/blob/master/zeebe-test-runner/src/main/kotlin/io/zeebe/bpmnspec/runner/zeebe/zeeqs/ZeeqsClient.kt

[aivinog1]

@saig0 thanks! I'm thinking about iterative solving this issue. First of all, I would like to replace calls to DB, after that, we would discuss UI (frameworks, architecture, etc.). If you okay with this, I will create two issues that decompose this problem :)

[psteinroe]

What is the progress on this? I could also contribute during the summer.

[aivinog1]

Hi @steinroe! Sorry for the late response :) I was a little busy and actually forget about this 😅. If you want to work on it, you can take this issue

[saig0]

@steinroe nice :tada:

I also plan to work on this in a few months (or at least in August).

My plan is to create a new project that contains a new frontend and a (stateless) backend for sending commands to Zeebe. The frontend will query all data from ZeeQS.

Why a new project?

• a major change in the architecture (e.g. the data storage will be replaced by ZeeQS)
• not much to reuse (almost a complete rewrite) - the frontend will fetch the data instead of server-side rendering
• better focus on helping users while developing and debugging processes (instead of being a tool for monitoring production environments)

[psteinroe]

Yes, that was also my train of thought - unfortunately, there is not much to reuse here considering the new architecture, so a new repo would make sense!

Although our motivation is to also use such a tool to monitor production environments, I think we can collaborate and create something useful for most use-cases. Lets use this thread to keep us up-to-date on our respective schedule. 🎉

[danshapir]

@saig0 any chance of this happening? Getting the data from ZeeQS means the data can be fetched from Elastic and not from Hazelcast + Postgres, this would be a huge improvement.

[saig0]

Getting the data from ZeeQS means the data can be fetched from Elastic and not from Hazelcast + Postgres

Currently, this is not possible in ZeeQS. It consumes the records also from Hazelcast :sweat_smile: Or, did I misunderstand the question?

However, my new project zeebe-play is almost ready. I need to write documentation and move it to the community hub. It is basically ZeeQS + UI. But it focuses on playing with one process (i.e. seeing how it is executed) rather than monitoring process instances.

[danshapir]

@saig0 ohh on the zeeqs docs it shows elastic as an exporter source.

Regarding Zeebe Play that sounds super interesting! What do you mean just one process? How do you browse it?

[saig0]

ohh on the zeeqs docs it shows elastic as an exporter source.

Yeah. This part is not implemented yet :see_no_evil:

Regarding Zeebe Play that sounds super interesting! What do you mean just one process? How do you browse it?

You will feel familiar with using the simple monitor :sweat_smile: but comes with more PLAY :video_game: :smile:

Stay tuned :rocket: I try to publish it in September :building_construction:

[saig0]

I'm closing this issue in favor of https://github.com/camunda-community-hub/zeebe-play. It solves exactly the requirement of showing the data from ZeeQS.