Access data of Kibana/elasticsearch db

[ConsciousML]

My actions before raising this issue

• [X] Read/searched the docs
• [X] Searched past issues

I would like to know how to access/explore the data logged by Kibana to create an automatic payroll for my annotators

Sent a message on gitter with no response.

[nmanovic]

@ConsciousML , if you want to extract data from elasticsearch db, you need to read documentation for ELK. Will it be useful if we provide some basic statistics using our REST API? What is the usual way to transfer money to data annotators? By card? Crypto? Some service?

[ConsciousML]

Hi @nmanovic,

Thank you for the quick response.

It will be very useful for us to be able to access some statistics using the REST API.

I'm currently working on building an annotation pipeline using CVAT and we didn't start the annotation yet.

I was thinking of deploying a script on cloud that will run every week to fetch the number of annotation completed per annotators and compute the payroll to be sent to the HR department that will then take care of the money transfer.

I'm browsing the ELK documentation to understand how to interact with this ecosystem but I can't understand much for now.

I built a CDC (capture on change) process that replicates the data from the postgres database into our data lake.

I'm thinking of the following logic to achieve my purpose without digging into ELK (which will require a lot of work on my end):

• Save the stage change of each annotation task (annotation, validation, acceptance) -> already done.
• Save the state change of each annotation task (new, rejected, confirmed) -> already done.
• At the end of every week, query the annotation task that are in the acceptance stage with confirmed state
• Pull the annotation.xml (tracking data in our case) for each task, compute the number of bounding boxes
• Compute the pricing based on the number of bounding box and add the amount to the annotator, reviewer and validator payroll.

This approach can work but the problem is that I will not have the time spent by the workers on each task. Is there a simple way to fetch the data from the "list of task" dashboard in Kibana where the task_id, user and time spent is stored ?

Do you have any other suggestion that can help me solving this ?

Best, Axel

[bsekachev]

Closed as outdated