Metrics

[mostafa]

Hey @riferrei,

Thanks for your awesome library. I'm using it in my project xk6-kafka, as you already know. I thought it would be a good idea to introduce some metrics into the library functions, so the usages, network traffic and a few different things can be measured. If I can find some time, I'll probably do that.

Also, I'd very happy to have your feedback on this. :pray:

[riferrei]

I'm a huge fan of metrics. Let's collaborate on that. How are you thinking about exporting then?

[mostafa]

Then let's do it!

In terms of metrics, xk6-kafka needs to eventually expprt the metrics to k6. The kafka-go package does a good job of exporting stats, which BTW I think would be the way to go, and I redefined and registered them in the k6 metrics registery. Then I exported them in the producer and consumer.

[AtakanColak]

Do you mean providing metrics for srclient, or metrics for schema registry via srclient?

[mostafa]

@AtakanColak I mean metrics for srclient. For example, a succeeded/failed dial to the endpoint, or the same when getting or creating a schema. You'll see what I meant if you look at the linked issue on my project.

Update: I just found out that Schema Registry also exposes some metrics, but I don't think it necessarily reflects what I want to do. Those metrics report the overall statistics of the SR instance, but I want the statistics of the timeframe I did load testing using xk6-kafka.

[AtakanColak]

I'm interested in doing this, if I do I'll mention @mostafa in the PR for it to be reviewed.

[mostafa]

@AtakanColak Would be very happy to review the PR.

[mostafa]

@AtakanColak Have you found any time to spend on this?

[AtakanColak]

@mostafa Sadly not, I had prioritised #86. I don't think I'll have the time for it in short term, but will work on it in the future if no one has done so.

[mostafa]

@AtakanColak No worries! I am also working on refactoring my extension and will see if I can do it after I've finished with the refactorings.

[AtakanColak]

@mostafa I've started this and I think below are the metrics we'll be able to provide, does that make sense to you?;

type SchemaRegistryClientMetrics struct {
	Timestamp                  time.Time
	CurrentSchemaCount         uint32
	DeletedSchemaCount         uint32
	CreatedAvroSchemaCount     uint32
	CreatedJSONSchemaCount     uint32
	CreatedProtobufSchemaCount uint32
	TotalHTTPRequestsCount     uint32
}

[mostafa]

@AtakanColak These are good as the baseline metrics. However, this is an HTTP client, and it would be nice to export more metrics, if possible, like the metrics k6 exports, which are available here. I also export many metrics from the writer and reader in xk6-kafka.

BTW, after releasing xk6-kafka v0.14.0, I am done refactoring the extension.

[AtakanColak]

If so then I'd like to leave this work to you.

[AtakanColak]

Dear @mostafa , are there any updates on this? If not, then I'm inclined to close this issue.

[AtakanColak]

Dear @mostafa , are there any updates on this? If not, then I'm inclined to close this issue.

[mostafa]

@AtakanColak

No updates from my side. Please close it. I'll re-open it when I have time to develop it.