livekit-cli
livekit-cli copied to clipboard
livekit
Load tester: latency is not being reported anymore
For some reason, mentions of latency were removed from the code and the load-test is not reporting it anymore (despite the examples in the documentation).
This removal was also never mentioned in any of the changelogs I could find here on the github releases of this repository.
A release I could find that still has it is 0.6.0: https://github.com/livekit/livekit-cli/blob/7acc22982fc6cd26da529521820279fb5d3cb5c6/cmd/livekit-load-tester/main.go#L253
Any information on this?
It's unclear why latency reporting was removed, but we can look into restoring this functionality 👍
I think the previous method of reporting latency was a bit hacky (encoding publishing time in the payload). In order to do this correctly, we should look at the sender reports
I think the previous method of reporting latency was a bit hacky (encoding publishing time in the payload). In order to do this correctly, we should look at the sender reports
Hi @davidzhao , if you're not already working on this internally, could you point me to how was it done before and expand on how should it be done now? I could try to take a look and send a draft PR
@avivace We're exploring what accurate performance tracing would entail, but suffice it to say that it touches several components. Anything quick would likely be inaccurate (the previous metrics included local processing times and were a simple averaging of all tracks).
Can you tell us a bit more about your use case here? Are you checking coarse e2e latency just as a smoke test, or are you relying on it more concretely? What other metrics would you like to see in an idea case?
@avivace We're exploring what accurate performance tracing would entail, but suffice it to say that it touches several components. Anything quick would likely be inaccurate (the previous metrics included local processing times and were a simple averaging of all tracks).
Can you tell us a bit more about your use case here? Are you checking coarse e2e latency just as a smoke test, or are you relying on it more concretely? What other metrics would you like to see in an idea case?
@rektdeckard thanks a lot for looking into this. To be honest, I'd say that at the moment we want this metric for e.g. a measure of the infrastructure health/status, but we may also want to rely on it for specific setups in which we may want to take an action if latency becomes bigger than a threshold (e.g. unsubscribing/muting when people are on the same track but also physically in the same room)
Me too, I really need this feature when doing stress test to observe the system behavior
