jetty-load-generator icon indicating copy to clipboard operation
jetty-load-generator copied to clipboard
verified publisher icon jetty-project

How do I visualize the histogram?

Open swaranga-netflix opened this issue 3 years ago • 8 comments

Running a load test from the commandline generates a report like:

{
  "report": {
    "beginInstant": "2023-03-17T02:19:35.460735978Z",
    "completeInstant": "2023-03-17T02:20:35.723580110Z",
    "recordingDuration": 59927,
    "availableProcessors": 4,
    "averageCPUPercent": 18.12186327810192,
    "requestRate": 99.99645797985681,
    "responseRate": 99.70362162675781,
    "sentBytesRate": 19841.020703724804,
    "receivedBytesRate": 564521.9056507027,
    "failures": 0,
    "1xx": 0,
    "2xx": 5975,
    "3xx": 0,
    "4xx": 0,
    "5xx": 0,
    "histogram": "1679019575.307,192.168.1.1,HISTFAAAAup42kVTzY7kNBD2VqprPF6P1+M2GctEWa83m4mi0GpaCDggYA8IcUZIcOIBVhw4ckW8Ac+zD8FbcETiwpHP6UXkz66qrz47rq+G334/KsU/q+vVvRv+xOfun6+/V1/8dfW8/fUJ/UKKDnRP+NxRB4PoBm+bqOvNHTdI1yCwRbWpIiZmxc0E5HZPEWbqmp9Y2BoyLFqENByGuROB6Yg0HhGjNZzCYgEgK1ZbLbaN2ihtDCDakm0za4zVLMZ4yzZ72NqIMyY744y1LuiovbfWJBPJRpPEpdD72HvX9yH12VkbbaohjsGGPg6pjzml3oUx+BxjDSnkGgIAQ9/XMXlfy5JqTsO8rGWo8zQP/bDWYY1zfxq3MmU31iWVskzLVOs6n8u81XG7XOb80Wlcz+sGx3retvP22TZP45e1XJa61vOlXMZ0ms7nqZTTPF/On6+n5eNPSy5pKYu7bOWU57VWpM+XU4hLmoZxLHOZwmkMQ/IVW/UlpSHkMPQxzANG/Gu0Procs8dB+pzH4GKIzjudfbLei9OImOI9OwnBOG9D8FbasRkces9uBwX22mgyxvS6Vc8YRkWsRBdEO20cozIhMIqqPYoSGABplwZAiETjYtTQEIgY1TVevNZKjOWWpowoJEIvjR7pgAOrnBY24CEBhJs0VNuG1tRkBm6AiPEoA0cTom5qhdKwLFOTmWrSsx0EylAxsAjvEuamV0U30nS6yxwpjU+6Xeu7dOUq/ybQHU9QcpM3XZMRbW5WVxSrRt3EDsJ3HcJg+98WwW7QD4q14sPeZ3S791Trl4d9G//R7oG9xfaOA8MBKc/o+j7sS95iTkf06jNMj/QCv3hHH2D+DX/XuvaRv0VIwUEHfiB6heB7gLxPT+E/4n5Jz3euDugr8107jXsMz/menvKH9BXA9wj/TYjfYJHXMBX9QO1Ub+kThAGmR+Qe6I9D+49XSHr7BMDXWO5H+qllvgDq2Lix5ZeYPYLhDf0LamtC/A==\n"
  },
  "config": {
    "threads": 25,
    "warmupIterationsPerThread": 0,
    "iterationsPerThread": 1,
    "runFor": 60,
    "usersPerThread": 1,
    "channelsPerUser": 128,
    "resourceRate": 100,
    "rateRampUpPeriod": 0,
    "scheme": "http",
    "host": "metacat.dyntest.netflix.net",
    "port": 7001,
    "transport": {
      "type": "http/1.1",
      "selectors": 1
    },
    "resource": {
      "method": "GET",
      "path": "...",
      "requestLength": 0,
      "responseLength": 0,
      "requestHeaders": {},
      "resources": []
    },
    "maxRequestsQueued": 1024,
    "connectBlocking": true,
    "connectTimeout": 5000,
    "idleTimeout": 15000
  }
}

How do I visualize the histogram data?

swaranga-netflix avatar Mar 17 '23 02:03 swaranga-netflix

Can you explain how you are running via commandline?

olamy avatar Mar 17 '23 03:03 olamy

Running it via the uber jar:

java -jar jetty-load-generator-starter-3.1.4-uber.jar \
  --host host --port port \
  --stats-file /tmp/stats.log \
  --display-stats true \
  --resource-json-path resource.json \
  -rt 1 -rtu m -t 25 -rr 100

swaranga-netflix avatar Mar 17 '23 04:03 swaranga-netflix

uhm. actually I guess you get something such

SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.

because the uber jar doesn't include any slf4j impl whereas we use slf4j as logging api and so to display the histogram... As a workaround you can download manually slf4-simple from here https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/2.0.6/slf4j-simple-2.0.6.jar And start the uber jar as is (note you cannot use the option -jar as it is not possible to use more than one jar with this option) :

java -cp slf4j-simple-2.0.6.jar:jetty-load-generator-starter-3.1.4-uber.jar org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter \
  --host host --port port \
  --stats-file /tmp/stats.log \
  --display-stats true \
  --resource-json-path resource.json \
  -rt 1 -rtu m -t 25 -rr 100

The output should be something such:

[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - load generator config: Config[t=2,i=-1,u=1,c=128,r=2,rf=10s,https://host:port]
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - load generation begin
[main] WARN org.eclipse.jetty.util.ssl.SslContextFactory.config - Trusting all certificates configured for Client@44a3ec6b[provider=null,keyStore=null,trustStore=null]
[main] WARN org.eclipse.jetty.util.ssl.SslContextFactory.config - No Client EndPointIdentificationAlgorithm configured for Client@44a3ec6b[provider=null,keyStore=null,trustStore=null]
[LoadGenerator@15bfd87-sender-1] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - load generation complete
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - 
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - ----------------------------------------------------
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - -------------  Load Generator Report  --------------
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - ----------------------------------------------------
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - https://host:443 over http/1.1
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - resource tree     : 1 resource(s)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - begin date time   : 2023-03-17 16:15:47 AEST
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - complete date time: 2023-03-17 16:16:00 AEST
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - recording time    : 11.311 s
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - average cpu load  : 20.678/1600
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - 
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - histogram:
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  143 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  286 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  429 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  572 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  716 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  859 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -       @               _  1,002 ms (2, 11.11%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -                    @  _  1,145 ms (7, 38.89%) ^50%
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -       @               _  1,289 ms (2, 11.11%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  1,432 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  1,575 ms (1, 5.56%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  1,718 ms (1, 5.56%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  1,862 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  2,005 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  2,148 ms (1, 5.56%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  2,291 ms (1, 5.56%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  2,435 ms (1, 5.56%) ^85%
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - @                     _  2,578 ms (0, 0.00%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  2,721 ms (1, 5.56%)
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter -    @                  _  2,864 ms (1, 5.56%) ^95% ^99% ^99.9%
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response times: 18 samples | min/avg/50th%/99th%/max = 924/1,511/1,145/2,864/2,864 ms
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - 
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - nominal resource rate (resources/s): 2.000
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - nominal request rate (requests/s)  : 2.000
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - request rate (requests/s)          : 1.999
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response rate (responses/s)        : 1.591
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - send rate (bytes/s)                : 2716.275
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - receive rate (bytes/s)             : 379568.426
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - failures          : 0
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response 1xx group: 0
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response 2xx group: 18
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response 3xx group: 0
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response 4xx group: 0
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - response 5xx group: 0
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - ----------------------------------------------------
[main] INFO org.mortbay.jetty.load.generator.starter.LoadGeneratorStarter - load generator report saved to: /tmp/stats.log

you can probably customise the output by providing a file simplelogger.properties via -cp option see documentation here https://www.slf4j.org/api/org/slf4j/simple/SimpleLogger.html

@sbordet I guess we should add one slf4j impl in the uber jar to make easy to use. WDYT? Which one?

olamy avatar Mar 17 '23 06:03 olamy

I guess we should add one slf4j impl in the uber jar to make easy to use.

I'm not sure... If we add one, then those that want to use a different one would not be able to, right?

sbordet avatar Mar 21 '23 17:03 sbordet

I guess we should add one slf4j impl in the uber jar to make easy to use.

I'm not sure... If we add one, then those that want to use a different one would not be able to, right?

Oh yes definitely that will be the problem.. 😀 I will at least change the README to explain how to have logging

olamy avatar Mar 21 '23 20:03 olamy

@swaranga-netflix btw did my comment answer your question?

olamy avatar Mar 21 '23 20:03 olamy

I will let you take the final call, but as a recent user, I expected the Uber jar to be batteries included and that included the logging stuff. I would not want to have an additional logger jar be a requirement. I think you can offer a system property that points to a logging properties file if I want to customize the format but otherwise taking a stance on the logging implementation and a default format seems pretty reasonable to me.

swaranga-netflix avatar Mar 21 '23 20:03 swaranga-netflix

@olamy - yes it did. Thank you.

swaranga-netflix avatar Mar 21 '23 20:03 swaranga-netflix