Can't generate histograms on the assertion_bucket error when running example

[coco98]

@0x777 Following the example directoy:

1. I ran the graphql server and tested it, all seemed ok
2. I ran the graphql benchmark suite and got this output:

====================
benchmark: query-comparison
  --------------------
  candidate: HeroNameQuery on hero_name at http://172.17.0.1:5000/graphql
    Warmup:
      ++++++++++++++++++++
      100Req/s Duration:60s open connections:20
      Running 1m test @ http://172.17.0.1:5000/graphql
        4 threads and 20 connections
        Thread calibration: mean lat.: 24.057ms, rate sampling interval: 102ms
        Thread calibration: mean lat.: 37.867ms, rate sampling interval: 101ms
        Thread calibration: mean lat.: 40.488ms, rate sampling interval: 106ms
        Thread calibration: mean lat.: 36.296ms, rate sampling interval: 96ms
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency    35.98ms   16.65ms 133.76ms   71.89%
          Req/Sec    24.77     21.41    52.00     41.66%
        5980 requests in 1.00m, 1.03MB read
        Socket errors: connect 0, read 0, write 0, timeout 1
      Requests/sec:     99.66
      Transfer/sec:     17.52KB
      ++++++++++++++++++++
      200Req/s Duration:60s open connections:20
      Running 1m test @ http://172.17.0.1:5000/graphql
        4 threads and 20 connections
        Thread calibration: mean lat.: 38.743ms, rate sampling interval: 121ms
        Thread calibration: mean lat.: 45.739ms, rate sampling interval: 123ms
        Thread calibration: mean lat.: 47.466ms, rate sampling interval: 124ms
        Thread calibration: mean lat.: 44.317ms, rate sampling interval: 110ms
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency    42.98ms   23.42ms 962.05ms   87.69%
          Req/Sec    49.72     12.15   120.00     77.33%
        12004 requests in 1.00m, 2.06MB read
      Requests/sec:    200.05
      Transfer/sec:     35.17KB
    Benchmark:
      ++++++++++++++++++++
      100Req/s Duration:300s open connections:20
      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.
      ++++++++++++++++++++
      200Req/s Duration:300s open connections:20
      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.
  --------------------
  candidate: HeroNameFriendsQuery on hero_name_friends at http://172.17.0.1:5000/graphql
    Warmup:
      ++++++++++++++++++++
      100Req/s Duration:60s open connections:20
      Running 1m test @ http://172.17.0.1:5000/graphql
        4 threads and 20 connections
        Thread calibration: mean lat.: 44.057ms, rate sampling interval: 125ms
        Thread calibration: mean lat.: 48.205ms, rate sampling interval: 123ms
        Thread calibration: mean lat.: 49.266ms, rate sampling interval: 121ms
        Thread calibration: mean lat.: 46.410ms, rate sampling interval: 112ms
        Thread Stats   Avg      Stdev     Max   +/- Stdev
          Latency    46.20ms   14.88ms 105.92ms   77.60%
          Req/Sec    24.81     17.46    45.00     63.81%
        6001 requests in 1.00m, 1.50MB read
        Non-2xx or 3xx responses: 1
      Requests/sec:     99.96
      Transfer/sec:     25.58KB
      ++++++++++++++++++++
      200Req/s Duration:60s open connections:20
      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.
    Benchmark:
      ++++++++++++++++++++
      100Req/s Duration:300s open connections:20
      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.
      ++++++++++++++++++++
      200Req/s Duration:300s open connections:20
      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.
 * Serving Flask app "bench" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:8050/ (Press CTRL+C to quit)

3. When I head to localhost:8050 I don't see any histograms:

This seems to be the culprit from the logs:

      wrk2: src/hdr_histogram.c:54: counts_index: Assertion `bucket_index < h->bucket_count' failed.

[coco98]

@0x777 I ran this again and got the same error, but one data point came through. How should we proceed? Can you describe the source of the error so that I can get someone from the team to follow up with wrk2?

[0x777]

This looks like an assertion failure in wrk2. I've encountered this once but couldn't reproduce it. We should raise an issue with wrk2 (https://github.com/giltene/wrk2/issues).

[jmatsushita]

I ran into the same problem. This issue in wrk2 was raised https://github.com/hasura/graphql-bench/issues/2 and hinted at a clock synchronization problem. I used the trick described here https://github.com/docker/for-mac/issues/2979 to disable the OSX VM's ntp service docker run -it --privileged --pid=host justincormack/nsenter1 /bin/sh -c 'kill -9 "$(pidof sntpc)"';.

After doing this the problem disappears. I expect that this won't survive an osx restart.

Hope this helps