google-cloud-cpp icon indicating copy to clipboard operation
google-cloud-cpp copied to clipboard

[Flake] Pub/Sub samples timeout in test

Open coryan opened this issue 4 years ago • 4 comments

https://source.cloud.google.com/results/invocations/29229a70-3153-456e-8e9e-f55f127bafad

7/7 Test #207: pubsub_samples ...............................***Timeout 1500.10 sec

coryan avatar Mar 17 '21 16:03 coryan

I was able to repro with:

CXX=clang++ CC=clang bazel test -c opt --runs_per_test=1024 --jobs=32 --test_env=PUBSUB_EMULATOR_HOST=localhost:8085 --test_env=GOOGLE_CLOUD_PROJECT=coryan-test --test_env=GOOGLE_CLOUD_CPP_AUTO_RUN_EXAMPLES=yes --test_env=GOOGLE_CLOUD_CPP_EXPERIMENTAL_LOG_CONFIG=lastN,100,ERROR --test_env=GOOGLE_CLOUD_CPP_ENABLE_TRACING=rpc,rpc-streams //google/cloud/pubsub/samples:samples

about 5 runs out of 1,024 have a problem 😞

coryan avatar Mar 17 '21 17:03 coryan

Running SubscribeAvroRecords() sample

Running PublishAvroRecords() sample

Waiting for session [SubscribeAvroRecords]... TIMEOUT
Standard exception raised: session timeout
  • GCB: https://console.cloud.google.com/cloud-build/builds;region=us-east1/a47ece6e-0bea-4a9e-954d-982ae9853396;tab=detail?project=cloud-cpp-testing-resources
  • Raw: https://storage.googleapis.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/9430/3787eabd7ed84dfdea7eafc194b7946ab7bba8bc/fedora-35-cmake-gcs-rest/log-a47ece6e-0bea-4a9e-954d-982ae9853396.txt

dbolduc avatar Jul 06 '22 16:07 dbolduc

@dbolduc this does not strike me as the same problem at all. In the original report the samples would deadlock and be killed by the test driver after 1500s. In the latest report the sample code itself got some kind of timeout from the client library, and aborted in about 66s.

coryan avatar Jul 07 '22 16:07 coryan

It is not the same problem. I probably should have opened a separate issue. I only filed it here because the title matched.

dbolduc avatar Jul 07 '22 16:07 dbolduc

No repeats in 90d, and probably reopened in error. Closing.

coryan avatar Oct 10 '22 13:10 coryan