google-cloud-cpp icon indicating copy to clipboard operation
google-cloud-cpp copied to clipboard
verified publisher icon googleapis

Flake DecompressiveTranscodingIntegrationTest.WriteAndReadCompressedJson retry exhausted

Open scotthart opened this issue 1 year ago • 0 comments

https://storage.googleapis.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/main/88d5094feed3c301579055c71ba16e9d6f21cff9/fedora-latest-bazel-coverage-default/log-ae508921-4547-4265-9abd-b1d153132734.txt

[ RUN      ] DecompressiveTranscodingIntegrationTest.WriteAndReadCompressedJson
google/cloud/storage/tests/decompressive_transcoding_integration_test.cc:107: Failure
Value of: insert
Expected: code is equal to OK and message is anything
  Actual: 928-byte object <60-51 C7-01 00-00 00-00 00-AD 29-36 10-7F 00-00 80-9F AD-58 FC-7F 00-00 10-20 00-00 00-00 00-00 80-9F AD-58 FC-7F 00-00 C0-BA 3D-36 10-7F 00-00 10-20 00-00 00-00 00-00 B0-F8 FF-FF FF-FF FF-FF ... 08-00 00-00 10-7F 00-00 20-A2 AD-58 FC-7F 00-00 00-3F 67-36 10-7F 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 F0-A0 AD-58 FC-7F 00-00 20-39 67-36 10-7F 00-00 B0-38 67-36 10-7F 00-00>, whose status is DEADLINE_EXCEEDED: Retry policy exhausted, with a last message of timeout [3s] while waiting for Close() error_info={reason=DEADLINE_EXCEEDED, domain=gcloud-cpp, metadata={gcloud-cpp.retry.on-entry=false, gcloud-cpp.retry.function=InsertObjectMedia, gcloud-cpp.source.function=TimeoutError, gcloud-cpp.source.line=121, gcloud-cpp.source.filename=google/cloud/storage/internal/grpc/stub.cc, gcloud-cpp.retry.reason=retry-policy-exhausted, gcloud-cpp.retry.original-message=timeout [3s] while waiting for Close(), gcloud-cpp.version=v2.27.0-rc}}, with a code that isn't equal to OK, but a message that is anything

2024-07-23T14:49:21.524026024Z [INFO] <139707604016640> Enabling logging for raw-client (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.524066719Z [INFO] <139707604016640> Enabling logging for rpc (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.524075012Z [INFO] <139707604016640> Enabling logging for rpc-streams (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.524761456Z [INFO] <139707604016640> Enabling logging for raw-client (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.524779687Z [INFO] <139707604016640> Enabling logging for rpc (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.524786217Z [INFO] <139707604016640> Enabling logging for rpc-streams (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:21.591627635Z [INFO] <139707604016640> Enabled logging for gRPC calls (google/cloud/storage/internal/storage_stub_factory.cc:91)
2024-07-23T14:49:21.591755124Z [INFO] <139707604016640> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
2024-07-23T14:49:21.598533452Z [INFO] <139707578459840> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
2024-07-23T14:49:21.600632718Z [INFO] <139707578459840> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
2024-07-23T14:49:21.669618256Z [DEBUG] <139707604016640> WriteObject(1) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:21.773004529Z [DEBUG] <139707604016640> Write(1)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:21.775071576Z [DEBUG] <139707604016640> Write(1)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:21.775958925Z [DEBUG] <139707604016640> Close(1)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:24.778622740Z [DEBUG] <139707578459840> Cancel(1)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:24.780225950Z [DEBUG] <139707604016640> Close(1)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:24.780520658Z [DEBUG] <139707578459840> Cancel(1)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:24.800089526Z [DEBUG] <139707604016640> WriteObject(2) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:24.802416051Z [DEBUG] <139707604016640> Write(2)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:24.803112996Z [DEBUG] <139707604016640> Write(2)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:24.803989069Z [DEBUG] <139707604016640> Close(2)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:27.806144628Z [DEBUG] <139707578459840> Cancel(2)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:27.807482751Z [DEBUG] <139707578459840> Cancel(2)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:27.807785533Z [DEBUG] <139707604016640> Close(2)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:27.829024669Z [DEBUG] <139707604016640> WriteObject(3) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:27.831496904Z [DEBUG] <139707604016640> Write(3)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:27.832215499Z [DEBUG] <139707604016640> Write(3)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:27.833025303Z [DEBUG] <139707604016640> Close(3)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:30.835764565Z [DEBUG] <139707578459840> Cancel(3)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:30.836855712Z [DEBUG] <139707604016640> Close(3)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:30.837159539Z [DEBUG] <139707578459840> Cancel(3)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:30.892531008Z [DEBUG] <139707604016640> WriteObject(4) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:30.895037515Z [DEBUG] <139707604016640> Write(4)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:30.895758594Z [DEBUG] <139707604016640> Write(4)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:30.896472903Z [DEBUG] <139707604016640> Close(4)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:33.899648609Z [DEBUG] <139707578459840> Cancel(4)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:33.900622914Z [DEBUG] <139707604016640> Close(4)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:33.900848060Z [DEBUG] <139707578459840> Cancel(4)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:33.997188709Z [DEBUG] <139707604016640> WriteObject(5) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:33.999538721Z [DEBUG] <139707604016640> Write(5)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:34.000209878Z [DEBUG] <139707604016640> Write(5)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:34.001021345Z [DEBUG] <139707604016640> Close(5)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:37.003672956Z [DEBUG] <139707578459840> Cancel(5)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:37.004767711Z [DEBUG] <139707604016640> Close(5)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:37.005015305Z [DEBUG] <139707578459840> Cancel(5)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:37.228534564Z [DEBUG] <139707604016640> WriteObject(6) (google/cloud/storage/internal/storage_logging_decorator.cc:296)
2024-07-23T14:49:37.230826758Z [DEBUG] <139707604016640> Write(6)() << google.storage.v2.WriteObjectRequest { write_object_spec { resource { name: "ob-4EghsE3nTfStI8PxB4K6XJ9KNofucZpOkpGywderXStBZHUibtce9tzSaoilHuoJJVcgHJeGDBr9EaQpB2Eu5XeDEewbDekG6QwjgKvs7MX3R8GTbVTXisXXYpDzn...<truncated>..." bucket: "projects/_/buckets/gcs-grpc-team-cloud-cpp-testing-bucket" content_encoding: "gzip" content_type: "text/plain" } if_generation_match: 0 } checksummed_data { content: "\037\213\010\000\000\000\000\000\000\003\234\335\313\0254={\226\3219Q|!\274\317Q\022q\220\000`s\206\037\014\006C\364T/\210`\367\240\'\275\356\3315\332]%\375\371\363\377>\361\317\377\372\027\377\356\357\377\372o\377\370\357\377\365\177\374\353_\375\303\337\376\327\177\371\353\337\374\355\237\376\372\017\377\370\237\377\353\177\377\353o\377\363\357\377\341\257\377\361\375\374\237\376\345\377\371\337\177\375\335\337\376\355?\373\377\253?...<truncated>..." crc32c: 2202592185 } object_checksums { crc32c: 2202592185 } finish_write: true } (./google/cloud/internal/streaming_write_rpc_logging.h:56)
2024-07-23T14:49:37.231474087Z [DEBUG] <139707604016640> Write(6)() >> true (./google/cloud/internal/streaming_write_rpc_logging.h:59)
2024-07-23T14:49:37.232184056Z [DEBUG] <139707604016640> Close(6)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:65)
2024-07-23T14:49:40.234490486Z [DEBUG] <139707578459840> Cancel(6)() << (void) (./google/cloud/internal/streaming_write_rpc_logging.h:49)
2024-07-23T14:49:40.235436087Z [DEBUG] <139707604016640> Close(6)() >> CANCELLED: CANCELLED (./google/cloud/internal/streaming_write_rpc_logging.h:71)
2024-07-23T14:49:40.235708497Z [DEBUG] <139707578459840> Cancel(6)() >> (void) (./google/cloud/internal/streaming_write_rpc_logging.h:51)
2024-07-23T14:49:40.267903231Z [INFO] <139707604016640> Enabling logging for raw-client (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.267930565Z [INFO] <139707604016640> Enabling logging for rpc (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.267936035Z [INFO] <139707604016640> Enabling logging for rpc-streams (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.268309712Z [INFO] <139707604016640> Enabling logging for raw-client (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.268321309Z [INFO] <139707604016640> Enabling logging for rpc (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.268325875Z [INFO] <139707604016640> Enabling logging for rpc-streams (google/cloud/storage/client_options.cc:238)
2024-07-23T14:49:40.287754359Z [INFO] <139707604016640> Enabled logging for gRPC calls (google/cloud/storage/internal/storage_stub_factory.cc:91)
2024-07-23T14:49:40.287812090Z [INFO] <139707604016640> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
2024-07-23T14:49:40.291017404Z [INFO] <139706108974784> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
2024-07-23T14:49:40.292889202Z [INFO] <139706108974784> Refreshing channel [0] (google/cloud/storage/internal/grpc/channel_refresh.cc:50)
[  FAILED  ] DecompressiveTranscodingIntegrationTest.WriteAndReadCompressedJson (18836 ms)

scotthart avatar Jul 23 '24 15:07 scotthart