gloo
gloo copied to clipboard
CI Test Flake: [AfterEach] Clustered redis backed rate limiting
Gloo Edge Version
1.13.x (beta)
Kubernetes Version
No response
Describe the bug
Flakes consistently in aftereach
https://00f74ba44b01bed185da38eada35ce74c725f20f82-apidata.googleusercontent.com/download/storage/v1/b/825641009090.cloudbuild-logs.googleusercontent.com/o/log-26eb0502-4374-4330-ad6d-a5f684d7a2c8.txt?jk=AFshE3XnEJnoHcmgsybo-zqHuEwUyD2sU9VFoSFTZ66oNzGXaKazs2c0n6OM_JGN1g7tuELwWCtxGcccVnvZctG4ai2YyUnf2ips33QyVOuRIkTe9XZU5TrxQddAzRWK_nJ7KvsAWbzwGRKODq-IWkFWrZP6YexeYeMghvJ8owzMkadX40gzyVTR9ntzitIitg_eWPu6pgzNxKJrLon19tAWRHxfQjFkUFUIo6zbd4C4BKpXeoMTjFF0Yy0JqnK_CIarrNRp3nMds-v1cjocqwn_IAQqoPo_uwaZ5ADuEoonOuhwOG1cMd-CR7ePOthDqDGM-ZR9Ux6DdFoPo7tEdQTBXzX2oa1iolbeSbmP4_C02hlAgkTg9ieqHoAvfK0b2SQQ35Fdrz5rmI-H3eFFQ70GsyhzMqf711Cry8M99FrnBAWLpd_fFXxRt1wxK9Ki80xgn-gTBCS1486eHKYZkcX7lT9EyKl0Pba--Q0jUHqFWXQ8sk4K8au1V5J5MejJzuOcbqm6VdGs62q_H0ywCRDdjD5ithaVfGblXjCOYawgwBJG0NT0pcKL3k_u6dmC6a2PvbFXTViZ-cFThRe3Pi1A3chknP-yOeA_52jAprKO4Qa0dQN-EbLZpAnU-gtEC27ah395IqhpaYMv7oqKZtaDRW7dizzUWtnxFGx8NvZS1GAgSU3IV7kzMXrq3moDEn-wwvE_bE_YQPqpF-UVjpg1xmLHo8XE1HKBdWhAgP8KQS4vWXsqK2G_IHktY4cGC9MXGazmzyPLrEwa0T4CP_as6mcFDkMzqM8x8AVsnQc54aLSRhf9huaBzyr6BAaf1HOw9fgmx2K83ShsmSE4iuu9DFgHOKLG1_5YdJ9oPQXsosczpf35yOMNLcl1aoLrhiY5Pj50TacmCC0nHCEnKoLhYXccx-n9FX8_mgz6i3QdZmFjWyr3Ca00BBt-scOw7mnxrYAa1pHUZVO0yKgeE2ONeX6gnRJ2woxho8EG8mHNVkLZdknr_8Ol9ittYvCUyWKVDo5kk5BJN53pM2Peb-LhspxP4nLkCdsGgwnVCCZ53ISlqsppRtQRKyNL1Do94gkm&isca=1
Steps to reproduce the bug
flakes
Expected Behavior
doesnt flake
Additional Context
No response
Step #12 - "test": [2022-09-16 18:57:29.569][86309][critical][assert] [external/envoy/source/extensions/filters/common/ratelimit/ratelimit_impl.cc:77] assert failure: response->overall_code() != envoy::service::ratelimit::v3::RateLimitResponse::UNKNOWN.
This line is the culprit. @Bslabe123 and I thought this was fixed by changing the rate-limiter to return nil instead of envoy::service::ratelimit::v3::RateLimitResponse::UNKNOWN in this recover
The fix was released in 0.7.4 of the rate-limiter, and 1.12 (where this flake occurred) is on 0.7.3
@inFocus7 I believe your recent PRs in solo-projects resolved this flake. Can you confirm and if so, close the issue?