grpc crash at LoadBalancedCall::PickSubchannelImpl
What version of gRPC and what language are you using?
grpc1.62.2 C++
What operating system (Linux, Windows,...) and version?
Windows 10
What runtime / compiler are you using (e.g. python version or version of gcc)
msvc visual studio 2022
What did you do?
It's a crash reported by our users, I'm not able to find out what exactly he did when the crash happens, and I have not found a way to reproduce it.
What did you expect to see?
grpc works correctly
What did you see instead?
by debugging the dmp file I got, this is the calling stack
Anything else we should know about your project / environment?
Sorry for the limit of info I provided, I am not able discover more currently. I just try to seek for some possibility if you team can find any clue about this crash, if not, that's ok. Within a month after we release our newest application version with updating grpc to 1.62.2, three users reported crash on this calling stack.
I looked through the code and did not see anything suspicious. There is little we can do with the limited info provided. We have not seen other reports even though the code in question have not changed a lot in the last several releases and is covered with tests on supported platforms.
Please try to get more information from your clients. See this guide - https://github.com/grpc/grpc/blob/v1.62.x/TROUBLESHOOTING.md. Please note that you need to follow the guide from 1.62 branch as there were significant changes in logging since 1.62. This reference provides more details on setting up tracing: https://github.com/grpc/grpc/blob/v1.62.x/doc/environment_variables.md
I looked through the code and did not see anything suspicious. There is little we can do with the limited info provided. We have not seen other reports even though the code in question have not changed a lot in the last several releases and is covered with tests on supported platforms.
Please try to get more information from your clients. See this guide - https://github.com/grpc/grpc/blob/v1.62.x/TROUBLESHOOTING.md. Please note that you need to follow the guide from 1.62 branch as there were significant changes in logging since 1.62. This reference provides more details on setting up tracing: https://github.com/grpc/grpc/blob/v1.62.x/doc/environment_variables.md
OK,thank you very much. I'll follow the guide and have a try. And if I get more information, I'll update.
not sure if this info helps but seemingly the same crash happens on firebase-ios-sdk as well. https://github.com/firebase/firebase-ios-sdk/issues/12968
@Kaz-flex is this failing on Mac or iOS? Is there any chance to use a build with debug symbols or tracing?
It's happening on iOS, Unfortunately I did not reproduce it manually but we observed it through crash tracking service. You can find stack trace of all the threads here if that's something that can help. https://github.com/firebase/firebase-ios-sdk/issues/13459
@eugeneo
I'm trying to release our app with GRPC_VERBOSITY=debug but not sure what to set for GRPC_TRACE because all would be just too many logs.
Do you have any suggestions on what to set for GRPC_TRACE and GRPC_VERBOSITY?
My recommendation would be -
GRPC_TRACE=call,channel,client_channel,client_channel_call,client_channel_lb_call,pick_first
GRPC_VERBOSITY=DEBUG (or initialize absl logging and set verbosity that way.)
More than 30 days have passed since label "disposition/requires reporter action" was added. Closing this issue. Please feel free to re-open/create a new issue if this is still relevant.