Further investigation on 32 bit CI build thread behavior in tests

[scotthart]

The multithreaded tests for query plan were failing using 1000 threads in 32 bit mode, but passing in 64 bit mode. Attempts to simplify the tests to debug, caused the failures to occur in other places in these tests. Reducing the threads from 1000 to 500 made the tests pass in 32 bit mode. For now, we'll use the higher number of threads in 64 bit tests, but cap them at 500 for 32 bit.

[scotthart]

500 is still too high as terminate called without an active exception is still encountered, but not every time. Will try lowering to 200.

Built in debug mode, during thread creation, the test would get to around thread 501 or 502 before failing.

Thread 1 "internal_query_" hit Breakpoint 1, std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:57
57      {
(gdb) bt
#0  std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:57
#1  0x08062629 in std::__terminate () at /usr/include/c++/14/x86_64-redhat-linux/32/bits/c++config.h:324
#2  std::thread::~thread (this=0x9fcdba0, __in_chrg=<optimized out>) at /usr/include/c++/14/bits/std_thread.h:182
#3  0x0806ec75 in std::_Destroy<std::thread> (__pointer=0x9fcdba0) at /usr/include/c++/14/bits/stl_construct.h:151
#4  0x0806d318 in std::_Destroy_aux<false>::__destroy<std::thread*> (__first=0x9fcdba0, __last=0x9fce378)
    at /usr/include/c++/14/bits/stl_construct.h:163
#5  0x0806b604 in std::_Destroy<std::thread*> (__first=0x9fcdba0, __last=0x9fce378) at /usr/include/c++/14/bits/stl_construct.h:196
#6  0x08067133 in std::_Destroy<std::thread*, std::thread> (__first=0x9fcdba0, __last=0x9fce378)
    at /usr/include/c++/14/bits/alloc_traits.h:944
#7  std::vector<std::thread, std::allocator<std::thread> >::~vector (this=0xffeb8374, __in_chrg=<optimized out>)
    at /usr/include/c++/14/bits/stl_vector.h:735
#8  0x08059bfd in google::cloud::bigtable_internal::v2_44::(anonymous namespace)::QueryPlanMultithreadedTest_RefreshInvalidatedPlan_Test::TestBody (this=0x9f85d80) at /workspace/google/cloud/bigtable/internal/query_plan_test.cc:271
#9  0xf6ac3a34 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (location=0xf6ac9b03 "the test body", 
    method=<optimized out>, object=0x9f85d80) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2612
#10 testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [clone .constprop.0] (object=object@entry=0x9f85d80, method=<optimized out>, location=0xf6ac9b03 "the test body")
    at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2648
#11 0xf6aae786 in testing::Test::Run (this=0x9f85d80) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2687
#12 testing::Test::Run (this=0x9f85d80) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2677
#13 0xf6aae9aa in testing::TestInfo::Run (this=<optimized out>) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2836
#14 0xf6aaec07 in testing::TestSuite::Run (this=0x9f725a0) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:3015
#15 testing::TestSuite::Run (this=0x9f725a0) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2968
#16 0xf6abb48b in testing::internal::UnitTestImpl::RunAllTests (this=0x9f72230)
    at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:5920
#17 0xf6ab9f6b in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (
    location=0xf6acd038 "auxiliary test code (environments or event listeners)", 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0xf6abb0a0 <testing::internal::UnitTestImpl::RunAllTests()>, object=0x9f72230) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2601
#18 testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (
    location=0xf6acd038 "auxiliary test code (environments or event listeners)", 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0xf6abb0a0 <testing::internal::UnitTestImpl::RunAllTests()>, object=0x9f72230) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:2648
#19 testing::UnitTest::Run (this=0xf6ae4160 <testing::UnitTest::GetInstance()::instance>)
    at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/src/gtest.cc:5484
#20 0xf7f150d2 in RUN_ALL_TESTS () at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googletest/include/gtest/gtest.h:2317
#21 main (argc=<optimized out>, argv=0xffeb8984) at /usr/src/debug/gtest-1.14.0-4.fc40.i386/googlemock/src/gmock_main.cc:71
#22 0xf64a8bc3 in __libc_start_call_main (main=main@entry=0x805444d <_start+45>, argc=argc@entry=2, argv=argv@entry=0xffeb8984)
    at ../sysdeps/nptl/libc_start_call_main.h:58
#23 0xf64a8c88 in __libc_start_main_impl (main=0x805444d <_start+45>, argc=2, argv=0xffeb8984, init=0x0, fini=0x0, 
    rtld_fini=0xf7f360e0 <_dl_fini>, stack_end=0xffeb897c) at ../csu/libc-start.c:360
#24 0x08054448 in _start ()