MDEV-36316/MDEV-36327/MDEV-36328 Msan debug innodb

[grooverdan]

trafficstars

• [x] The Jira issue number for this PR is: MDEV-36316/MDEV-36327/MDEV-36328

Description

MSAN + CMAKE_BUILD_TYPE=Debug has stack overrun issues

Just a POC at the moment. Some tests still failing:

main.subselect_sj2_jcl6                  w2 [ fail ]
        Test ended at 2025-05-23 10:34:57

CURRENT_TEST: main.subselect_sj2_jcl6
mysqltest: In included file "/source/mysql-test/main/subselect_sj2.test": 
included from /source/mysql-test/main/subselect_sj2_jcl6.test at line 18:
At line 893: query 'call p1()' failed: ER_STACK_OVERRUN_NEED_MORE (1436): Thread stack overrun:  444864 bytes used of a 458752 byte stack, and 16000 bytes needed. Consider increasing the thread_stack system variable.

Release Notes

non-user facing

How can this PR be tested?

CMAKE_BUILD_TYPE=Debug under MSAN like https://mariadb.com/kb/en/compile-and-using-mariadb-with-sanitizers-asan-ubsan-tsan-msan/ but without CMAKE_C,CXX_FLAGS=-O2

Basing the PR against the correct MariaDB version

• [ ] This is a new feature or a refactoring, and the PR is based against the main branch.
• [X] This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

• [ ] I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
• [ ] For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.