Performance regressions in BaseBenchmarks due to #54647 (Cleanup `MemoryRef`)

[Zentrik]

Both [["array", "index", ("sumeach_view", "BitMatrix")]] and [["array", "index", ("sumlinear_view", "BitMatrix")]] have a ~50% regression in min wall time.

[["union", "array", ("skipmissing", "filter", "Union{Missing, Int8}", 1)]], [["union", "array", ("skipmissing", "filter", "Union{Missing, Int64}", 1)]], [["union", "array", ("skipmissing", "filter", "Bool", 0)]] and [["union", "array", ("skipmissing", "filter", "Int64", 0)]] have a ~20% regression in min wall time.

All were bisected to fa038d95b99301072d41514084e9d68a4c0776e9. Here's the nanosoldier report, https://github.com/JuliaCI/NanosoldierReports/blob/master/benchmark/by_date/2024-06/10/report.md. The BitMatrix regression doesn't show up in the report due to a high tolerance, the data is at https://tealquaternion.camdvr.org/compare.html?start=b8e714dfa3d81edd56ccda853322b6e42d94cf60&end=77c28ab286f48afe2512e2ae7f7310b87ca3345e&stat=min-wall-time&name=BitM instead.