[BUG] Segfault in stdlib's CI, in `builtin/test_reversed.mojo`

[gabrieldemarmiesse]

Bug description

When working on https://github.com/modularml/mojo/pull/2368 I encountered a segfault in the CI even though the tests were passing locally. After restarting the CI, the tests were passing again. So it seems there is a flaky test. If someone could investigate, that would be great.

reating build directory for building the stdlib running the tests in.
Packaging up the Standard Library.
Successfully created /Users/runner/work/mojo/mojo/build/stdlib.mojopkg
Packaging up the test_utils.
-- Testing: 80 tests, 3 workers --
Testing:  0.. [10](https://github.com/modularml/mojo/actions/runs/8774779628/job/24076341331?pr=2368#step:9:11).. 20.
FAIL: Mojo Standard Library :: builtin/test_reversed.mojo (22 of 80)
******************** TEST 'Mojo Standard Library :: builtin/test_reversed.mojo' FAILED ********************
Exit Code: 139

Command Output (stderr):
--
RUN: at line 13: mojo -debug-level full /Users/runner/work/mojo/mojo/stdlib/test/builtin/test_reversed.mojo
+ mojo -debug-level full /Users/runner/work/mojo/mojo/stdlib/test/builtin/test_reversed.mojo
Please submit a bug report to https://github.com/modularml/mojo/issues and include the crash backtrace along with all the relevant source codes.
Stack dump:
0.	Program arguments: mojo -debug-level full /Users/runner/work/mojo/mojo/stdlib/test/builtin/test_reversed.mojo
#0 0x0000000102871384 llvm_strlcpy (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x1000c5384)
#1 0x000000010286f670 llvm_strlcpy (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x1000c3670)
#2 0x0000000102871a70 llvm_strlcpy (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x1000c5a70)
#3 0x00000001986c7584 (/usr/lib/system/libsystem_platform.dylib+0x180477584)
#4 0x0000000300198de0 
#5 0x0000000102bffd64 __jit_debug_register_code (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x100453d64)
#6 0x00000001027ca67c _mh_execute_header (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x10001e67c)
#7 0x00000001027b37a8 _mh_execute_header (/Users/runner/.modular/pkg/packages.modular.com_nightly_mojo/bin/mojo+0x1000077a8)
#8 0x000000019830e0e0 
mojo crashed!
Please file a bug report.
[19522:93928:20240421,184455.219101:WARNING crash_report_exception_handler.cc:257] UniversalExceptionRaise: (os/kern) failure (5)
/Users/runner/work/mojo/mojo/build/stdlib/builtin/Output/test_reversed.mojo.script: line 1: 19520 Segmentation fault: [11](https://github.com/modularml/mojo/actions/runs/8774779628/job/24076341331?pr=2368#step:9:12)  mojo -debug-level full /Users/runner/work/mojo/mojo/stdlib/test/builtin/test_reversed.mojo

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
********************
Failed Tests (1):
  Mojo Standard Library :: builtin/test_reversed.mojo

Here is the full log: https://github.com/modularml/mojo/actions/runs/8774779628/job/24076341331?pr=2368

Steps to reproduce

Wouldn't that be convenient?

System information

Macos
modular-v0.7.2
packages/2024.4.1618/mojo-arm64-apple-darwin22.6.0-2024.4.1618-17-0.tar.zst

(surely more system information can be found in the CI's logs)

[JoeLoser]

Is it because of https://github.com/modularml/mojo/commit/abf2975d48fc7631e0e2ca741340289ec622f5d0? Want to try reverting that and see if it passes consistently on CI? It just merged a few hours ago.

[helehex]

oh no

[helehex]

i was unable to reproduce

[JoeLoser]

@ConnorGray FYI - something to keep an eye out for if you see any flakiness this week for CI

[keith]

I've started seeing this on the same test, I'm not sure if it's related to this original issue. I believe the newest iteration of it was introduced with https://github.com/modularml/mojo/pull/2340, cc @jayzhan211

The 2 variations I've seen are:

builtin/test_reversed.mojo:43:21: AssertionError: `left == right` comparison failed:
   left: 140196055111384
  right: 4

And then also it hitting this assert: https://github.com/modularml/mojo/blob/f5f5109541c31615a68a3c4b58bd1e75b59625f6/stdlib/src/utils/variant.mojo#L398 (which seems to be more common in my testing now)

[martinvuyk]

had this just now in nightly 2024.5.2205

FAIL: Mojo Standard Library :: builtin/test_reversed.mojo (29 of 97)
******************** TEST 'Mojo Standard Library :: builtin/test_reversed.mojo' FAILED ********************
Exit Code: 133

Command Output (stdout):
--
get: wrong variant type

[JoeLoser]

had this just now in nightly 2024.5.2205

FAIL: Mojo Standard Library :: builtin/test_reversed.mojo (29 of 97)
******************** TEST 'Mojo Standard Library :: builtin/test_reversed.mojo' FAILED ********************
Exit Code: 133

Command Output (stdout):
--
get: wrong variant type

Do you have a consistent repro or just from a run on CI?

[martinvuyk]

Do you have a consistent repro or just from a run on CI?

just from a run on CI

[JoeLoser]

Fixed with https://github.com/modularml/mojo/commit/4d089aa492e74fc82c3952155264782278c79063, thanks @gabrieldemarmiesse!