hummingbird
hummingbird copied to clipboard
coredumps on Ubuntu pipeline runs
We thought that TVM was causing Ubuntu out-of-memory errors, so we skipped all the TVM tests in Ubuntu (#709).
But now, I am seeing additional coredumps in Ubuntu runs that aren't related to TVM. Example 1 Example 2
It reaches the end of the Test with Pytest
stage successfully, and then dies:
========== 598 passed, 66 skipped, 1410 warnings in 214.69s (0:03:34) ==========
/home/runner/work/_temp/c325d19a-2f4b-46db-bd6e-51d55c415279.sh: line 1: 2350 Aborted (core dumped) pytest
Error: Process completed with exit code 134.
It also appears to be transient. Is this at all related to what you were seeing @mshr-h ? Maybe the problem is bigger than TVM?
I was seeing a similar error, like all the tests are passed but the stage fails. Can you try to skip the TVM installation in the pipeline? I'm guessing when the Python interpreter imports TVM, something happens.
It's been a full week (with many runs) of this and no error. Maaaaaybe the memory issue was transient on the github-actions side? Let's hope 🤞...I will leave this open a month or so to see if happens again.