Chore: ideas on improving CI

[NobodyXu]

• [x] ~~split --release compilation to a separate job, since it basically invalids caching~~ it doesn't have much impact on pipeline
• [ ] run tests on older version of macOS and ubuntu
• [ ] run tests on arm/risc-v using cross
• [ ] run tests on M1
• [ ] run tests with llvm-lib #1026
• [ ] run test on Emscripten
• [ ] run tests on musl