[CI] Add cargo-hack into CI pipeline

[alxiong]

cargo-hack is super useful for testing code with different selection/combination of feature flags and rust version etc, which is important to ensure that the feature flags we defined are not mutually exclusive and any combination would work properly.

This issue aims to add this tool to our CI pipeline.

Before any PR, we should discuss the list of desirable usage of the tool.

• [ ] define a list of usage of cargo-hack on jellyfish

---

UPDATE (2022-12-06): a concrete motivating example is that, without such check, we forget to update our test_api.rs which conditionally complied with test_api feature flag, and was out of sync with our new API. But our CI or normal workflow never catches it cause we only compile using default or parallel flag most of the time.

[tessico]

Good idea. I would just be mindful of the running time of the actions. While it's good to ensure good coverage, it sounds like cargo-hack could lead to exponentially long running time of the actions (depending on how good caching is for this). Having reasonable running time for builds is beneficial for development work. If we decide to include it cargo-hack, maybe it's worth adding custom triggers, e.g. only running cargo-hack if Cargo.toml was changed, or something along these lines.

[alxiong]

Having reasonable running time for builds is beneficial for development work. If we decide to include it cargo-hack, maybe it's worth adding custom triggers

Yes, that's a good point, we could also make it a cronjob that runs on a regular cadence (e.g. bi-daily during night time) but not on a per-commit frequency.