Unit-test timings with Clang ASAN

[sayerhs]

Looking at the unit tests I notice that the unit tests for the matrix free kernels take a significant amount of time to run on the Clang build with address-sanitizer turned on.

[----------] 2 tests from ConductionSimulationFixture
[ RUN      ] ConductionSimulationFixture.difference_from_exact_solution_is_smallish
[       OK ] ConductionSimulationFixture.difference_from_exact_solution_is_smallish (157472 ms)
[ RUN      ] ConductionSimulationFixture.heat_conduction_reduces_peak_value
[       OK ] ConductionSimulationFixture.heat_conduction_reduces_peak_value (156612 ms)
[----------] 2 tests from ConductionSimulationFixture (314092 ms total)

I talked to @rcknaus over email and he mentioned that he sees a slowdown with ASAN but it is still significantly faster than what's happening on the test machine. This has led to timeouts for unit tests. Is this expected behavior on the test machine, due to oversubscribing of processors by all the nightly tests running on the system? Or is this something that needs further investigation?

[jrood-nrel]

I'm moving to Clang 10 soon and will revisit this at that point. Clang 6 seemed to do well with ASAN, but I'm not convinced ASAN in Clang 7 has been that great and it's been running things really slowly. I had to change and blacklist more things in Clang 7 and when using 6 I felt like we were finding memory leaks a lot more. Clang 7 has been silent on that front, even though I verified in the past that it was catching leaks by creating one intentionally.

[psakievich]

Closing this for now. The current dashboard is about 6x faster than what is referenced here. While still not super fast it seems good enough for now. If there is still a concern please reopen.