CacheLoaderTest fails occasionally in GitHub Action CI

[djeong20]

In GitHub Action CI, CacheLoaderTest.load_async_03_p in the memory unit test fails occasionally.

25/40 unittest_memory                  FAIL             1.59s   exit status 1
>>> MALLOC_PERTURB_=137 /home/runner/work/nntrainer/nntrainer/build/test/unittest/memory/unittest_memory --gtest_output=xml:/home/runner/work/nntrainer/nntrainer/build/unittest_memory.xml

...

[----------] Global test environment tear-down
[==========] 83 tests from 5 test suites ran. (1553 ms total)
[  PASSED  ] 82 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] CacheLoaderTest.load_async_03_p

 1 FAILED TEST

Ran it for the second time without making any changes, and it passed without any problems.

Failed log: https://github.com/nnstreamer/nntrainer/actions/runs/8042775990/job/21963882527

[taos-ci]

:octocat: cibot: Thank you for posting issue #2486. The person in charge will reply soon.

[djeong20]

CacheLoaderTest.load_async_02_p fails as well. https://github.com/nnstreamer/nntrainer/actions/runs/10988854586/job/30505972316?pr=2737