Error in 2024.3 build with RUN_TESTS=1

[stevegrubb]

Describe the bug When building the 2024.3 release using the Red Hat Dockerfile and with the unit tests enabled, the build eventually ends with the following:

[2024-08-28 14:35:19.804][22736][serving][info][llmtemplate_test.cpp:491] Closing configFile succeed unknown file: Failure C++ exception with description "filesystem error: directory iterator cannot open directory: No such file or directory [/ovms/llm_testing/facebook/opt-125m]" thrown in SetUp(). [4,967 / 4,968] Testing //src:ovms_test; 162s local [4,967 / 4,968] Testing //src:ovms_test; 1s local FAIL: //src:ovms_test (see /root/.cache/bazel/_bazel_root/bc57d4817a53cab8c785464da57d1983/execroot/ovms/bazel-out/k8-opt/testlogs/src/ovms_test/test.log)

Might be related to PR #2547? What is unclear is if it downloads the model or expects one to be supplied with bind mounts. If it does download a model, it might be good to make that more verbose in the build output.

To Reproduce Steps to reproduce the behavior:

podman build --tag model_server:1.3 --build-arg BASE_IMAGE=nvcr.io/nvidia/cuda:11.8.0-runtime-ubi8 --build-arg RELEASE_BASE_IMAGE=nvcr.io/nvidia/cuda:11.8.0-runtime-ubi8 --build-arg VERBOSE_LOGS=ON --build-arg RUN_TESTS=1 --build-arg CHECK_COVERAGE=0 --build-arg NVIDIA=1 --build-arg GPU=1 --build-arg ov_use_binary=0 --build-arg BASE_OS=redhat --build-arg ov_source_org=openvinotoolkit --build-arg ov_contrib_org=openvinotoolkit --build-arg ov_source_branch='releases/2024/3' --build-arg ov_contrib_branch='releases/2024/3' --build-arg ov_tokenizers_branch='releases/2024/3' --build-arg JOBS=16 --build-arg debug_bazel_flags="--strip=always --define MEDIAPIPE_DISABLE=0 --define PYTHON_DISABLE=0 --//:distro=redhat --local_ram_resources=23552 --local_cpu_resources=16" -f Dockerfile.redhat .

Expected behavior When the 2024.2-release branches are used, the build completes. Or when RUN_TESTS=0, the build completes.