openvinotoolkit
2025.4 Dockerfile.redhat fails when RUN_TESTS=1
When the RUN_TESTS is set to 1, the build fails because the default python is 3.9 and the tokenizer's tests require python >= 3.10. (Tokenizer Commit e469d9b).
Preparing metadata (pyproject.toml): finished with status 'done' ERROR: Package 'openvino-tokenizers' requires a different Python: 3.9.23 not in '>=3.10' Error: building at STEP "RUN if [ "$RUN_TESTS" == "1" ] ; then mkdir -p demos/common/export_models/ && mv export_model.py demos/common/export_models/ && ./prepare_llm_models.sh /ovms/src/test/llm_testing docker && ./run_unit_tests.sh ; fi": while running runtime: exit status 1
UBI 9 has both python 3.11 and python 3.12 in addition to the default 3.9. I'd suggest standardizing on python 3.12 for UBI 9. However, that likely means creating a virtual environment to isolate it from the OS python 3.9. Or maybe the tokenizer deprecation of 3.9 was too soon? If tokenizer is OK with going back to 3.9, we can revisit this when we upgrade to UBI 10.
@dkalinowski we are compiling tokenizers from source so actually we could use here that version instead of relying on pip packages. Alternatively we could update the default python version to 3.12. That means the binary ovms package would also require python3.12 but that should be ok for users while all the versions on RH are official anyway.
@dtrawins I already tried doing that in this PR: https://github.com/openvinotoolkit/model_server/pull/3854 However, after installing python 3.12 and trying to use that for model export I found another issues (described in the PR)
I can look more into migrating to 3.12. However, I would like to understand if running the tokenizer on 3.9 will malfunction or is it deprecated just because the python community deprecated it? (IOW, this is advisory and not functional.)