[CI] Fix minimal-install python 3.11: build wheel with unsupported tags.
What happened + What you expected to happen
In this PR #34147 , a line of set -xe was added into install-minimal.sh, which caused CI test: Minimal install 3.11 failed.
https://buildkite.com/ray-project/oss-ci-build-branch/builds/3652#0187c3e5-133d-4258-bfbe-43e979b29507

We fixed the test itself by removing set -xe as a workaround, but we still need to find the root bug.
Versions / Dependencies
Master branch
Reproduction script
A simple reproduction PR: https://github.com/ray-project/ray/pull/34939
Issue Severity
Medium: It is a significant difficulty but I can work around it.
This P2 issue has seen no activity in the past 2 years. It will be closed in 2 weeks as part of ongoing cleanup efforts.
Please comment and remove the pending-cleanup label if you believe this issue should remain open.
Thanks for contributing to Ray!