dpctl Build with numpy 2.0

Build conda packages with NumPy 2.0 in the build environment. The meta.yaml file for dpctl was changed to not use pin_compatible('numpy', min='x.x', max='x') per migration guidelines from conda-forge.

Testing should still pick NumPy 1.26.4 from Intel channel.

[x] Have you provided a meaningful PR description?
[ ] Have you added a test, reproducer or referred to an issue with a reproducer?
[ ] Have you tested your changes locally for CPU and GPU devices?
[ ] Have you made sure that new changes do not introduce compiler warnings?
[ ] Have you checked performance impact of proposed changes?
[x] If this PR is a work in progress, are you opening the PR as a draft?

Jul 25 '24 21:07 oleksandr-pavlyk

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. :crossed_fingers:

Jul 25 '24 22:07 github-actions[bot]

coverage: 87.883%. remained the same when pulling 318d55c680490915a4e917aba40f3d59101a0f79 on build-with-numpy-2.0 into 52edb6d68c374793de90b4267348bf7439bf43c5 on master.

Jul 25 '24 22:07 coveralls

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_166 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Jul 25 '24 22:07 github-actions[bot]

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_167 ran successfully. Passed: 895 Failed: 0 Skipped: 119

Jul 26 '24 12:07 github-actions[bot]

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_173 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Jul 26 '24 14:07 github-actions[bot]

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_173 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Jul 26 '24 15:07 github-actions[bot]

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_177 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Jul 28 '24 14:07 github-actions[bot]

In order for this PR to be built in the internal CI with 2.0, I need to change the mamba build flag to --numpy 2.0.0. I can try to do this and restart the checks

Jul 29 '24 09:07 ekomarova

For some reason, the environment still resolved in favor of 1.26.4, even despite --numpy 2.0.0 flag and the absence of any numpy pinning in recipe.. I believe the reason for this is https://github.com/IntelPython/dpctl/commit/486637a20382bc8e3454d17ca60731353e060a63 that deleted numpy from host. For this to work, we need to return numpy to host

Jul 29 '24 09:07 ekomarova

@ekomarova This PR modified conda-packages.yml workflow file to remove use of Intel channel from the conda build command and only use conda-forge. I just recheck that for both platforms and all version of Python (3.9, 3.10, 3.11, and 3.12) installed numpy 2.0.1 into the build environment.

The test environment continues to use Intel channel and thus 3.9, 3.10, and 3.11 install numpy 1.26.4 into test environment from there, while for Python 3.12 the latest numpy gets installed from conda-forge channel.

Jul 29 '24 16:07 oleksandr-pavlyk

@oleksandr-pavlyk Update: The pinning in the recipe https://github.com/IntelPython/dpctl/blob/master/pyproject.toml#L12 is always a priority compared to --numpy <version>. For --numpy <version> to work as expected, the pinning should not be anywhere in host, as it was previously here https://github.com/IntelPython/dpctl/commit/486637a20382bc8e3454d17ca60731353e060a63#diff-330610924437ff65867549da646165faa154ae5201567339b33169f521c084b2L33. Current situation in internal CI:

strict channel priority -c gold -c conda-forge
pinning numpy >= 1.23 in the recipe
using --numpy 2.0.0

In the current situation, conda can only solve the environment so that it satisfies the first two conditions. Since numpy 1.26 is available on gold and it satisfies the strict channel priority and pinning in the recipe numpy >=1.23, and --numpy 2.0.0 is no longer important in this case. We had a similar thing when we needed to build with 1.24. It was the situation when there was a pinning in the recipe, and option --numpy was ignored Example: https://github.com/IntelPython/dpnp/pull/1771

Jul 29 '24 17:07 ekomarova

It looks like in order to make this work in internal CI, we need to rebuild numpy from conda-forge with 2025 mkl, otherwise we will get conflicts. We can plan this activity. Details can be found here: https://github.com/IntelPython/dpctl/pull/1770#issuecomment-2258749260

Jul 31 '24 13:07 ekomarova

Just for information: building numpy 2.0 with conda-forge recipe with 2025 mkl is WIP

Aug 01 '24 13:08 ekomarova

Update: We expect a built numpy 2.0 package in the next few days

Aug 07 '24 17:08 ekomarova

We managed to fix the windows problem. Next, I plan to test this PR with built numpy 2.0, and then add the status

Aug 09 '24 09:08 ekomarova

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_280 ran successfully. Passed: 895 Failed: 0 Skipped: 119

Aug 09 '24 15:08 github-actions[bot]

The package is hosted on a separate channel https://af01p-igk.devtools.intel.com/ui/repos/tree/General/idp-conda-pkgserver-igk-local/numpy, but I can't test this PR yet due to channel problems. As soon as we fix them, I'll restart the checks

Aug 09 '24 18:08 ekomarova

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_285 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Aug 12 '24 20:08 github-actions[bot]

We need to fix something in CI logic in order for the conda build to pass, but in general this PR is now being built with numpy 2.0.1, and tested with 1.26.4 in internal CI

Aug 13 '24 12:08 ekomarova

Array API standard conformance tests for dpctl=0.18.0dev0=py310hdf72452_308 ran successfully. Passed: 894 Failed: 1 Skipped: 119

Aug 14 '24 00:08 github-actions[bot]