[maintenance] lazy load dpnp.tensor/dpnp and prepare for array_api lazy importing

[icfaust]

Description

Dpctl and dpnp are quasi-dependencies which will silently error out if not installed. This is done at import time throughout the codebase, meaning that it is mixed into the codebase in a difficult manner. As the number of supported data frameworks are increased, such a strategy is unsustainable. Lazy loading of the necessary packages must be done, as the load time of follow-on frameworks like PyTorch are non-negligible (>1s). If we were to follow the same strategy, load times of sklearnex would be even longer even if pytorch isn't used but is available. This will compound as we would add framework support. Cleanly separating and isolating their use is necessary.

Therefore we need to first move dpnp and dpctl.tensor support to a lazy loading approach which will then be extended by follow-on frameworks. The next step will be pytorch queue extraction, which will require this infrastructure.

The strategy will follow that of array_api_compat which can check for namespaces without importing the actual modules, and for the direct use of the frameworks, a depedency injection + monkeypatching scheme is used with decorator lazy_import.

---

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed. This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way. For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

• [ ] I have reviewed my changes thoroughly before submitting this pull request.
• [ ] I have commented my code, particularly in hard-to-understand areas.
• [ ] I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
• [ ] Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
• [ ] I have added a respective label(s) to PR if I have a permission for that.
• [ ] I have resolved any merge conflicts that might occur with the base branch.

Testing

• [ ] I have run it locally and tested the changes extensively.
• [ ] All CI jobs are green or I have provided justification why they aren't.
• [ ] I have extended testing suite if new functionality was introduced in this PR.

Performance

• [ ] I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
• [ ] I have provided justification why performance has changed or why changes are not expected.
• [ ] I have provided justification why quality metrics have changed or why changes are not expected.
• [ ] I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 77.46479% with 32 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
onedal/datatypes/_sycl_usm.py	82.05%	5 Missing and 2 partials :warning:
onedal/linear_model/logistic_regression.py	0.00%	5 Missing :warning:
onedal/utils/_third_party.py	88.88%	4 Missing and 1 partial :warning:
onedal/ensemble/forest.py	0.00%	4 Missing :warning:
sklearnex/ensemble/_forest.py	0.00%	4 Missing :warning:
onedal/_device_offload.py	75.00%	3 Missing :warning:
onedal/datatypes/tests/common.py	60.00%	2 Missing :warning:
onedal/utils/_array_api.py	83.33%	1 Missing and 1 partial :warning:

Flag	Coverage Δ
azure	79.82% <76.76%> (+0.06%)	:arrow_up:
github	73.40% <65.51%> (-0.14%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
onedal/datatypes/__init__.py	100.00% <100.00%> (ø)
onedal/datatypes/_data_conversion.py	92.30% <100.00%> (+1.68%)	:arrow_up:
onedal/utils/_sycl_queue_manager.py	72.13% <100.00%> (-1.72%)	:arrow_down:
sklearnex/_device_offload.py	77.33% <100.00%> (-1.42%)	:arrow_down:
onedal/datatypes/tests/common.py	90.74% <60.00%> (-1.42%)	:arrow_down:
onedal/utils/_array_api.py	82.92% <83.33%> (-1.86%)	:arrow_down:
onedal/_device_offload.py	76.66% <75.00%> (+1.05%)	:arrow_up:
onedal/ensemble/forest.py	72.63% <0.00%> (-0.87%)	:arrow_down:
sklearnex/ensemble/_forest.py	83.24% <0.00%> (-0.50%)	:arrow_down:
onedal/linear_model/logistic_regression.py	28.26% <0.00%> (-1.37%)	:arrow_down:
... and 2 more

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[icfaust]

/intelci: run

[icfaust]

/intelci: run

[icfaust]

Pull Request Overview

This PR implements lazy-loading for dpnp, dpctl.tensor, and array_api support to mitigate import-time performance overhead and decouple heavy dependencies from estimator initialization. Key changes include refactoring import paths from deprecated helper modules to a new _third_party module, updating functions in several modules (e.g. logistic regression, ensemble forests, device offload) to use lazy evaluation, and adding a test in test_common.py to verify that only numpy and pandas are loaded on estimator import.

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated no comments.

Show a summary per file File Description tests/run_examples.py Updated import path for dpctl availability. sklearnex/tests/test_memory_usage.py Removed unused dpctl/dpnp imports from dpep_helpers. sklearnex/tests/test_common.py Added new test to validate lazy import behavior for data frameworks. onedal/ensemble/_forest.py Replaced get_unique_values_with_dpep with new inline unique extraction. onedal/_device_offload.py Updated handling of output conversion and lazy data extraction. onedal/utils/_third_party.py Introduced new helper functions for lazy importing and third-party checks. onedal/utils/_array_api.py Added caching mechanism for mapping array types to SYCL namespaces. onedal/tests/utils/_dataframes_support.py Modified dpnp availability checks using try/except. onedal/linear_model/logistic_regression.py Updated unique value extraction using _get_sycl_namespace. onedal/datatypes/* Various adjustments to imports and data conversion functions. onedal/common/tests/test_sycl.py Updated dpctl availability checks to use new module. Comments suppressed due to low confidence (1) onedal/ensemble/forest.py:321

• The variable 'xp' is used without being defined. It should be initialized by extracting the array namespace from X (e.g. by adding '_, xp, _ = _get_sycl_namespace(X)' before using xp.unique).

                self.classes_ = xp.unique(y)

Low confidence recommendation for onedal/ensemble/forest.py is incorrect, as xp is defined beforehand.

[icfaust]

/intelci: run

[icfaust]

Will rerun because of private-CI issues.

[icfaust]

/intelci: run

[Alexsandruss]

/intelci: run

[ethanglaser]

/intelci: run

[icfaust]

We are having private CI infrastructure issues, especially with the GPU runners, so this will be on hold until those run properly.

[icfaust]

/intelci: run

[icfaust]

/intelci: run