oneDAL icon indicating copy to clipboard operation
oneDAL copied to clipboard

KMeans Init Sparsity Support

Open md-shafiul-alam opened this issue 1 year ago • 8 comments

Add sparsity support to KMeans Init and fix a few bugs in daal sparse kmeans++ init, onedal kmeans++ init, and kmeans infer. Specific changes planned or made in this PR.

  • [x] Fix distance calculation for sparse data in daal KMeans++
  • [x] Allow oneDAL Kmeans++ init to take n_trials same as daal and scikit-learn
  • [x] Fix difference between daal Kmeans++ dense and sparse results
  • [x] Implement KMeans init sparse support for CPU (just calling daal implementation - cpu)
  • [x] Fix oneDAL KMeans sparse infer on GPU
  • [x] Update Kmeans infer for sparse data to allow result options same as dense

I have verified that

  • [x] Daal kmeans init results are same for sparse and dense data
  • [x] oneDAL kmeans init results are same for sparse and dense data
  • [ ] oneDAL kmeans init results are same on cpu and gpu
    • Not same for dense data unless we compute initial centroids for dense GPU using cpu implementation

md-shafiul-alam avatar Jun 07 '24 13:06 md-shafiul-alam

/azp run CI

md-shafiul-alam avatar Jun 18 '24 20:06 md-shafiul-alam

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Jun 18 '24 20:06 azure-pipelines[bot]

/intelci: run

md-shafiul-alam avatar Jun 18 '24 20:06 md-shafiul-alam

/intelci: run

md-shafiul-alam avatar Jun 20 '24 07:06 md-shafiul-alam

Job with relavant changes in sklearnex http://intel-ci.intel.com/ef2ed706-d31d-f1b8-adb9-a4bf010d0e2e

md-shafiul-alam avatar Jun 20 '24 07:06 md-shafiul-alam

/intelci: run

md-shafiul-alam avatar Jun 24 '24 03:06 md-shafiul-alam

/azp run CI

md-shafiul-alam avatar Jun 25 '24 07:06 md-shafiul-alam

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Jun 25 '24 07:06 azure-pipelines[bot]

/azp run CI

md-shafiul-alam avatar Jul 11 '24 18:07 md-shafiul-alam

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Jul 11 '24 18:07 azure-pipelines[bot]

/intelci: run

md-shafiul-alam avatar Jul 11 '24 18:07 md-shafiul-alam

/intelci: run

Alexandr-Solovev avatar Jul 15 '24 07:07 Alexandr-Solovev

/intelci: run

md-shafiul-alam avatar Jul 15 '24 14:07 md-shafiul-alam

/intelci: run

md-shafiul-alam avatar Jul 16 '24 12:07 md-shafiul-alam

/azp run CI

md-shafiul-alam avatar Jul 22 '24 23:07 md-shafiul-alam

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Jul 22 '24 23:07 azure-pipelines[bot]

/intelci: run

md-shafiul-alam avatar Aug 06 '24 12:08 md-shafiul-alam

The failures are not related.

md-shafiul-alam avatar Aug 07 '24 07:08 md-shafiul-alam