yunikorn-site icon indicating copy to clipboard operation
yunikorn-site copied to clipboard

[YUNIKORN-2886] update Spark operator documentation for YuniKorn integration

Open ryankert01 opened this issue 5 months ago • 3 comments

What is this PR for?

Spark Operator 2.0 has been released with full YuniKorn support. We need to update the website and push this information.

Spark Operator with YuniKorn details:

  • Support gang scheduling with Yunikorn
  • Set schedulerName to Yunikorn
  • Account for spark.executor.pyspark.memory in Yunikorn gang scheduling

See Spark Operator v2.0.0 tag for details.

What type of PR is it?

  • [ ] - Bug Fix
  • [x] - Improvement
  • [ ] - Feature
  • [ ] - Documentation
  • [ ] - Hot Fix
  • [ ] - Refactoring

Todos

  • [ ] - Task

What is the Jira issue?

https://issues.apache.org/jira/browse/YUNIKORN-2886

How should this be tested?

Screenshots (if appropriate)

Questions:

  • [ ] - The licenses files need update.
  • [ ] - There is breaking changes for older versions.
  • [ ] - It needs documentation.

ryankert01 avatar Sep 27 '24 09:09 ryankert01