Feat/sparkconnect ingress

[rahul810050]

Summary

This PR introduces first-class Ingress support for SparkConnect by extending the CRD, controller logic, and Helm chart. This enables users to expose Spark Connect Server externally (HTTP/TCP ingress) in a consistent, configurable way using:

spec:
  server:
    ingress:
      enable: true
      annotations:
      tls:
      ingressClassName:
      path:
      host:

This aligns SparkConnect with the existing SparkApplication ingress support and improves usability for client applications that need stable external endpoints.

Key Changes:

1. CRD Enhancements

• Added new optional field: spec.server.ingress with subfields:

  • enable
  • path
  • host
  • ingressClassName
  • annotations
  • tls

• Regenerated:

  • CRD YAML
  • deepcopy methods
  • schema validation

2. Controller Enhancements

• Added new createOrUpdateIngress() reconciliation step.

• Added mutateIngress() helper for:

  • setting labels
  • wiring correct service backend
  • applying ingress annotations
  • TLS configuration
  • host/path rules

• Added conditions/state updates when ingress becomes ready.

3. Helm Chart Updates

• Updated:

  • values.yaml
  • templates/controller/rbac.yaml (added ingress permissions)
  • SparkConnect values block (default disabled)

• Ingress can now be enabled via Helm:

controller:
 sparkconnect:
   ingress:
     enable: true
     host: "connect.example.com"
     path: "/"
     ingressClassName: "nginx"

4. RBAC Updates

Added required permissions:

• networking.k8s.io/v1
  • ingresses
  • ingresses/status

5. Tests / Envtest

• Updated CRD references
• Added e2e-compatible envtest setup
• Fixed paths: BinaryAssetsDirectory: bin/k8s/1.32.0-linux-amd64

Testing

Local Testing

export KUBEBUILDER_ASSETS="$PWD/bin/k8s/v1.32.0-linux-amd64"
go test ./... -v

All unit tests pass.

Ingress creation test

Applied:

apiVersion: sparkoperator.k8s.io/v1alpha1
kind: SparkConnect
metadata:
  name: demo
spec:
  server:
    ingress:
      enable: true
      host: demo.connect.local

Verified:

• ingress created
• rules point to the correct service
• ready condition set
• TLS optional configuration works

Why This Matters

This completes a missing capability for Spark Connect in Kubernetes:

• Enables secure remote clients
• Provides stable, DNS-based endpoint for Spark Connect
• Integrates with existing ingress ecosystems (NGINX, Traefik, Istio ingress-gateway)
• Improves parity with SparkApplication’s UI ingress

Issue #2715

Checklist

• [x] CRD updated
• [x] Deepcopy regenerated (make generate)
• [x] Controller reconciliation updated
• [x] Helm chart updated
• [x] RBAC updated
• [x] Tests passing locally

[google-oss-prow[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign yuchaoran2011 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[ChenYi015]

A rebase should be made once #2742 gets merged, code changes from two different PRs mixes up.

[rahul810050]

A rebase should be made once #2742 gets merged, code changes from two different PRs mixes up.

Thanks for the clarification @ChenYi015 ! Understood Ill wait for #2742 to get merged first. Once it’s merged, Ill rebase my branch on top of the updated master to ensure this PR contains only the intended changes without mixing code from multiple PRs.