Task 5: InferenceService PVC Volume Support

[slin1237]

Task 5: InferenceService PVC Volume Support

Overview

Update the InferenceService controller components to properly handle PVC volumes instead of host paths when BaseModel uses PVC storage.

Scope

• Modify volume creation logic in base.go
• Update volume mount handling for PVC subpaths
• Adjust node selector logic for PVC-based models
• Handle RWO vs RWX access mode differences

Files to Modify

• pkg/controller/v1beta1/inferenceservice/components/base.go
• pkg/controller/v1beta1/inferenceservice/utils/utils.go (if needed)
• Related component files that use base.go functions

Implementation Details

Key Functions to Update

1. UpdatePodSpecVolumes

   • Detect PVC storage type from BaseModel
   • Create PVC volume source instead of hostPath
   • Set appropriate mount path
   • Handle error cases gracefully

2. UpdateVolumeMounts

   • Add volume mounts with subpath support
   • Ensure read-only mounting for model volumes
   • Check if volume mount is necessary based on annotations

3. UpdatePodSpecNodeSelector

   • Skip node selector for PVC-based models
   • Let Kubernetes scheduler handle PVC accessibility
   • Consider future pod affinity enhancements for RWO PVCs
   • Maintain existing behavior for downloaded models

4. ProcessBaseAnnotations

   • Add storage type annotation for PVC models
   • Include PVC name and subpath in annotations
   • Useful for debugging and observability

Design Considerations

Volume Creation:

• PVC volumes should reference the PVC name from the parsed URI
• Always mount as read-only for model data integrity
• Handle both presence and absence of subpaths

Node Scheduling:

• PVC models don't need node selectors (unlike downloaded models)
• Kubernetes scheduler handles PVC accessibility constraints
• RWO PVCs will naturally limit pod placement
• RWX PVCs allow flexible scheduling

Backward Compatibility:

• Preserve existing hostPath behavior for non-PVC models
• Ensure fine-tuned serving logic remains intact
• Don't break existing InferenceService deployments

Test Cases

1. PVC Volume Creation:

   • PVC volume created correctly with claim name
   • Host path volume created for non-PVC storage
   • Error handling for invalid PVC URI

2. Volume Mount Handling:

   • Correct mount path with subpath for PVC
   • Mount without subpath when not needed
   • Read-only mounts enforced

3. Node Selector Logic:

   • No node selector for PVC storage
   • Node selector present for downloaded models
   • Fine-tuned serving cases handled correctly

4. Annotation Processing:

   • Storage type annotation added
   • PVC details in annotations
   • Backward compatibility maintained

Constants to Add

New constants needed in pkg/constants/constants.go:

• StorageTypeAnnotationKey: Track storage type in pod annotations
• PVCNameAnnotationKey: Record PVC name for debugging
• PVCSubPathAnnotationKey: Record subpath for debugging

Acceptance Criteria

• [ ] InferenceService pods mount PVC volumes correctly
• [ ] Subpath handling works for nested model directories
• [ ] Node selector logic adapted for PVC storage
• [ ] Annotations include PVC information
• [ ] Backward compatibility maintained for existing storage types
• [ ] Unit tests cover all PVC scenarios
• [ ] Integration tests verify pod creation with PVC volumes

Dependencies

• Task 1: Storage URI Parsing
• Task 4: BaseModel Controller (for PVC validation)

Estimated Effort

3-4 hours

[carlory]

Hi @slin1237, can I pick up this task?

[slin1237]

Hi @slin1237, can I pick up this task?

100% Thank you so much. I should be able to finish task 4 today so u can use it for testing

[carlory]

/assign