In-tree storage plugin to CSI Driver Migration

[davidz627]

Feature Description

• One-line feature description (can be used as a release note): Migrate the internals of the in-tree plugins to call out to CSI Plugins while maintaining the original API
• Primary contact (assignee): @davidz627
• Responsible SIGs: sig-storage
• Design proposal link (community repo): https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md
• KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190129-csi-migration.md
• PRs:
  • [x] https://github.com/kubernetes/kubernetes/pull/69830
• Link to e2e and/or unit tests:
  • GCE: https://testgrid.k8s.io/sig-gcp-compute-persistent-disk-csi-driver#Migration%20Kubernetes%20Master%20Driver%20Latest
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @saad-ali @jsafrane
• Approver (likely from SIG/area to which feature belongs): @saad-ali
• Feature target (which target equals to which milestone):
  • Alpha release: 1.13
  • Beta release: 1.17
  • Stable release target (x.y): 1.19

[davidz627]

/sig storage /cc @ddebroy

[kacole2]

@davidz627 this release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement for 1.13 if there is a high level of confidence it will meet the following deadlines: Docs (open placeholder PRs): 11/8 Code Slush: 11/9 Code Freeze Begins: 11/15 Docs Complete and Reviewed: 11/27

Do you feel confident it will make these? If so we can include it for 1.13 tracking. Thanks!

[saad-ali]

@kacole2 This is multi quarter work with other components depending on it so we want to get it started ASAP (this quarter). That said as we approach end of the month @davidz627 and @ddebroy will make a call about if it is feasible to ship something this quarter or not. If not, we will remove it from the 1.13 milestone. For now let's track it.

[tfogo]

Hi @davidz627 and @ddebroy, do you think we will be able to land this in 1.13? Please keep in mind this release has an aggressive timeline and we should only include it if you're confident it will meet the deadlines @kacole2 outlined above. Thanks!

[tfogo]

Also, I'm assuming this enhancement won't need any documentation updates on k/website since it's an internal change. Please let me know if we will need documentation. Thanks!

[davidz627]

@tfogo no documentation required. It's looking like we wont hit our 1.13 target and will target 1.14 instead. @saad-ali

[AishSundar]

@davidz627 and @saad-ali is it safe to move this out to 1.14? if so I can do that.

[AishSundar]

Synced with @davidz627 offline. This is now punted to 1.14. No revert / rollback needed in k/k because feature is being developed on a separate branch to enable atomic commit for the feature when it's ready

@kacole2 @tfogo

[claurence]

@davidz627 Hello - I’m the enhancement’s lead for 1.14 and I’m checking in on this issue to see what work (if any) is being planned for the 1.14 release. Enhancements freeze is Jan 29th and I want to remind that all enhancements must have a KEP - I don't see a KEP for this issue can you please drop a link for the KEP.

[davidz627]

@claurence Work on this started before the KEP process started so we have a design in the "old" style. https://github.com/kubernetes/community/blob/b7daf2fdfa7c4984d9236336b4f1ccc9f048db98/contributors/design-proposals/storage/csi-migration.md We have a significant amount of work planned for the 1.14 release including but not limited to: https://github.com/kubernetes/kubernetes/pull/71901 <- PR will be broken up into smaller PR's soon https://github.com/kubernetes/kubernetes/pull/70909 https://github.com/kubernetes/kubernetes/pull/72950

As well as some work that has already merged: https://github.com/kubernetes/kubernetes/pull/72770

[claurence]

@davidz627 - apologies for getting back to this so late - thanks for linking the original design proposal doc - we do want all issues to have a KEP for 1.14 - even if it's just pointing to the design proposal doc. And that it spells out graduation criteria and testing plan.

[saad-ali]

KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1234-20190129-csi-migration.md

[dims]

Looks like it got renamed Saad - https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190129-csi-migration.md

[spiffxp]

Adding tracked/yes label to reconcile against https://bit.ly/k8s114-enhancements FYI @claurence @lachie83 @lledru @ameukam

[makoscafee]

Hi, @davidz627 👋 I'm the v1.14 docs shadow.

Does this enhancement require any new docs (or modifications)?

Just a friendly reminder we're looking for a PR against k/website (branch dev-1.14) due by Friday, March 1. It would be great if it's the start of the full documentation, but even a placeholder PR is acceptable. Let me know if you have any questions!

[makoscafee]

@tfogo no documentation required. It's looking like we wont hit our 1.13 target and will target 1.14 instead. @saad-ali

Sorry! From this i assume it wont need any documentation still, right ?

[thockin]

Does this need an API review?

[dims]

@thockin it's a simple translation and does not change API in any way.

@ddebroy please confirm

[liggitt]

I thought this was the item the CSIDriver/CSINodeInfo APIs were being added under. Are those objects added under https://github.com/kubernetes/enhancements/issues/557 instead?

[dims]

@liggitt the PR for "Make CSINodeInfo and CSIDriver Core APIs"? that would be https://github.com/kubernetes/kubernetes/pull/73883

which points to https://github.com/kubernetes/enhancements/issues/770 and https://github.com/kubernetes/enhancements/issues/603

[liggitt]

@liggitt the PR for "Make CSINodeInfo and CSIDriver Core APIs"? that would be kubernetes/kubernetes#73883

which points to #770 and #603

thanks, updated https://github.com/kubernetes-sigs/architecture-tracking/projects/3 with those and removed this

[msau42]

The API changes for #557 #603 #770 and this are all covered by kubernetes/kubernetes#73883

[claurence]

@davidz627 I see a few open PRs mentioned above ^^ are any of these blocking for the 1.14 release? Or are all PRs for 1.14 merged?

[ddebroy]

We do need kubernetes/kubernetes#74926 reviewed and merged. /cc @msau42

[davidz627]

also: https://github.com/kubernetes/kubernetes/pull/74835 https://github.com/kubernetes/kubernetes/pull/74858 https://github.com/kubernetes/kubernetes/pull/73840

[kacole2]

Hello, I'm the Enhancement Lead for 1.15. Is this feature going to be graduating alpha/beta/stable stages in 1.15? Please let me know so it can be tracked properly and added to the spreadsheet.

Once coding begins, please list all relevant k/k PRs in this issue so they can be tracked properly.

/milestone clear

[msau42]

The plan is to move to beta in 1.15.

[kacole2]

/milestone v1.15 /stage beta

[davidz627]

Our project board can be accessed here for beta milestone tracking: https://github.com/orgs/kubernetes-csi/projects/13

[justaugustus]

/assign @davidz627