feat: add poddefault for automatic Jupyterlab authentication

[juliusvonkohout]

Description of your changes: @Bobgy A poddefault is automatically created according to https://github.com/kubeflow/pipelines/issues/5138

Checklist:

• [x] The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

[google-oss-robot]

Hi @juliusvonkohout. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[google-oss-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign zijianjoy after the PR has been reviewed. You can assign the PR to them by writing /assign @zijianjoy in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• manifests/kustomize/OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[thesuperzapper]

@Bobgy this is probably something we shouldn't decide for users, (not all admins will want to auto-mount a token).

---

Also, I think this PR highlights that we should extend the main Profile Controller to remove the need for the hacky pipeline-profile-controller python script. For example, we can make a new ProfileResourceTemplate CRD, which defines arbitrary K8S templates to be automatically added to each Profile (by the profile controller).

Then, the Kubeflow Pipelines YAML would only need to include those ProfileResourceTemplate resources.

[juliusvonkohout]

@Bobgy this is probably something we shouldn't decide for users, (not all admins will want to auto-mount a token).

Also, I think this PR highlights that we should extend the main Profile Controller to remove the need for the hacky pipeline-profile-controller python script. For example, we can make a new ProfileResourceTemplate CRD, which defines arbitrary K8S templates to be automatically added to each Profile (by the profile controller).

Then, the Kubeflow Pipelines YAML would only need to include those ProfileResourceTemplate resources.

It is not auto mounted. it is just providing the poddefault that might/can be used in a Jupyterlab. https://github.com/kubeflow/kubeflow/pull/6160 would auto mount it by default. But if you have a cleaner solution that is of course appreciated.

[juliusvonkohout]

Is the profile controller able to monitor these resources and recreate them if they have been deleted?

[ca-scribner]

Yes, I believe the profile controller repeatedly does a reconciliation loop (every 10s maybe? something like that) and if it ever sees a missing resource, it creates a new one. The sync.py script does not do really detailed inspection of the objects though, so for example if you edited the proposed PodDefault and just deleted the spec, the profile controller wouldn't notice that or fix anything

[ca-scribner]

@thesuperzapper I don't know if admins will want to auto-mount tokens, but this feature feels like an easy quality of life improvement for users that doesn't change much for admins. It creates an object that users have permission to create for themselves (any user could create this PodDefault in their namespace), it just means they don't need to know how.

Having seamless access to the pipeline client from notebooks feels like a feature most users would expect. I really like the ProfileResourceTemplate CRD idea and would love to have that. In the interim though, this feels like a nice low-cost fix.

[ca-scribner]

If there is agreement on making this change, the tests in pipelines/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/test_sync.py should be updated

[juliusvonkohout]

If there is agreement on making this change, the tests in pipelines/manifests/kustomize/base/installs/multi-user/pipelines-profile-controller/test_sync.py should be updated

Actually i and @thesuperzapper want to get rid of metacontroller and pipelines-profile-controller as described in https://github.com/kubeflow/pipelines/pull/7219#issuecomment-1024086393. This WIP pull request also contains the change you proposed above. Are you willing to help there?

[ca-scribner]

I am for removing the profile controller and metacontroller, and really like the ProfileResourceTemplate idea (@thesuperzapper have you opened that as a separate feature issue? I didn't see one).

Re #7219 I like what you've outlined (adding comment there too). It would be good to get someone from the pipelines group to express buy-in. I cannot contribute to either of these immediately, but can probably help within a few weeks.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kimwnasptd]

@juliusvonkohout aside from the part of exposing this PodDefault on each user namespace (discussion about Profile Controller and Metacontroller) I'd like to at least bring more awareness to users regarding this PodDefault.

Could you at least create a PR and add this in https://github.com/kubeflow/kubeflow/tree/master/components/admission-webhook/examples?

I'm looking into a restructure of the docs and would love to have a section of common use-cases of PodDefaults, and that's why I want to try and populate this dir as much as possible

[thesuperzapper]

@kimwnasptd the pipeline docs I added a while ago show an example of using a PodDefault to automatically mount the KFP ServiceAccount token and set KF_PIPELINES_SA_TOKEN_PATH.

Expand the "Full Kubeflow (from inside cluster)" section

[juliusvonkohout]

closed due to inactivity