Technical checklist for joining the BinderHub Federation

[sgibson91]

As I'm working through adding the Turing to the Federation, I thought I'd create a checklist of the technical things that need to happen and maybe we can have a discussion on the minimum configuration needed too (e.g. what to do about grafana, matomo, etc.).

Please feel free to add to or expand the below!

Checklist

In a branch/fork of the mybinder.org-deploy repo:

• [ ] Add a YAML file to config with your BinderHub configuration.
  • Minimum requirements for this file:
    • [ ] Your hub URL
    • [ ] Your container registry URL
    • [ ] Your container registry token URL for OAuth
    • [ ] Ingress annotations for subdomains (Binder and JupyterHub, etc)
    • [ ] Resource requests
    • [ ] Extra Volumes
    • [ ] JupyterHub Resources
    • [ ] Single user limits
    • [ ] Proxy
    • [ ] Scheduling
    • [ ] grafana ingress and data sources
    • [ ] prometheus ingress
    • [ ] nginx-ingress config
  • Top level keys not required:
    • proxyPatches
    • federationRedirect
• [ ] Add a YAML file to secrets/config with secret configuration
  • Minimum requirements for this file:
    • [ ] Container registry username
    • [ ] Container registry password
    • [ ] API token
    • [ ] Secret token
• [ ] Add your cluster's authentication procedure to deploy.py
  • Minimum requirements for this file:
    • [ ] A function containing commands to access your cluster
    • [ ] An if/elif statement in main handling your cluster
    • [ ] Add a file with your authentication info to secrets
• [ ] Add an extra run of deploy.py to .travis.yml
  • Minimum requirements for this file:
    • [ ] If necessary, add an extra step to stage 1 installing any required CLIs
    • [ ] Add an extra step to stage 5 executing deployment on your cluster
    • [ ] Add an extra step to stage 5 verifying the deployment on your cluster

Questions

• Is a config/<deployment>/<deployment>_mybinder_org_ingress.yaml file necessary? What's its purpose?
  • Yes, this is how e.g. OVH redirects binder.mybinder.ovh to ovh.mybinder.org. If managing your own subdomain, this will be required.

Other important stuff

• RE deploy.py args:

  "release" is the name of the helm thing being deployed, and "cluster" is the name of the kubernetes cluster configuration. We have assumptions about what those two mean in a number of places, and it's definitely not clear everywhere. The only way to be really clear is to have no names re-used

• Does the new cluster use kube-dns or coredns? secrets/ban.py may have trouble.

[sgibson91]

Could we create a "federation template"? E.g. fill out these YAML files, the values that are already filled are what the BinderHub Federation expects for a consistent UX, blank fields are for your own keys/subdomains/other.

[manics]

There's some duplicated config between each binderhub.... is there enough in common to be worth having a common config which each binderhub overrides as necessary?

[sgibson91]

The easiest way to achieve that would be to put the common config in mybinder/values.yaml. It'd be interesting to know the rest of the team's opinion on that.

[manics]

I was thinking of a common config in this repo and take advantage of Helm's ability to merge multiple configs (latest takes precedence)

[yuvipanda]

Could we create a "federation template"? E.g. fill out these YAML files, the values that are already filled are what the BinderHub Federation expects for a consistent UX, blank fields are for your own keys/subdomains/other.

At UC Berkeley, we use an internal cookiecutter for this. See http://docs.datahub.berkeley.edu/en/latest/howto/new-hub.html for docs, and https://github.com/berkeley-dsep-infra/datahub/tree/staging/deployments/template for our template.

[sgibson91]

I was thinking of a common config in this repo and take advantage of Helm's ability to merge multiple configs (latest takes precedence)

Would that work given that the travis file updates a cluster at a time due to the different auth requirements?

Thanks @yuvipanda, that looks very cool!

[manics]

Would that work given that the travis file updates a cluster at a time due to the different auth requirements?

I image so... deploy.py already passes multiple configs to Helm: https://github.com/jupyterhub/mybinder.org-deploy/blob/cd7fb5b6839d0c17c42500eb7faec198337bdaac/deploy.py#L136-L138

[sgibson91]

Ah, yes sorry. My brain wasn't present when I wrote that!

[betatim]

What do you mean with "template for the federation"?

Looking at https://github.com/jupyterhub/mybinder.org-deploy/blob/master/config/prod.yaml I think an individual cluster probably doesn't need the following top level keys: static, proxyPatches, redirector, matomo, gcsProxy or federationRedirect.

I'd definitely run a prometheus instance (so we can add it as a data source to grafana.mybinder.org). A local grafana is probably a good idea as well but can get away without one.

Off the top of my head I don't know if we need https://github.com/jupyterhub/mybinder.org-deploy/blob/master/config/ovh/ovh_mybinder_org_ingress.yaml or not. I thought listing the hostname inn the ingress already defined in ovh.yaml would be enough.

[sgibson91]

What do you mean with "template for the federation"?

I mean a blank config file that can just be filled in to add a new cluster to the federation. It contains only the necessary keys required. Something like:

binderhub:
  config:
    hub_url: PUT_YOUR_JUPYTERHUB_URL_HERE

  registry:
    url: PUT_YOUR_REGISTRY_URL_HERE

etc...

I'm making a lot of assumptions around what's in the OVH deployment being "correct" and "necessary".

I'd definitely run a prometheus instance (so we can add it as a data source to grafana.mybinder.org). A local grafana is probably a good idea as well but can get away without one.

Is there a recommended guide on how to set this up?

[betatim]

You will get a basic grafana and prometheus via the mybinder chart as that chart depends on those two. Was more thinking out loud which of the top level keys should and shouldn't be copied/are relevant.

The best we have is looking at the OVH config and the gke config. There are differences which are part "gke" vs "ovh" and part "gke does stuff that other federation members don't do". Which is annoying but :-/

[sgibson91]

Yeah, I'd be happy documenting those differences somewhere for the next federation member :)

[sgibson91]

One action to take would be to refactor the deployment configs so we have one prod.yaml that is applied to every member and then have a gke.yaml, turing.yaml, ovh.yaml with their specific overrides.

(paraphrasing a gitter chat with @minrk)

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/how-can-i-support-mybinder-org/4692/4