Support array parameters

[jaxxstorm]

This may be a javascript question, so apologies for spamming your issues I'm trying to build a list of ingress rules based on a parameter, I have this in my params file

publicIngressHosts:
  - foo.example.com
  - bar.example

and then I tried this

const publicIngressHosts = std.param.Object('publicIngressHosts', {})

for (rule in publicIngressHosts) {
  rules.host = rule
}

const publicIngress = new k8s.extensions.v1beta1.Ingress('app-public-ingress', {
  metadata: {
    namespace: 'app',
    annotations: {
      'ingress.kubernetes.io/allow-http': "true",
      'ingress.kubernetes.io/ssl-redirect': "true",
      'kubernetes.io/ingress.class': 'public',
    },
  },
  spec: {
    rules: rules,
  }
})

and getting

Error: invalid type for param 'publicIngressHosts': cannot convert ["foo.example.com" "bar.example.com"] to Params
    at getParameter (@jkcfg/std/std_param.js:40:19)
    at Object (@jkcfg/std/std_param.js:56:12)
    at catalog.js:8:38```

is there a better way of doing this ?

[dlespiau]

You indeed found something that we're currently lacking in the std lib. Support array parameters directly. That's because I couldn't find a nice way to express typing Array<object>, Array<string>, etc. Maybe it's fine to just have std.param.Array and not try to type the content of the array.

Let's leave this issue open to track that.

As a work-around, what you can do is define the array as a sub-field for an object and define the parameter to be that wrapping object. Note how we define an ingress object that has a publicHosts field (our array!) define the parameter to point at ingress and then use ingress.publicHosts. I used some functional like construct (with Array.map) but your iterative approach works just as well. That gives:

$ cat params.yaml 
ingress:
  publicHosts:
  - domain: foo.example.com
    paths:
    - path: /foo
      service: foo.ns.svc
      port: 80
    - path: /bar
      service: bar.ns.svc
      port: 90
  - domain: bar.example.com
    paths:
    - path: /foo
      service: foo.ns.svc
      port: 80
    - path: /bar
      service: bar.ns.svc
      port: 90

$ cat array-param.js 
import * as std from '@jkcfg/std';
import * as k8s from '@jkcfg/kubernetes/api';

const ingress = std.param.Object('ingress');

const rulesFromParams = (ingress) => ingress.publicHosts.map(def => ({
  host: def.domain,
  http: {
    paths: def.paths.map(pathDef => ({
      path: pathDef.path,
      backend: {
        serviceName: pathDef.service,
        servicePort: pathDef.port,
      },
    })),
  },
}));

const publicIngress = new k8s.extensions.v1beta1.Ingress('app-public-ingress', {
  metadata: {
    namespace: 'app',
    annotations: {
      'ingress.kubernetes.io/allow-http': "true",
      'ingress.kubernetes.io/ssl-redirect': "true",
      'kubernetes.io/ingress.class': 'public',
    },
  },
  spec: {
    rules: rulesFromParams(ingress),
  }
})

std.write(publicIngress, '', { format: std.Format.YAML });

$ jk run -f params.yaml array-param.js 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/allow-http: "true"
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: public
  name: app-public-ingress
  namespace: app
spec:
  rules:
  - host: foo.example.com
    http:
      paths:
      - backend:
          serviceName: foo.ns.svc
          servicePort: 80
        path: /foo
      - backend:
          serviceName: bar.ns.svc
          servicePort: 90
        path: /bar
  - host: bar.example.com
    http:
      paths:
      - backend:
          serviceName: foo.ns.svc
          servicePort: 80
        path: /foo
      - backend:
          serviceName: bar.ns.svc
          servicePort: 90
        path: /bar

[jaxxstorm]

Holy hell, what an amazing answer! thank you so much!

[jaxxstorm]

It would be awesome if you could update the answer to be a more generic function, like for example if I have the following:

ingress:
  publicHosts:
    - foo.example.com
  internalHosts:
    - bar.example.net

[dlespiau]

I'm not too sure what internalHosts represents here. It is the DNS name to access the service from inside the cluster (so foo.ns.svc.cluster.local) and repeat the same paths than for the external DNS name?

[jaxxstorm]

we use two different ingress classes, one for external and one for internal addresses.

So what I want to be able to do is have a generic function that takes the ingress object, and loop through all the hosts in internalHosts (as an array) and all the hosts in publicHosts (as an array) and create ingress objects for them. So far I have

const httpConfig = {
  paths: {
    path: '/',
    backend: {
      serviceName: 'app-svc',
      servicePort: 8080,
    }
  }
}

const rulesFromParams = (ingress) => ingress.internalHosts.map(def => ({
  host: def,
  http: httpConfig,
}));

const publicIngress = new k8s.extensions.v1beta1.Ingress('app-public-ingress', {
  metadata: {
    namespace: 'app',
    annotations: {
      'ingress.kubernetes.io/allow-http': "true",
      'ingress.kubernetes.io/ssl-redirect': "true",
      'kubernetes.io/ingress.class': 'public',
    },
  },
  spec: {
    rules: rulesFromParams(ingress, internalIngress),
  }
})

but this means I need to write another function, rulesFromPublicParams which isn't very DRY. I guess I'm asking how I can make something like this

const rulesFromParams = (ingress) => ingress.<any map inside ingress>.map(def => ({
  host: def,
  http: httpConfig,
}));

but again, I don't know the javascript lingo for it

[dlespiau]

May not be what you asked for, an easier way is to give me input and expected output :)

$ cat params.yaml 
ingress:
  publicHosts:
  - domain: foo.example.com
    paths:
    - path: /foo
      service: foo.ns.svc
      port: 80
    - path: /bar
      service: bar.ns.svc
      port: 90
  privateHosts:
  - domain: bar.example.com
    paths:
    - path: /foo
      service: foo.ns.svc
      port: 80
    - path: /bar
      service: bar.ns.svc
      port: 90

import * as std from '@jkcfg/std';
import * as k8s from '@jkcfg/kubernetes/api';

const ingressDef = std.param.Object('ingress');

const rulesFromParams = (hostDef) => ({
  host: hostDef.domain,
  http: {
    paths: hostDef.paths.map(pathDef => ({
      path: pathDef.path,
      backend: {
        serviceName: pathDef.service,
        servicePort: pathDef.port,
      },
    })),
  },
});

// I parameterized the ingress creation as a function using an arrow function
// because it looks cool :) that's basically the same as a normal function.
//
// kind is either 'public' or 'private'
// host is the of host definition coming from the input parameters.
//
// function ingress(kind, host) {
//   ...
//    return new k8s.extensions.v1beta1.Ingress(....)
// }
//
// and
// 
// const ingress = (kind, hosts) => { ... }
//
// Yes, typing would be nice here, that's why we do support typescript as well :)
const ingress = (kind, host) => new k8s.extensions.v1beta1.Ingress(`app-public-${kind}`, {
  metadata: {
    namespace: 'app',
    annotations: {
      'ingress.kubernetes.io/allow-http': "true",
      'ingress.kubernetes.io/ssl-redirect': "true",
      'kubernetes.io/ingress.class': kind,
    },
  },
  spec: {
    rules: rulesFromParams(host),
  }
})

// Cycle through all definitions, create corresponding Ingress objects and put all the
// Ingress objects into the objects array. We use array destructing (...) to flatten the
// two arrays into the objects array.
const objects = [
  ...ingressDef.publicHosts.map(host => ingress('public', host)),
  ...ingressDef.privateHosts.map(host => ingress('private', host)),
]

std.write(objects, '', { format: std.Format.YAMLStream });

$ jk run -f params.yaml array-param.js 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/allow-http: "true"
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: public
  name: app-public-public
  namespace: app
spec:
  rules:
    host: foo.example.com
    http:
      paths:
      - backend:
          serviceName: foo.ns.svc
          servicePort: 80
        path: /foo
      - backend:
          serviceName: bar.ns.svc
          servicePort: 90
        path: /bar
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/allow-http: "true"
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: private
  name: app-public-private
  namespace: app
spec:
  rules:
    host: bar.example.com
    http:
      paths:
      - backend:
          serviceName: foo.ns.svc
          servicePort: 80
        path: /foo
      - backend:
          serviceName: bar.ns.svc
          servicePort: 90
        path: /bar

[jaxxstorm]

thank you SO much

[dlespiau]

I have the feeling you're writing a "micro service definition". You write a bit of yaml and you end up with all the k8s objects generated for you (Deployment, Service, public and private Ingress, ...) and maybe more (say one coud generate Dockerfiles, dashboards, prom operator custom resources for alerts, ...).

If that's the case, I'm quite interested by the objects you end up generating, that use case is one of the goals for this project!