Cannot configure blackhole route: "unknown field `route-type`"

[nirs]

What happened:

Install release 0.80.1 on OpenShift 4.14.3 based on https://github.com/nmstate/kubernetes-nmstate/releases/tag/v0.80.1

Applied this policy:

kind: NodeNetworkConfigurationPolicy
metadata:
  name: blackhole-perf2
spec:
  desiredState:
    routes:
      config:
        - destination: 1.2.3.10/32
          route-type: blackhole

route-type was added here: https://github.com/nmstate/nmstate/pull/2453

Policy failed to configure:

apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationEnactment
metadata:
  creationTimestamp: "2023-12-14T01:36:13Z"
  generation: 1
  labels:
    nmstate.io/node: perf1-d8zsg-master-0
    nmstate.io/policy: blackhole-perf2
  name: perf1-d8zsg-master-0.blackhole-perf2
  ownerReferences:
  - apiVersion: v1
    kind: Node
    name: perf1-d8zsg-master-0
    uid: c0caa7a6-8d9b-45e2-87d9-21c004b18467
  resourceVersion: "10450770"
  uid: aac91a33-ab3b-49c9-96ce-71ccc6bcac02
status:
  conditions:
  - lastHeartbeatTime: "2023-12-14T01:37:12Z"
    lastTransitionTime: "2023-12-14T01:37:12Z"
    reason: FailedToConfigure
    status: "False"
    type: Progressing
  - lastHeartbeatTime: "2023-12-14T01:37:12Z"
    lastTransitionTime: "2023-12-14T01:37:12Z"
    message: |-
      error reconciling NodeNetworkConfigurationPolicy on node perf1-d8zsg-master-0 at desired state apply: "",
       failed to execute nmstatectl set --no-commit --timeout 480: 'exit status 1' '' 'Using 'set' is deprecated, use 'apply' instead.
      Provide file is not valid NetworkState or NetworkPolicy: unknown field `route-type`, expected one of `state`, `destination`, `next-hop-interface`, `next-hop-address`, `metric`, `table-id`, `weight`
      '
    reason: FailedToConfigure
    status: "True"
    type: Failing
  - lastHeartbeatTime: "2023-12-14T01:37:12Z"
    lastTransitionTime: "2023-12-14T01:37:12Z"
    reason: FailedToConfigure
    status: "False"
    type: Available
  - lastHeartbeatTime: "2023-12-14T01:37:12Z"
    lastTransitionTime: "2023-12-14T01:37:11Z"
    reason: FailedToConfigure
    status: "False"
    type: Pending
  - lastHeartbeatTime: "2023-12-14T01:37:12Z"
    lastTransitionTime: "2023-12-14T01:37:11Z"
    reason: SuccessfullyConfigured
    status: "False"
    type: Aborted
  desiredState:
    routes:
      config:
      - destination: 1.2.3.10/32
        route-type: blackhole
  desiredStateMetaInfo: {}
  policyGeneration: 2

What you expected to happen:

blackhole route configured.

How to reproduce it (as minimally and precisely as possible):

See what happened

Anything else we need to know?:

Environment:

• NetworkManager version (use nmcli --version)

# nmcli --version
nmcli tool, version 1.42.2-8.el9_2

• Kubernetes version (use kubectl version):

$ oc version
Client Version: 4.13.4
Kustomize Version: v4.5.7
Server Version: 4.14.3
Kubernetes Version: v1.27.6+b49f9d1

• OS (e.g. from /etc/os-release):

# cat /etc/os-release 
NAME="Red Hat Enterprise Linux CoreOS"
ID="rhcos"
ID_LIKE="rhel fedora"
VERSION="414.92.202311150705-0"
VERSION_ID="4.14"
VARIANT="CoreOS"
VARIANT_ID=coreos
PLATFORM_ID="platform:el9"
PRETTY_NAME="Red Hat Enterprise Linux CoreOS 414.92.202311150705-0 (Plow)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:9::coreos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://docs.openshift.com/container-platform/4.14/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
REDHAT_BUGZILLA_PRODUCT_VERSION="4.14"
REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
REDHAT_SUPPORT_PRODUCT_VERSION="4.14"
OPENSHIFT_VERSION="4.14"
RHEL_VERSION="9.2"
OSTREE_VERSION="414.92.202311150705-0"

---

I guess route-type is not included in the latest release?

Why I want it: replace the manual ip route on every node here: https://github.com/nirs/oc-blackhole

[qinqon]

@nirs can you dump a NNS so we can see the nmstate version ?

[qinqon]

@cathay4t is this configuration supported at RHEL 9.2 ?

    routes:
      config:
        - destination: 1.2.3.10/32
          route-type: blackhole

[nirs]

@nirs can you dump a NNS so we can see the nmstate version ?

I think I already uninstalled nmstate on this system, but the version was this: https://github.com/nmstate/kubernetes-nmstate/releases/tag/v0.80.1

[maiqueb]

@cathay4t is this configuration supported at RHEL 9.2 ?

    routes:
      config:
        - destination: 1.2.3.10/32
          route-type: blackhole

@nirs do you know if this is working in nmstate ?

@qinqon assuming this setting is exposed by nmstate, is anything holding this feature back ? Seems a reasonable and simple ask.

[nirs]

@maiqueb I assume this works in nmstate based on https://github.com/nmstate/nmstate/pull/2453 merged Nov 24, 2023.

[qinqon]

@nirs is https://github.com/nmstate/kubernetes-nmstate/releases/tag/v0.81.0 working for you ?

It has nmstateversion 2.2.22

# nmstatectl version
nmstatectl 2.2.22

From what I see it was merged on 2.2.20 https://github.com/nmstate/nmstate/commit/f8c57a88fe05ed33fa4e8dddcb7c20957c94211a

[nirs]

@nirs is https://github.com/nmstate/kubernetes-nmstate/releases/tag/v0.81.0 working for you ?

Thanks, will test it.