k8s icon indicating copy to clipboard operation
k8s copied to clipboard

Jetstream limits are not parseable

Open c16a opened this issue 3 years ago • 2 comments

When there are Jetstream limits required to be set at account level, there is a parsing error that the server throws. This arises in the below line.

https://github.com/nats-io/k8s/blob/8a45fa066fba7c9469ff6d1e329dec0e5113ec7e/helm/charts/nats/templates/configmap.yaml#L527-L528

Since the toRawJson modifier is applied, the below example configuration fails to be parsed by the server. The nkeys authorisation in the below example works fine, so there is probably nothing wrong with the YAML itself. But it's just the `512M that's being templated as a string that's causing the issue.

nats:
  nats:
    jetstream:
      enabled: true
      memStorage:
        enabled: true
        size: 1G

  auth:
    enabled: true
    basic:
      accounts:
        {
          "A": {
            "jetstream": {
              "max_mem": 512M
            },
            "users": [
              {
                "nkey": "UBPNSTOECNRNSQLIGHT6FQIKUSKA637N2FBODGYQXBAJOCZ4264DME2J"
              }
              
            ]
          }
        }

When I run helm template --values values.yaml on the above, I get the below nats.conf rendered. Other fields are removed for brevity. Please note that the max_mem of the server-level jetstream is not double-quoted - it's just the account level setting.

jetstream {
  max_mem: 1G
}

authorization {
}

# This does not work
accounts:{
  "A":
    {
      "jetstream": {
        "max_mem": "512M"
      },
      "users":[{"nkey":"UBPNSTOECNRNSQLIGHT6FQIKUSKA637N2FBODGYQXBAJOCZ4264DME2J"}]
    }
}

If I manually override the account level max_mem and remove the quotes, server starts up successfully.

jetstream {
  max_mem: 1G
}

authorization {
}

# This works
accounts:{
  "A":
    {
      "jetstream": {
        "max_mem": 512M
      },
      "users":[{"nkey":"UBPNSTOECNRNSQLIGHT6FQIKUSKA637N2FBODGYQXBAJOCZ4264DME2J"}]
    }
}

The parsing error originated here

https://github.com/nats-io/nats-server/blob/71e2968ec176acb75de06c2e3fd4634c24057133/server/opts.go#L1612-L1652

c16a avatar Dec 12 '21 08:12 c16a

Thanks for the bug report, I think this might work if you use 512M as an integer instead like 512 * 1024 * 1024, but we should fix this to support human syntax. By the way I would suggest to use Gi and Mi as the units since those would be power of two.

wallyqs avatar Dec 13 '21 05:12 wallyqs

Thanks!

Just tried out your way - using 512 * 1024 * 2014 also gets quoted to a string which fails. 512Mi fails too.

c16a avatar Dec 13 '21 07:12 c16a

Supported in nats-1.0.0-beta.0 via this syntax: << 512Mi >>

That will un-quote the JSON string for parsing numbers and variables in a NATS Config file

caleblloyd avatar May 03 '23 19:05 caleblloyd