Removing machine.install.extraKernelArgs doesn't remove them

[szinn]

Bug Report

Description

Add machine.install.extraKernelArgs section with logging specification.

Note exiting the editor indicates the change was applied, but actually you need to do a talosctl upgrade with the same installer version to actually apply the change.

Remove the machine.install.extraKernelArgs section and do the upgrade again. Expectation is that the logging entry is removed, however, logs still continue to be sent according to the previous setting

Also tried setting machine.install.extraKernelArgs to [] but same result, logging still is happening

Logs

Environment

• Talos version: [talosctl version --nodes <problematic nodes>] Client: Tag: v1.0.4 SHA: f6696063 Built: Go version: go1.17.7 OS/Arch: darwin/arm64 Server: NODE: 10.0.40.64 Tag: v1.0.4 SHA: f6696063 Built: Go version: go1.17.7 OS/Arch: linux/amd64 Enabled: RBAC

• Kubernetes version: [kubectl version --short] Client Version: v1.24.0 Kustomize Version: v4.5.4 Server Version: v1.23.6

• Platform: Proxmox

[smira]

Some kernel args are special - they persist from the boot kernel arguments. Which kernel arg specifically have you tried?

If that's the case, removing them from GRUB entry on boot is also required. We should need to find a better solution here.

[szinn]

specifically the one for logging:

machine: install: extraKernelArgs: - talos.logging.kernel=tcp://host:5044/

I think the expectation would be that at the end of the editmachineconfig and a talosctl upgrade that the grub entries would match what is in the machineconfig spec.

[smira]

it sounds like it should be like that, but there's also ISO/PXE boot environment kernel args, so things are more complicated a bit, and some kernel args are persisted from the boot environment. we need to improve this, but there's no "easy" fix now. specifying empty value for the arg should disable the feature

[neilmfrench]

Having the same issue

it sounds like it should be like that, but there's also ISO/PXE boot environment kernel args, so things are more complicated a bit, and some kernel args are persisted from the boot environment. we need to improve this, but there's no "easy" fix now. specifying empty value for the arg should disable the feature

What do you mean by empty value for the arg? I've tried removing extraKernelArgs entirely, setting it to [], and now: extraKernelArgs:

• talos.logging.kernel=""

None of the above have worked

[smira]

there is no easy fix at the moment for it.

one way around is to press e during boot to enter GRUB, remove the arg, boot, upgrade, this should get rid of that arg.