fluent-operator icon indicating copy to clipboard operation
fluent-operator copied to clipboard

Add bearer token auth for loki

Open raynay-r opened this issue 1 year ago • 5 comments

What this PR does / why we need it:

I was searching for an alternative of the openshift-logging-operator. None of the available operators supported bearer token authentication for the Loki output. Since bearer authentication is very useful together wiith Loki and the openshift oauth proxy I decided to make this contribution.

Which issue(s) this PR fixes:

Fixes #1221 (Not my ticket, by coincidence someone else seems to be missing this feature)

Does this PR introduced a user-facing change?

 * [Fluent-Bit] Additional Parameter "bearerToken" for (Cluster) Output of type Loki
 * [Fluent-Bit] Additional Parameter "uri" for (Cluster) Output of type Loki
 * [Fluentd] Additional Parameter "bearerTokenFile" for ClusterOutput of type Loki

Additional documentation, usage docs, etc.:

- [Usage]: https://docs.fluentbit.io/manual/pipeline/outputs/loki
- [Other doc] https://github.com/fluent/fluent-operator/blob/master/docs/plugins/fluentbit/output/loki.md
- [Usage] https://github.com/fluent/fluent-operator/blob/master/docs/plugins/fluentd/output/loki.md
- [Other doc] https://docs.fluentbit.io/manual/pipeline/outputs/loki

raynay-r avatar Jun 28 '24 21:06 raynay-r

I had to update the controller-gen version, otherwise the generation step would segfault. This caused a lot of changes. I am not sure how this should be handled.

raynay-r avatar Jun 28 '24 21:06 raynay-r

@raynay-r Thanks so much for you first contribution. But upgrade the contrller-gen version might not be a good practice for this MR. Can you please try to fix segfault? Feel free to share the problem you have when generating code here. Let us know the details so that we can help you.

cw-Guo avatar Jul 01 '24 02:07 cw-Guo

image conflicts need to be resolved @raynay-r

cc @wanjunlei

benjaminhuo avatar Jul 01 '24 06:07 benjaminhuo

Here is the output I get running make build on the main branch. You can also see in the output that I am using go 1.22.3. Is the go version maybe not compatible with the controller-gen version?

❯ go version
go version go1.22.3 linux/amd64
❯ make build
go get k8s.io/[email protected]
go mod download
/home/rene/Projects/fluent-operator/main/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa054af]

goroutine 354 [running]:
go/types.(*Checker).handleBailout(0xc0004a5400, 0xc001b35d40)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/check.go:367 +0x88
panic({0xbbf300?, 0x12ab8a0?})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/runtime/panic.go:770 +0x132
go/types.(*StdSizes).Sizeof(0x0, {0xdb6b18, 0x12b4040})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/sizes.go:228 +0x30f
go/types.(*Config).sizeof(...)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/sizes.go:333
go/types.representableConst.func1({0xdb6b18?, 0x12b4040?})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/const.go:76 +0x9e
go/types.representableConst({0xdbcdf0, 0x1280200}, 0xc0004a5400, 0x12b4040, 0x0)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/const.go:92 +0x192
go/types.(*Checker).arrayLength(0xc0004a5400, {0xdbb140, 0xc001844b60?})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:510 +0x2d3
go/types.(*Checker).typInternal(0xc0004a5400, {0xdb9760, 0xc001ccbe60}, 0x0)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:299 +0x49d
go/types.(*Checker).definedType(0xc0004a5400, {0xdb9760, 0xc001ccbe60}, 0xc001b35328?)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:180 +0x37
go/types.(*Checker).varType(0xc0004a5400, {0xdb9760, 0xc001ccbe60})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:145 +0x25
go/types.(*Checker).structType(0xc0004a5400, 0xc001d056b0, 0xc001d056b0?)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/struct.go:113 +0x19f
go/types.(*Checker).typInternal(0xc0004a5400, {0xdb96d0, 0xc001cce408}, 0xc0012c3db0)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:316 +0x1345
go/types.(*Checker).definedType(0xc0004a5400, {0xdb96d0, 0xc001cce408}, 0xc8815d?)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/typexpr.go:180 +0x37
go/types.(*Checker).typeDecl(0xc0004a5400, 0xc0012c3db0, 0xc001cd6580, 0x0)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/decl.go:615 +0x44d
go/types.(*Checker).objDecl(0xc0004a5400, {0xdc23e0, 0xc0012c3db0}, 0x0)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/decl.go:197 +0xa7f
go/types.(*Checker).packageObjects(0xc0004a5400)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/resolver.go:681 +0x425
go/types.(*Checker).checkFiles(0xc0004a5400, {0xc001527a40, 0x3, 0x3})
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/check.go:408 +0x1a5
go/types.(*Checker).Files(...)
        /nix/store/vz8d6wmfcf38l3h3vymwqr6c5zxp5jmp-go-1.22.3/share/go/src/go/types/check.go:372
sigs.k8s.io/controller-tools/pkg/loader.(*loader).typeCheck(0xc000363080, 0xc0007e65c0)
        /home/rene/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/loader/loader.go:286 +0x36a
sigs.k8s.io/controller-tools/pkg/loader.(*Package).NeedTypesInfo(0xc0007e65c0)
        /home/rene/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/loader/loader.go:99 +0x39
sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check(0xc0008c1bc0, 0xc0007e65c0)
        /home/rene/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/loader/refs.go:268 +0x2b7
sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check.func1(0x4f?)
        /home/rene/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/loader/refs.go:262 +0x53
created by sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check in goroutine 149
        /home/rene/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/loader/refs.go:260 +0x1c5
make: *** [Makefile:56: generate] Error 2

raynay-r avatar Jul 01 '24 07:07 raynay-r

Here is the output I get running make build on the main branch. You can also see in the output that I am using go 1.22.3. Is the go version maybe not compatible with the controller-gen version?

it's possible. Mine is 1.22.0, and it works well. go version go1.22.0 darwin/arm64

cw-Guo avatar Jul 01 '24 20:07 cw-Guo