trillian
trillian copied to clipboard
google
Add scaffolding to stand up Trillian on k8s.
Signed-off-by: Ville Aikas [email protected]
Fixes #2672
Here's the pieces for scaffolding and for createtree. I split them into two different directories because then applying just the scaffolding bits is easier and more well contained. Apply all the everythings :)
Createtree is a standalone job that makes creating trees in the k8s cluster easier and then because the entry is stuffed into configmap, it's easy to 'depend' on that for other jobs that may need it. For example, we have also a setup for spinning up a CTLog on top of this in sigstore, but wasn't sure if that might be of interest. I noticed that in README.md one of the steps after spinning up Trillian is to install CTLog on top of it.
https://github.com/sigstore/scaffolding/tree/main/config/ctlog
After creating the README-SCAFFOLDING.md in the examples dir, I noticed the PR template sez to add it to docs/ but I'm not sure if this belongs there since it's related to running tests, etc.
Lastly, the added kind test in github workflows only checks the configmap that the treeID was created, might be good to add some examples to test the grpc api as well, but figured that could be a followup if that makes sense.
Checklist
- [ ] I have updated the CHANGELOG.
- [ ] I have updated documentation accordingly (including the feature implementation matrix).
@roger2hk Thanks for the gcbrun.
trillian-pr-tests are failing but I can't see why, can somebody share what the problem is so I can try to fix it :) https://github.com/google/trillian/pull/2754/checks?check_run_id=6774522500
running golangci-lint
level=warning msg="[runner] Can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\""
level=error msg="Running error: 1 error occurred:\n\t* can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\"\n\n"
Thanks for the comments! Sorry for tardy response, was out on PTO, will pick this back up this week! @mhutchinson totes understood about life sometimes getting very busy :)
This is failing running the lint checks:
Already have image (with digest): gcr.io/trillian-opensource-ci/trillian_testbase
running gofmt
running goimports
running golangci-lint
level=warning msg="[runner] Can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\""
level=error msg="Running error: 1 error occurred:\n\t* can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\"\n\n"
https://github.com/golangci/golangci-lint/discussions/1920 may be relevant. Holler if you can't work this out and I'll take a deeper look.
This is still causing some problem with the linter. Fortunately it can be reproduced locally:
golangci-lint version
golangci-lint has version v1.47.3 built from (unknown, mod sum: "h1:ri7A2DgtFpxgqcMSsU3qIT0IBm/SCdYgXlvmJx4szUU=") on (unknown)
golangci-lint run
WARN [runner] Can't run linter goanalysis_metalinter: buildir: failed to load package v1: could not load export data: no export data for "k8s.io/client-go/applyconfigurations/meta/v1"
ERRO Running error: 1 error occurred:
* can't run linter goanalysis_metalinter: buildir: failed to load package v1: could not load export data: no export data for "k8s.io/client-go/applyconfigurations/meta/v1"
I think this is because client-go is min version go1.18: https://github.com/kubernetes/client-go/blob/master/go.mod
https://github.com/golangci/golangci-lint/discussions/1920#discussioncomment-2578691
edit: but then again we're using v0.23.8, which is go1.16: https://github.com/kubernetes/client-go/blob/v0.23.8/go.mod
Hah, I've just spent ages messing about with looking into the linters and go versions. The answer appears to be as simple as running go build to find the root cause:
go build ./...
# k8s.io/client-go/applyconfigurations/meta/v1
../../go/pkg/mod/k8s.io/[email protected]/applyconfigurations/meta/v1/unstructured.go:64:38: cannot use doc (variable of type *"github.com/googleapis/gnostic/openapiv2".Document) as type *"github.com/google/gnostic/openapiv2".Document in argument to proto.NewOpenAPIData
Root cause is probably https://github.com/kubernetes/client-go/issues/1084. @vaikas can I leave this with you?
Thank you so much @mhutchinson for digging into this. I'll keep looking at it, oh what joy! 🤣
Ok, added a replace to go.mod and now things are building:
vaikas@villes-mbp trillian % go mod tidy -go=1.17 -compat=1.17
vaikas@villes-mbp trillian % git status
On branch k8s-createtree
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: go.mod
modified: go.sum
no changes added to commit (use "git add" and/or "git commit -a")
vaikas@villes-mbp trillian % go build ./...
vaikas@villes-mbp trillian %
Let's try this now :)
Error is
Already have image (with digest): gcr.io/trillian-opensource-ci/trillian_testbase
running gofmt
running goimports
running golangci-lint
level=warning msg="[runner] Can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\""
level=error msg="Running error: 1 error occurred:\n\t* can't run linter goanalysis_metalinter: buildir: failed to load package version: could not load export data: no export data for \"sigs.k8s.io/release-utils/version\"\n\n"
Looks like a similar issue to the last one but with a different module. Oddly though, I can't reproduce this one locally with go build or golangci-lint. Will have a quick poke at this, but due to RL considerations I may need to leave this for @AlCutter to take over next week.
@vaikas if you want to revisit this PR, the cloudbuild results are now viewable so it would be easier to debug.
oooooh, yes, thank you for the pointer, I'll try to carve some time in the next couple of days :)
On Mon, Dec 12, 2022 at 10:30 PM Juan Antonio Osorio < @.***> wrote:
@vaikas https://github.com/vaikas if you want to revisit this PR, the cloudbuild results are now viewable so it would be easier to debug.
— Reply to this email directly, view it on GitHub https://github.com/google/trillian/pull/2754#issuecomment-1347810939, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWB45GGBHAOLK7P5B3TB6DWNAJX3ANCNFSM5YCUENJA . You are receiving this because you were mentioned.Message ID: @.***>
I'm back! Getting this PR in would be a great accomplishment for my return if you're up for it @vaikas :-)
Welcome back, happy new year!!!! Yes, I'll make time for it, this time I really mean it.
On Mon, Jan 9, 2023 at 2:04 PM Martin Hutchinson @.***> wrote:
I'm back! Getting this PR in would be a great accomplishment for my return if you're up for it @vaikas https://github.com/vaikas :-)
— Reply to this email directly, view it on GitHub https://github.com/google/trillian/pull/2754#issuecomment-1375524623, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWB45CMHJYWF46YALXJZCTWRP5EBANCNFSM5YCUENJA . You are receiving this because you were mentioned.Message ID: @.***>
@mhutchinson would you mind hitting the magic test button (/gcbrun) for me please? :)