Conformance testing for Linux/ARM64

[odidev]

Hi Team,

I am working with k0s on Linux/ARM64 machine.

I can see that GitHub Action’s release.yml file runs Smoke tests for Linux/ARM64. Also, binaries are available for Linux/ARM64 platform.

However, release.yml file also runs conformance testing only for x64. May I know, do you plan to run those conformance tests for Linux/ARM64 platform as well?

If yes, I will be happy to contribute. Kindly share your opinion on the same.

[ncopa]

sounds like a good idea to me 👍

[odidev]

I have forked the k0s repo and tried using GitHub Action’s self-hosted runners for ARM64 to run conformance tests. But the jobs waited to be picked up by runners and finally failed.

So, I am following the CI commands on my AWS Ubuntu Linux/ARM64 instance to run conformance tests. Below are the steps:

- Installed terraform < https://www.terraform.io/downloads > 
- wget the source code [v1.23.3+k0s.0](https://github.com/k0sproject/k0s/releases/tag/v1.23.3%2Bk0s.0) 
- Downloaded ARM64 k0s binary in the root of the repo and renamed it as k0s. 
- cd k0s/inttest/conformance/terraform 
- Terraform init 
- eval `ssh-agent -s` 
- Export KOS_VER=v1.23.3+k0s.0 && export TF_VAR_k0s_version=${KOS_VER} 
- terraform apply -auto-approve 
- ./get_outputs.sh 
- ./deploy-k0s.sh 
- cd ../.. && make check-conformance 

Make command fails with the following logs:

$  make check-conformance 

mkdir -p bin 
curl -L --silent https://github.com/vmware-tanzu/sonobuoy/releases/download/v0.53.2/sonobuoy_0.53.2_linux_arm64.tar.gz | tar -C bin/ -zxv sonobuoy 
sonobuoy 
K0S_PATH=/home/odidev/K0S/k0s-1.23.3-k0s.0/k0s SONOBUOY_PATH=/home/ odidev /K0S/k0s-1.23.3-k0s.0/inttest/bin/sonobuoy \ 
        go test -count=1 -v -timeout 240m github.com/k0sproject/k0s/inttest/sonobuoy/ -run ^TestConformanceSuite -kubernetes-version=v1.23.3 
go: downloading github.com/avast/retry-go v2.6.0+incompatible 
go: downloading github.com/weaveworks/footloose v0.0.0-20200609124411-8f3df89ea188 
go: downloading github.com/google/go-cmp v0.5.5 
=== RUN   TestConformanceSuite 
=== RUN   TestConformanceSuite/TestConformance 
    vmsuite.go:73: failed to marshall terraform json: json: cannot unmarshal string into Go value of type common.TerraformMachineData 
    suite.go:63: test panicked: runtime error: index out of range [0] with length 0 
        goroutine 8 [running]: 
        runtime/debug.Stack() 
                /usr/local/go/src/runtime/debug/stack.go:24 +0x88 
        github.com/stretchr/testify/suite.failOnPanic(0x400041e4e0) 
                /home/ odidev /go/pkg/mod/github.com/stretchr/[email protected]/suite/suite.go:63 +0x48 
        panic({0x11c2240, 0x40000430e0}) 
                /usr/local/go/src/runtime/panic.go:1038 +0x224 
        github.com/k0sproject/k0s/inttest/common.(*VMSuite).GetConfig(0x4000108870) 
                /home/ odidev /K0S/k0s-1.23.3-k0s.0/inttest/common/vmsuite.go:78 +0x374 
        github.com/k0sproject/k0s/inttest/common.(*VMSuite).InitMainController(0x4000108870) 
                /home/ odidev /K0S/k0s-1.23.3-k0s.0/inttest/common/vmsuite.go:84 +0x38 
        github.com/k0sproject/k0s/inttest/sonobuoy.(*ConformanceSuite).TestConformance(0x4000108870) 
                /home/ odidev /K0S/k0s-1.23.3-k0s.0/inttest/sonobuoy/cncf_conformance_test.go:41 +0x34 
        reflect.Value.call({0x40004309c0, 0x400045e000, 0x13}, {0x129e87b, 0x4}, {0x4000073e78, 0x1, 0x1}) 
                /usr/local/go/src/reflect/value.go:543 +0x590 
        reflect.Value.Call({0x40004309c0, 0x400045e000, 0x13}, {0x4000073e78, 0x1, 0x1}) 
                /usr/local/go/src/reflect/value.go:339 +0x8c 
        github.com/stretchr/testify/suite.Run.func1(0x400041e4e0) 
                /home/ odidev /go/pkg/mod/github.com/stretchr/[email protected]/suite/suite.go:158 +0x41c 
        testing.tRunner(0x400041e4e0, 0x4000108900) 
                /usr/local/go/src/testing/testing.go:1259 +0x100 
        created by testing.(*T).Run 
                /usr/local/go/src/testing/testing.go:1306 +0x350 
--- FAIL: TestConformanceSuite (0.00s) 
    --- FAIL: TestConformanceSuite/TestConformance (0.00s) 
FAIL 
FAIL    github.com/k0sproject/k0s/inttest/sonobuoy      0.022s 
FAIL 
make: *** [Makefile:35: check-conformance] Error 1 

I have tried with master branch as well, but the error logs are same.
Also, same issue has been replicated on Ubuntu Linux/AMD64 AWS instance as well.

I could not find similar issue in terraform or in k0s GitHub repo.

Can you please provide some pointers on this?

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days

[odidev]

@twz123,

I again followed the above steps and replicated the same issue on both AMD64 and ARM64 platforms using k0s version 1.24.2.
Also, I am unable to run conformance tests using runners as Maxbuffer length exceeds.

Error: stderr maxBuffer length exceeded 

Can you please tell me if I am missing any step in the conformance testing, as mentioned above?

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days

[github-actions[bot]]

The issue is marked as stale since no activity has been recorded in 30 days