gokey test suite takes hours on mips

[bdrung]

The Debian package gokey 0.1.0-1 executes the test suite on build and takes normally a few minutes, but it takes hours on mips (running into a timeout for mipsel):

arch	gokey / s	gokey/rsa / s	total / s	total / min
amd64	62,699	9,48	72	1,2
arm64	55,506	14,007	70	1,2
armel	794,092	232,153	1026	17,1
armhf	209,169	55,441	265	4,4
i386	137,309	35,016	172	2,9
mips	6315,264	1370,946	7686	128,1
mips64el	4239,958	991,576	5232	87,2
mipsel	timeout	timeout
ppc64	138,461	36,532	175	2,9
ppc64el	40,098	10,276	50	0,8
s390x	68,397	14,412	83	1,4
sparc64	324,662	71,095	396	6,6

Source: https://buildd.debian.org/status/package.php?p=gokey

Can the test suite be made faster for mips?

[ignatk]

Do mips builds run on real hardware or under some emulator? Go's goroutines are known not to be friendly with qemu-user emulators - we had similar issues when running arm64 builds in an emulator.

[bdrung]

I just confirmed: The mips* buildds are bare metal. So you can exclude emulators from the equation. Even if the maps buildds are not that fast than amd64, they should not be five times or more slower than armel (which belongs to the slower architectures).

[ignatk]

Yes, agree. The issue we had with the emulators - golang "multitasking" implementation hanged there (making the builds either crash or MUCH slower). When we tried a workaround, like pinning the build process to a single CPU with taskset - it worked normally (was a bit slower due to emulation, but not unreasonably slow).

The other thing to check is the status of entropy in the OS RNG. Golang uses getrandom system call on systems, which have it, which will block initially, until the entropy pool gets initialised.

I'm trying to set up a mipsel VM in the meantime to check it.

[bdrung]

According to https://lists.debian.org/debian-wb-team/2019/07/msg00006.html, the build host has a decent amount of entropy.

[ignatk]

I've setup a Debian mipsel installation in QEMU and installed Go from buster repositories:

ignat@debmips:~$ uname -a
Linux debmips 4.9.0-9-4kc-malta #1 Debian 4.9.168-1 (2019-04-12) mips GNU/Linux

So far I can see generating RSA keys with Go in general is unreasonably slow. A simple standalone Go program, which generates 4096 bit RSA key:

package main

import (
        "crypto/rand"
        "crypto/rsa"
        "fmt"
)

func main() {
        _, e := rsa.GenerateKey(rand.Reader, 4096)
        fmt.Println(e)
}

x64:

ignat@36com10:~$ time go run keygen.go
<nil>

real	0m0.971s
user	0m0.998s
sys	0m0.110s

mipsel:

ignat@debmips:~$ time go run keygen.go
<nil>

real    16m32.847s
user    11m8.760s
sys     4m28.340s

Even built-in Go RSA tests (which generate 1024 bit key according to the source code) x64:

ignat@36com10:~$ go test -v -run TestKeyGeneration crypto/rsa
=== RUN   TestKeyGeneration
--- PASS: TestKeyGeneration (0.02s)
PASS
ok  	crypto/rsa

mipsel:

ignat@debmips:~$ go test -v -run=TestKeyGeneration crypto/rsa
=== RUN   TestKeyGeneration
--- PASS: TestKeyGeneration (32.77s)
PASS
ok      crypto/rsa

I also tried to cross-compile keygen.go to MIPS on x64 using upstream Go 1.12, but it is still slow:

ignat@debmips:~$ time ./keygen
<nil>

real    23m33.520s
user    16m22.312s
sys     7m7.276s

Given that gokey tests "generate" up to 7 keys per key type - several hours for tests is quite expected. Not sure what to advise here other than disabling at least RSA 4096 on MIPS.

[ignatk]

arm64 used to have a similar issue: https://go-review.googlesource.com/c/go/+/76270

[bdrung]

Can you open an golang upstream bug report to speed up mips?

[ignatk]

Created https://github.com/golang/go/issues/33224

[ignatk]

still an issue for go 1.18