tinygo-org
MIPS on Linux support
It seems that tinygo doesn't not support MIPS based embedded devices out of the box, although LLVM does support the following:
mips - MIPS (32-bit big endian)
mips64 - MIPS (64-bit big endian)
mips64el - MIPS (64-bit little endian)
mipsel - MIPS (32-bit little endian)
My intent is to execute the tinygo compiled binary within a mipsel linux box, no bare metal support, so I assume that's feasible considering llvm support. This would be extremely useful for routers with a limited set of storage and ram.
As far as I can tell, there are only a few things required to add support for such a target:
- a runtime arch file - provides the equivalent GOARCH value, bitness of the cpu, etc. (example here)
- a mapping from the LLVM arch name to the GOARCH name here
- helper program selection guesses here
I can't think of anywhere else where changes would be necessary. After the appropriate changes are added, you should be able to build for MIPS by supplying an appropriate LLVM target to the -target flag.
@deadprogram This issue seems to be regarding a Linux target, rather than baremetal.
Is there specific MIPS hardware you are looking to support?
Yeah, It would be great to have bare metal support for mips24kc out of the box since that's basically used by most routers. I'll try tinkering around, might learn something from it.
UPDATE:
It seems that after manually building LLVM as mentioned in Getting Started doesn't contain mips arch as a target
LLVM (http://llvm.org/):
LLVM version 10.0.1
Optimized build.
Default target: x86_64-unknown-linux-gnu
Host CPU: skylake
Registered Targets:
aarch64 - AArch64 (little endian)
aarch64_32 - AArch64 (little endian ILP32)
aarch64_be - AArch64 (big endian)
arm - ARM
arm64 - ARM64 (little endian)
arm64_32 - ARM64 (little endian ILP32)
armeb - ARM (big endian)
avr - Atmel AVR Microcontroller
riscv32 - 32-bit RISC-V
riscv64 - 64-bit RISC-V
thumb - Thumb
thumbeb - Thumb (big endian)
wasm32 - WebAssembly 32-bit
wasm64 - WebAssembly 64-bit
x86 - 32-bit X86: Pentium-Pro and above
x86-64 - 64-bit X86: EM64T and AMD64
Yeah, It would be great to have bare metal support for mips24kc out of the box since that's basically used by most routers.
My intent is to execute the tinygo compiled binary within a mipsel linux box, no bare metal support, so I assume that's feasible considering llvm support.
Are you interested in baremetal or Linux support? The two are quite different. Running under Linux would be relatively easy, needing basically the changes that @jaddr2line mentioned above. Baremetal support will take a lot more work and will usually be very chip specific. To get an idea of what is needed, see this PR (but note that quite a few things changed since then to avoid duplicated code).
It seems that after manually building LLVM as mentioned in Getting Started doesn't contain mips arch as a target
Yes, you need to modify this line to add Mips to the enabled targets: https://github.com/tinygo-org/tinygo/blob/a9ba6ebad9ad85cacd9674a1af9229489a811f68/Makefile#L157
It may be easier to instead use a prebuilt version of LLVM, which you can easily install on MacOS and most Linux distributions. It will have Mips already included as a target (no need to rebuild LLVM).
Yes, you need to modify this line to add Mips to the enabled targets:
Thank you for the tip, I was already digging inside, getting that little mipsy mips support in LLVM 😄
Are you interested in baremetal or Linux support?
I am honestly interested in having it work in both Linux and bare metal but I do understand that bringing bare metal support is a bit out of my skill scope, but I am very eager to learn and contribute
UPDATE:
Well, after some tinkering around, I was successfully able to compile mipsel targeted binaries (statically linked) and as well as executing them
Here are the file size differences between golang compiled binaries and tinygo:
/opt/home/admin # ls -l
-rwxr-xr-x 1 admin root 514908 Apr 28 06:33 hello_mipsel_tinygo
-rwxr-xr-x 1 admin root 5704 Apr 28 06:17 hello_mipsel_tinygo_dynlink
-rwxr-xr-x 1 admin root 851968 Apr 28 06:14 hello_mipsle_go
package main
func main() {
print("HELLO\n")
}
Note: the binaries were compiled with omitted DWARF debugging information as well as symbol table generation using -ldflags "-s -w" to achieve the lowest sizes without using upx.
a brief ldd look into the dynamically linked binary:
/opt/home/admin # ldd hello_mipsel_tinygo_dynlink
checking sub-depends for 'not found'
libc.so.6 => not found (0x00000000)
/lib/ld.so.1 => /lib/ld.so.1 (0x00000000)
What's interesting here is the produced (by default) dynamically linked binary that is small in size albeit depending on libc.so.6 (glibC) which is missing by default on OpenWRT (muslC), so in order to fix this, an openwrt musl toolchain should be used but this out of the scope of this issue but I will eventually document that as soon as I'm finished. This should allow the OpenWRT community to adopt tinygo.
I'm also interrested on compiling for MTK7628NN with Linux support. With Go I select GOARCH=mipsle GOMIPS=softfloat. It's possible with TinyGo ? Do you have any documents ? Thanks
I'm also interrested on compiling for MTK7628NN with Linux support. With Go I select GOARCH=mipsle GOMIPS=softfloat. It's possible with TinyGo ? Do you have any documents ? Thanks
Yes, it's possible and beautiful, I'm currently working on it, will have a PR and a documentation published soon.
UPDATE:
Currently the compilation is spewing a warning concerning wrong floating point setting but the binary is functioning properly, yet to figure out where to put the -msoft-float flag in tinygo/llvm given that the musl toolchain already uses -msoft-float
/home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: warning: /tmp/tinygo498244004/main uses -msoft-float (set by /home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/lib/crt1.o), /tmp/tinygo498244004/main.o uses -mhard-float
UPDATE:
Seems like -msoft-float flag is being ignored by llvm/clang, but considering that the binaries are working properly, it isn't critical atm.
UPDATE:
After having configured both the mips and mipsel toolchains, I went ahead and tried to compile a sample golang code using the fmt library, the compiler spewed the following:
../tinygo/tinygo build -target "mipsel-openwrt-linux" -o hello main.go
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:310:21: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1716:23: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1048:21: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:732:22: unknown GOOS/GOARCH for syscall: linux/mipsle
../../../usr/lib/go-1.13/src/syscall/zsyscall_linux_mipsle.go:1005:22: unknown GOOS/GOARCH for syscall: linux/mipsle
I figured that compiler/syscall.go needs some work as well in order for the go libraries to compile and by the looks of it, it became much more challenging..
UPDATE:
After hours of thorough studying of mips specs and docs, I have found out that the /o32 ABI only has 4 argument registers which is less than syscall6 arg length 🤔... additionally, the syscall manpage mentions that the syscall convention passes arguments 5+ through 8 on the user stack but that requires a different implementation in gollvm(?).. I am currently limited by my lack of knowledge in this area so any input would be appreciated! 😄
UPDATE:
I had no luck in assembling a working solution so far so it's a roadblock at the moment, basically everything works but syscalls
Currently the compilation is spewing a warning concerning wrong floating point setting but the binary is functioning properly, yet to figure out where to put the -msoft-float flag in tinygo/llvm given that the musl toolchain already uses -msoft-float
/home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: warning: /tmp/tinygo498244004/main uses -msoft-float (set by /home/adam/toolchain/toolchain-mips_24kc_gcc-8.4.0_musl/bin/../lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/lib/crt1.o), /tmp/tinygo498244004/main.o uses -mhard-float
You probably need to set a flag (-soft-float maybe?) in the BuildTags property. This list of flags is for how TinyGo compiles the binary, and is independent of CFlags (which is used to build the C parts). You can figure out the needed flag with the Clang -v flag (look for -target-fetature).
While some programs may work, I do not think programs that have float operations will work without this change. Try running testdata/float.go for example.
After hours of thorough studying of mips specs and docs, I have found out that the /o32 ABI only has 4 argument registers which is less than syscall6 arg length thinking... additionally, the syscall manpage mentions that the syscall convention passes arguments 5+ through 8 on the user stack but that requires a different implementation in gollvm(?).. I am currently limited by my lack of knowledge in this area so any input would be appreciated! smile
Ah really, they are passing syscall arguments on the stack? That complicates things a bit.
I think the easiest option is to write assembly functions that do the syscall itself. Calling those functions will make sure LLVM puts those values on the stack in the right place. There is a bit of a complication however with the success/error value (in $a3) so the function may need to do a bit more than just call syscall and return.
Source: https://www.linux-mips.org/wiki/Syscall
Any update on this?
Unfortunately not, I have decided that writing code for MIPS based embedded devices is much better done via C/C++. I haven't had the time to figure it out, if one could provide some examples on passing 5+ arguments via stack for syscalls then I could probably try it out again.
Is there no reference value for the system call of the golang source code Go\src\syscall. Although C/C++ is good, cross-compilation is troublesome, and development efficiency is not good. Rust solves the cross-compilation problem well.
Any update on this? This will really be very beneficial for running go code on routers.
We could do it, but I haven't seen a lot of demand (except for this issue). I'm somewhat worried about the maintenance burden this will cause, because it would add yet another architecture. We currently support arm, arm64, 386, amd64, avr, xtensa, and wasm. But if there is enough demand or a good use case, this would certainly be interesting to support.
Writing networking and server code for routers with go should be a very interesting use case. The main reason why go isn't even considered is because of the huge binary size, but if tiny go can be made to work with these routers, it will be a very good alternative to C/C++ for many cases.
Example: https://mender.io/blog/why-did-we-choose-golang-over-c https://stackoverflow.blog/2022/04/04/comparing-go-vs-c-in-embedded-applications/
We could do it, but I haven't seen a lot of demand (except for this issue). I'm somewhat worried about the maintenance burden this will cause, because it would add yet another architecture. We currently support arm, arm64, 386, amd64, avr, xtensa, and wasm. But if there is enough demand or a good use case, this would certainly be interesting to support.
I think mips is much more important than 386 and amd64. Mips are only embedded devices such as routing. On the contrary, 386 amd64 devices have huge space and can use official compilers.
You could target wasm and then run wasm on mips...
On Wed, 21 Sept 2022, 6:05 am dosgo, @.***> wrote:
We could do it, but I haven't seen a lot of demand (except for this issue). I'm somewhat worried about the maintenance burden this will cause, because it would add yet another architecture. We currently support arm, arm64, 386, amd64, avr, xtensa, and wasm. But if there is enough demand or a good use case, this would certainly be interesting to support.
I think mips is much more important than 386 and amd64. Mips are only embedded devices such as routing. On the contrary, 386 amd64 devices have huge space and can use official compilers.
— Reply to this email directly, view it on GitHub https://github.com/tinygo-org/tinygo/issues/1075#issuecomment-1253148196, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALP3FCFUTUKY2C764GXDX3V7J3OJANCNFSM4MRREVEA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
So if we would add support for MIPS, which would it be? Apparently there are 4 MIPS architecture variants that we could support:
mips - MIPS (32-bit big endian) mips64 - MIPS (64-bit big endian) mips64el - MIPS (64-bit little endian) mipsel - MIPS (32-bit little endian)
I don't think 64-bit MIPS needs TinyGo, so that leaves it with mips and mipsel. I don't know which of the two architectures (big or little endian) is used more widely, but I would expect a little-endian system to be much easier to add support for since all architectures currently supported by TinyGo are little endian.
The mips endian seems to be OS dependent! https://openwrt.org/docs/platforms/start is little endian more.
We (Jangala - a charity that makes internet access systems for schools and clinics for marginalised communities around the world) would love to see this added.
We currently use Lua for our codebase but are thinking of using TinyGo on our MCUs and would love to use it on our Arm64 and MIPS based boards that are running OpenWrt/Linux. The Ath79 target (a hugely popular router and AP chipset family) is big endian.
If this became available it could be hugely popular amongst the OpenWrt community (who currently don't see Go as a viable option). For example, the Mender IoT remote package that @mutexstream references generates a 2MB+ binary which is prohibitive on this kind of target.
Wasm seems interesting but by necessity the code we write interacts with system libraries and the kernel directly, which I guess (not knowing a huge amount about it) might be more difficult with Wasm
I can look into adding support for MIPS. But again: which variant would be preferred or would actually be used in practice? There are four to pick from for initial support. "MIPS" is really more like 4 different architectures under a single name. (Note: big endian is probably going to be more difficult than little endian).
Just looking at the number of end user devices that would be hit, I'd probably suggest the following order of priority:
- mipseb (32 bit big endian)
- mipsel (32 bit little endian)
The rationale for this is that just the ath79 family alone represents probably hundreds of millions of target devices worldwide
I did some experimenting with MIPS support and managed to get very limited support working:
$ GOARCH=mips tinygo run -gc=leaking ./testdata/alias.go
x
apple
true
false
I have no plans currently to continue working on this, but if there's anybody interested feel free to complete support for MIPS. The branch is here: https://github.com/tinygo-org/tinygo/compare/dev...mips
What's left to do for full support? And do you have any docs or example pull requests that can help me learn how to add a new architecture?
What's left to do for full support?
I was stuck at syscall support. There are probably a few more things but that's a big one. To implement this, you need to understand how system calls work on MIPS (there are two calling conventions and I don't know which one is usually used) and implement it using inline assembly in compiler/syscall.go. There are a number of other architectures you can look at for help.
And do you have any docs or example pull requests that can help me learn how to add a new architecture?
Not really, except for looking at what's already there and the various MIPS documents. Examples: syscall (2), wiki, instruction set, etc.
I also need this to run go on openwrt router. I agree that Mips is more important than amd64 and i386. The only deterrent for not using go for embedded is the binary size. Performance-wise, go is very good alternative to C/C++.
Since netdev in #3704 is just around the corner, having a support for MIPS would open up a lot of possibilities for usage in routers (looking at you, OpenWRT guys :slightly_smiling_face:).
