Compiling golang project for ish

[pigscanflyyyy]

Good day! I'm trying to compile a simple golang project to be executed using this ala-termux app. Is it possible to do that? How do i target this device achitecture? What toolchain do i use? Any tips? Thank you very much.

[ahmadnbl]

Since it's using Alpine Linux, you can install go with Alpine's package manager. Just simply run

# apk add go

I tried to install it on my iphone xs

Screenshoots

But idk I can't compile simple hello-world program in there. It takes forever when I invoke to run the code. Also I can't cancel it (by trigger Ctrl+C), and suddenly it become freezing.

Screenshoots

[emkey1]

The go language currently triggers some sort of race condition in iSH which causes it to lock up forever. It's easy to reproduce.

I dug into it a bit and it looked like iSH was waiting for a lock to clear. I'll see if I can dig up some more information.

[rjgonzale]

It happens here too, just go env is required to lock iSH. I do not know how to debug this but @emkey1 if you need help let me know.

[dozer47528]

same problem

[maxime-aknin]

Same here, tried a simple hello world but unfortunately go run main.go hangs forever and freezes the app 🤷‍♂️

[alonelucky]

I have same problem.

[YUX]

same here

[hbouhadji]

+1

[bqrs01]

Have the same problem.

[JasonS05]

A while back (a year ago maybe?) I could use go to compile or do anything else I might want to do with go and it worked fine but recently I tried to use go and it hanged. The go program I compiled a while back still works, but of course recompiling hangs.

[pyang55]

same problem too

[feyeleanor]

Also having the same problem.

[jmcarbo]

Compiling even the simplest golang program hangs the whole ish environment. I have tried both with the release version and the beta version.

[alexandervlpl]

Has anybody figured out how to get a go program to run?

Currently go build can't even download packages without freezing. I copied them over manually but still can't compile.

I also tried cross-compiling a statically linked binary for i386, but it crashes in iSH.

[JasonS05]

As I've mentioned before, already-compiled go programs work without issue (although now my mandelbrot program's output is buggy, but I think that's a terminal issue, not a go issue), but the go compiler freezes iSH when run. So, I decided to test other versions of the compiler to see if one works. The version installed by apk add go is go-1.12.12-r0 so I know that doesn't work, but just to be sure I uninstalled it and installed it again but this time by getting the apk file from wget "http://uk.alpinelinux.org/alpine/v3.10/community/x86/go-1.12.12-r0.apk" and then executing apk add ./go-1.12.12-r0.apk and as expected it behaved the same. I also tried newer versions using the same installation method, but every newer version failed with Error relocating /usr/bin/go: __nanosleep_time64: symbol not found when I tried to run it, so it looks like some kind of linking error. Maybe it needs a newer version of some system library, but I don't know what I need to update. After trying several of these newer versions and finally finding out it happens for even the next-newest version after the current one I decided to test the oldest available version, go-1.5.4-r0, and found that when I executed go version it just straight up crashed iSH, so I've pretty much lost hope in there being a version of go that works with the current iSH. Maybe somewhere between go-1.12.12-r0 and go-1.5.4-r0 there's a version that works, but I don't have the patience to look for it.

[alexandervlpl]

Currently iSH freezes on me even while running absolutely nothing, in foreground. I also tried sshd, git and other tools: everything runs for a bit then freezes or crashes the emulator. It just doesn't seem to be stable right now.

The lesson is: once your stuff runs reasonably well in iSH, back up the filesystem and avoid updating anything (first and foremost iSH itself). If you have an old device lying around, test updates there first. And remember: this can never be some magical iOS version of Termux (:heart:). If you really need this sort of thing to work reliably on iOS, you're out of luck at least until this nice emulator idea becomes a much bigger project. Kudos to @tbodt for trying though!

[JasonS05]

@alexandervlpl yeah it does seem rather unstable right now. Twice while messing around with trying to get go to work it just crashed in the middle of executing a command (once while unzipping a tarball, again while use apk to add a package). A year or two back when I used this app more it didn't seem to have any stability issues, but now it randomly crashes, golang freezes, and the terminal has issues. The terminal used to work great except that clear didn't clear the stuff above the top of the screen (which it still doesn't do) but now simple terminal things like printing -bash: gigiuvy: command not found don't happen reliably and can even cause the prompt for the next command to disappear unless I make the keyboard go away and then bring it back again. Similar issues have happened while editing a command where it looks like I'm editing one spot but actually I'm editing a different spot a couple characters away. And also that mandelbrot program I mentioned before used to print a beautiful mandelbrot with terminal color codes but now a significant portion of the lines are bugged and have large sections missing so that there's a large ugly black space after the end of the line since it didn't get to reach the edge of the screen. Also for the first time ever I played with using split-screen mode while waiting for a package to install and that completely broke the progress bar filling the screen with white and freezing any further progress of the download. So maybe something relating to that is the cause of some of these issues? I kind of wish I could downgrade iSH, but I don't know if there's any way.

[emkey1]

The only hard crash bug I'm aware of occurs when accessing /proc with something like top while doing something else. The root cause is an attempt to destroy a RW lock that has already been freed. If you can recompile iSH it can be worked around with the following code.

In sync.h, around line 135...

#ifdef JUSTLOG
    if (pthread_rwlock_destroy(&lock->l) != 0) printk("URGENT: pthread_rwlock_destroy error(PID: %d Process: %s) \n",current_pid(), current_comm());
#else
    if (pthread_rwlock_destroy(&lock->l) != 0) __builtin_trap();
#endif

At the end of log.c

char* current_comm() {
    if (current)
        return current->comm;
    return calloc(1, 1); 
}

You would also have to #define JUSTLOG of course.

I've also wrapped the other lock destroy functions in that section of sync.h, but I've never seen anything logged from them so I'm not including that code. It's fairly easy to figure out what to do if you want to do that.

[emkey1]

Just to be clear, this is NOT a fix. It flags the error condition and continues on. I do not know the root cause, just how to reproduce it and generally where it is in the code.

Having said that, I've had no additional issues with iSH with this code in place. It is no more, or less stable with the exception that it seldom if ever crashes when running with top active.

The only hard crash bug I'm aware of occurs when accessing /proc with something like top while doing something else. The root cause is an attempt to destroy a RW lock that has already been freed. If you can recompile iSH it can be worked around with the following code.

In sync.h, around line 135...

#ifdef JUSTLOG
    if (pthread_rwlock_destroy(&lock->l) != 0) printk("URGENT: pthread_rwlock_destroy error(PID: %d Process: %s) \n",current_pid(), current_comm());
#else
    if (pthread_rwlock_destroy(&lock->l) != 0) __builtin_trap();
#endif

At the end of log.c

char* current_comm() {
    if (current)
        return current->comm;
    return calloc(1, 1); 
}

You would also have to #define JUSTLOG of course.

I've also wrapped the other lock destroy functions in that section of sync.h, but I've never seen anything logged from them so I'm not including that code. It's fairly easy to figure out what to do if you want to do that.

[sohaha]

Looking forward to have a new progress

[simbadMarino]

having the same problem trying to compile with CGO enables

[lijh8]

iSH: 1.2.3
iOS: 15.4.1,  on iPhone Xs:

Can't build go source code file.

Is it because alpine needs: musl libc, 
but the go language needs: libgcc

# 
# apk add go
(1/13) Installing libgcc (10.3.1_git20210424-r2)
(2/13) Installing libstdc++ (10.3.1_git20210424-r2)
(3/13) Installing binutils (2.35.2-r2)
(4/13) Installing libgomp (10.3.1_git20210424-r2)
(5/13) Installing libatomic (10.3.1_git20210424-r2)
(6/13) Installing libgphobos (10.3.1_git20210424-r2)
(7/13) Installing gmp (6.2.1-r1)
(8/13) Installing isl22 (0.22-r0)
(9/13) Installing mpfr4 (4.1.0-r0)
(10/13) Installing mpc1 (1.2.1-r0)
(11/13) Installing gcc (10.3.1_git20210424-r2)
(12/13) Installing musl-dev (1.2.2-r3)
(13/13) Installing go (1.16.10-r0)
Executing busybox-1.33.1-r6.trigger
OK: 425 MiB in 27 packages
# 
# go version
go version go1.16.10 linux/386
# 
# pwd
/root/helloGo
# ls
go.mod    hello.go
# 
# cat hello.go 
package main

import "fmt"

func main() {
        fmt.Println("Hello")
}
#
# cat go.mod 
module hello

go 1.16
# 
# go run .

It does not build or run go source code files. nothing happens. just freeze here.

[emkey1]

I posted this to #dev on the discord as well. go does seem to work better on recent builds of iSH, but it still crashes after a small number of iterations building "Hello World".

I think I understand why Go causes iSH to have fits. From what I can tell, it launches multiple threads/sub processes to compile or run something as simple as "Hello World". These threads in turn communicate via a region of memory. They are constantly reading and writing to that shared region of memory.

Here's where things get complicated. iSH uses a relatively simple memory locking scheme that works well in most cases. Where it breaks down is when there are multiple threads trying to read and write to the same general region of memory. It is entirely possible in this scenario for one thread to initiate a write request only to have that region of memory deallocated by another thread before it can complete.

Wrapping all the memory related locking functions with an additional lock to keep things coherent leads to a different issue. The lock/unlock functions are blocking, meaning they will wait forever for access to a lock to become available. If they are wrapped with an additional layer of locking, that additional layer can easily deadlock.

I think this means that in an ideal world the iSH memory related lock/unlock functions (read_wrlock/read_wrunlock, etc) would be non blocking and thread safe. Figuring out how to get there is giving me a bit of a headache.

[ThomasChr]

Thanks for the thorough explaination. Very much appreciated!

[JasonS05]

I posted this to #dev on the discord as well. go does seem to work better on recent builds of iSH, but it still crashes after a small number of iterations building "Hello World".

I think I understand why Go causes iSH to have fits. From what I can tell, it launches multiple threads/sub processes to compile or run something as simple as "Hello World". These threads in turn communicate via a region of memory. They are constantly reading and writing to that shared region of memory.

Here's where things get complicated. iSH uses a relatively simple memory locking scheme that works well in most cases. Where it breaks down is when there are multiple threads trying to read and write to the same general region of memory. It is entirely possible in this scenario for one thread to initiate a write request only to have that region of memory deallocated by another thread before it can complete.

Wrapping all the memory related locking functions with an additional lock to keep things coherent leads to a different issue. The lock/unlock functions are blocking, meaning they will wait forever for access to a lock to become available. If they are wrapped with an additional layer of locking, that additional layer can easily deadlock.

I think this means that in an ideal world the iSH memory related lock/unlock functions (read_wrlock/read_wrunlock, etc) would be non blocking and thread safe. Figuring out how to get there is giving me a bit of a headache.

I do remember that golang used to work well before. It would occasionally crash with some error in the golang runtime (something about mspinners or something, I forget) that if googled would only point to the line of source code in the golang project that's responsible for printing that error, but most of the time my go program (a multithreaded mandelbrot) worked fine. So, if it worked at one point, it should be possible to go into the github history and find that point and see what it did differently and move that forward to the present code, right? Unless there have been incompatible changes since then.

[emkey1]

I

I posted this to #dev on the discord as well. go does seem to work better on recent builds of iSH, but it still crashes after a small number of iterations building "Hello World". I think I understand why Go causes iSH to have fits. From what I can tell, it launches multiple threads/sub processes to compile or run something as simple as "Hello World". These threads in turn communicate via a region of memory. They are constantly reading and writing to that shared region of memory. Here's where things get complicated. iSH uses a relatively simple memory locking scheme that works well in most cases. Where it breaks down is when there are multiple threads trying to read and write to the same general region of memory. It is entirely possible in this scenario for one thread to initiate a write request only to have that region of memory deallocated by another thread before it can complete. Wrapping all the memory related locking functions with an additional lock to keep things coherent leads to a different issue. The lock/unlock functions are blocking, meaning they will wait forever for access to a lock to become available. If they are wrapped with an additional layer of locking, that additional layer can easily deadlock. I think this means that in an ideal world the iSH memory related lock/unlock functions (read_wrlock/read_wrunlock, etc) would be non blocking and thread safe. Figuring out how to get there is giving me a bit of a headache.

I do remember that golang used to work well before. It would occasionally crash with some error in the golang runtime (something about mspinners or something, I forget) that if googled would only point to the line of source code in the golang project that's responsible for printing that error, but most of the time my go program (a multithreaded mandelbrot) worked fine. So, if it worked at one point, it should be possible to go into the github history and find that point and see what it did differently and move that forward to the present code, right? Unless there have been incompatible changes since then.

I tried going back at one point, but I wasn't clever enough to figure out what change had broken things. I'm rewriting the bulk of the internal locking code so that A, I'll understand it and B, I know who to blame for my troubles. :)

I'm doing this on my iSH-AOK fork BTW. It's very much a one step forward, two steps back kinda deal unfortunately.

[emkey1]

OK, new or additional theory. iSH may not do a perfect job of cleaning up when a task/thread exits. I think it's also the case that it will kill/cleanup an exiting task/thread when there are in flight operations like memory being accessed for instance.

It's really difficult to tell what is going on, both because iSH is doing some really complicated stuff and because Xcode is kinda crap. My very slowly decreasing cluelessness also figures in.

[emkey1]

An update on my latest findings.

Doing the following two things has greatly improved the reliability of go under iSH. It has likely fixed many other problems as well.

1. Rewriting most of the RW lock/unlock functions to be non blocking. This reduces the odds of a deadlock. I briefly nanosleep() between attempts at getting a lock.
2. Adding a counter to the task struct that is incremented and decremented whenever portions of the code are entered or exited that would be sensitive to the task associated with the struct going away. I then wait until the counter has gone to zero before I reap a task.

I may back port these changes to iSH and submit a PR, but it would take a lot of time and I know PR's aren't a high priority right now as the developers are devoting their limited time to other things. In the meantime these changes will be present in the next release of my fork of iSH (iSH-AOK).

[Twiinner]

Good day, lads! I've recently successfully compile a simple "Golang hello world" inside the iSH, with the version of go is 1.18.6, for sure not build-from source, just use the command

apk add go --repository=http://dl-cdn.alpinelinux.org/alpine/latest-stable/community

You may use the --repository option to choose different version of go, without the need to build from source.

[mxdlzg]

Looking forward to new progress. I am trying to run frp in iSH, but failing.