go icon indicating copy to clipboard operation
go copied to clipboard

Published 20 hours ago • verified publisher icon golang

runtime: crosscompiled to AIX/ppc64 not working in IBMi PASE (AIX7.2) due to "not in usable address space"

Open rfx77 opened this issue 4 years ago • 74 comments
trafficstars

I want to crosscompile a simple Hello-World example to IBM-i PASE environment. This is a AIX 7.2 subsystem on the IBMi (AS400). Normally AIX compiled code should work in this environment.

But when i want to run the test application i get the following error:

runtime: memory allocated by OS [0x700000028000000, 0x70000002c000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

runtime stack:
runtime.throw(0x1000d9f06, 0x2e)
        /usr/lib/go-1.14/src/runtime/panic.go:1116 +0x68 fp=0xffffffffffff590 sp=0xffffffffffff550 pc=0x100032e08
runtime.(*mheap).sysAlloc(0x180037440, 0x400000, 0x100062a18, 0x9001000a01a1450)
        /usr/lib/go-1.14/src/runtime/malloc.go:706 +0x838 fp=0xffffffffffff660 sp=0xffffffffffff590 pc=0x10000a9a8
runtime.(*mheap).grow(0x180037440, 0x1, 0x0)
        /usr/lib/go-1.14/src/runtime/mheap.go:1286 +0x18c fp=0xffffffffffff6f0 sp=0xffffffffffff660 pc=0x10002586c
runtime.(*mheap).allocSpan(0x180037440, 0x1, 0x2a000100062a18, 0x18004d8e8, 0x900000000332200)
        /usr/lib/go-1.14/src/runtime/mheap.go:1124 +0x748 fp=0xffffffffffff780 sp=0xffffffffffff6f0 pc=0x100025578
runtime.(*mheap).alloc.func1()
        /usr/lib/go-1.14/src/runtime/mheap.go:871 +0x7c fp=0xffffffffffff7e8 sp=0xffffffffffff780 pc=0x10005e26c
runtime.systemstack(0xffffffffffff840)
        /usr/lib/go-1.14/src/runtime/asm_ppc64x.s:295 +0xd0 fp=0xffffffffffff808 sp=0xffffffffffff7e8 pc=0x1000604c0

...

Is this a problem which can be solved easily? How can i fix this?

This is the go code i compiled

package main

import "fmt"

func main() {
   fmt.Println("hello world22e")

}

i tried the crosscompile from windows and linux with the same result.

greetings, Franz

rfx77 avatar Mar 15 '21 11:03 rfx77

1.14 is out of support, please test with 1.16

AlexRouSg avatar Mar 15 '21 12:03 AlexRouSg

same problem


runtime: memory allocated by OS [0x700000028000000, 0x70000002c000000) not in usable address space: base outside usable address space
fatal error: memory reservation exceeds address space limit

runtime stack:
runtime.throw(0x1000ce532, 0x2e)
        C:/develop/tools/go/go1.16.2/src/runtime/panic.go:1117 +0x68 fp=0xffffffffffff548 sp=0xffffffffffff508 pc=0x100034f78
runtime.(*mheap).sysAlloc(0x1800376a0, 0x400000, 0x9001000a01a0e50, 0x9001000a01a0e68)
        C:/develop/tools/go/go1.16.2/src/runtime/malloc.go:720 +0x840 fp=0xffffffffffff618 sp=0xffffffffffff548 pc=0x10000b290
runtime.(*mheap).grow(0x1800376a0, 0x1, 0x0)
        C:/develop/tools/go/go1.16.2/src/runtime/mheap.go:1346 +0xa8 fp=0xffffffffffff6a8 sp=0xffffffffffff618 pc=0x1000269c8
runtime.(*mheap).allocSpan(0x1800376a0, 0x1, 0x2c000180058c78, 0x9001000a00908e8)
        C:/develop/tools/go/go1.16.2/src/runtime/mheap.go:1173 +0x624 fp=0xffffffffffff730 sp=0xffffffffffff6a8 pc=0x1000267d4
runtime.(*mheap).alloc.func1()
        C:/develop/tools/go/go1.16.2/src/runtime/mheap.go:910 +0x6c fp=0xffffffffffff790 sp=0xffffffffffff730 pc=0x1000608ac
runtime.systemstack(0xffffffffffff7e8)
        C:/develop/tools/go/go1.16.2/src/runtime/asm_ppc64x.s:295 +0xd0 fp=0xffffffffffff7b0 sp=0xffffffffffff790 pc=0x100065010
runtime.(*mheap).alloc(0x1800376a0, 0x1, 0x2c01000000000000, 0x0)
        C:/develop/tools/go/go1.16.2/src/runtime/mheap.go:904 +0x70 fp=0xffffffffffff810 sp=0xffffffffffff7b0 pc=0x100025ce0

rfx77 avatar Mar 15 '21 13:03 rfx77

ping @trex58

AlexRouSg avatar Mar 15 '21 13:03 AlexRouSg

ping @Helflym

trex58 avatar Mar 15 '21 14:03 trex58

@rfx77 This is normal. On AIX 7.2, the pointers allocated by mmap are between 0x0A00000000000000-0x0AFFFFFFFFFFFFFF. On AIX 7.1 and I guess on IBMi (which as far as I remember is based on AIX 7.1), the default is 0x0700000000000000-0x07FFFFFFFFFFFFFF. I know that on AIX 7.2, it's possible to switch thanks to an environment variable. But I don't know if IBMi have such thing. The relevant doc was https://www.ibm.com/support/knowledgecenter/ssw_aix_72/performance/1TB_segment_aliasing.html for AIX, if I remember correctly. If it's not then a patch is needed. Among the things to do:

  • Modify arenaBaseOffset = 0x0700000000000000 at (https://github.com/golang/go/blob/master/src/runtime/malloc.go#L305
  • Change the first pointer address (https://github.com/golang/go/blob/master/src/runtime/malloc.go#L542)
  • And maybe change these values in https://github.com/golang/go/blob/master/src/runtime/lfstack_64bit.go.

There are some issues related to this #38966 #35451 as along as some commits (check the history of malloc.go they should be listed there).

However, that might not be the only issue regarding IBMi. I don't know.

Helflym avatar Mar 15 '21 14:03 Helflym

On IBM-i 7.4 the Pase Runtime is derived from AIX 7.2, Technology Level 2. https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rzalf/rzalfwhatsnew.htm

rfx77 avatar Mar 15 '21 15:03 rfx77

Strange maybe that AIX feature is there on IBMi or it's set up to the other segment. You can try the VMO tunable which controls that on AIX. It should be something like VMM_CNTRL=ESID_ALLOCATOR=0 according to the documentation.

Helflym avatar Mar 15 '21 15:03 Helflym

The ENV Variable VMM_CNTRL has no effect.

I checked the sourcecode but i am not deep enough in the go source to check what the problem is. can someone guide me to the point where i can change the memory allocation and what to change?

Or can i help someone who knows more with my testing. We have a IBMi 7.4 TR3 running so i could help.

rfx77 avatar Mar 15 '21 16:03 rfx77

The ENV Variable VMM_CNTRL has no effect.

Maybe there is another way to change the default mmap segment. If as you said, IBMi 7.4 is based on AIX 7.2, then the IBMi devs should have said something about that.

I checked the sourcecode but i am not deep enough in the go source to check what the problem is. can someone guide me to the point where i can change the memory allocation and what to change?

I've listed the main things to do above. The AIX comments in malloc.go normally explained what's going on. You "just" need to replace 0x0A00000000000000 by 0x0700000000000000

Helflym avatar Mar 16 '21 07:03 Helflym

I did what you suggested and it works. The simple Hello World example runs on IBMi

i also tried a simple rest-webserver. works also

How can we proceed and what should we test to check if it stable?

rfx77 avatar Mar 16 '21 12:03 rfx77

Alright good news! You should be able to bootstrap Go for IBMi with src/bootstrap.bash. After that, you should be able to rebuild and test golang directly on IBMi with src/all.bash.

Helflym avatar Mar 16 '21 13:03 Helflym

Ok. I compiled the bootstrap and it runs on IBM-i. But make.bash does not.

here is the output:


GOROOT_BOOTSTRAP=/develop/go-aix-ppc64-bootstrap/ ./make.bash

Building Go cmd/dist using /develop/go-aix-ppc64-bootstrap/. (devel +120b9eb1c3 Tue Mar 16 13:06:17 2021 +0000 aix/ppc64)
runtime: lfstack.push invalid packing: node=0x70000002839e580 cnt=0x1 packed=0x800000141cf2c001 -> node=0xb0000002839e580
runtime: lfstack.push invalid packing: node=0x700000028408380 cnt=0x1 packed=0x800000142041c001 -> node=0xb00000028408380
fatal error: lfstack.push
runtime: lfstack.push invalid packing: node=0x70000002808e500 cnt=0x1 packed=0x8000001404728001 -> node=0xb0000002808e500
fatal error: lfstack.push
runtime: lfstack.push invalid packing: node=0x7000000280289e0 cnt=0x1 packed=0x800000140144f001 -> node=0xb000000280289e0
fatal error: lfstack.push
runtime: lfstack.push invalid packing: node=0x7000000282a4840 cnt=0x1 packed=0x8000001415242001 -> node=0xb000000282a4840
fatal error: lfstack.push
runtime: lfstack.push invalid packing: node=0x7000000284982c0 cnt=0x1 packed=0x8000001424c16001 -> node=0xb000000284982c0
fatal error: lfstack.push
fatal error: lfstack.push
runtime: lfstack.push invalid packing: node=0x700000028304660 cnt=0x1 packed=0x8000001418233001 -> node=0xb00000028304660
fatal error: lfstack.push

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x700000028408380)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x700000028185e00, 0x700000028408380, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x700000028185e00)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

goroutine 1 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x7000000281d8c60, 0x16, 0x16, 0x7000000281ba800)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.loadImport(0x10069efc0, 0x7000000280220a0, 0x700000028189200, 0x70000002818c610, 0xa, 0x70000002802a01c, 0xf, 0x0, 0x700000028115b80, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:670 +0x130
cmd/go/internal/load.PackagesAndErrors(0x10069efc0, 0x7000000280220a0, 0x700000028196040, 0x1, 0x1, 0x0, 0x0, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:2415 +0x6bc
cmd/go/internal/work.runBuild(0x10069efc0, 0x7000000280220a0, 0x180042fc0, 0x700000028196040, 0x1, 0x1)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/work/build.go:375 +0x74
main.main()
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/main.go:195 +0xbbc

goroutine 19 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x70000002832c320, 0x5, 0x5, 0x7000000280a4000)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818cb57, 0x5)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 20 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x700000028294460, 0x5, 0x5, 0x70000002813a000)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c8d7, 0x5)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 21 [runnable]:
go/scanner.(*Scanner).scanComment(0x700000028237220, 0x10000000a, 0x25b)
        /opt/go/go-aix-ppc64-bootstrap/src/go/scanner/scanner.go:221 +0x2b0
go/scanner.(*Scanner).Scan(0x700000028237220, 0x100543d20, 0x10018b838, 0x700000000000010, 0x700000026c93ca8)
        /opt/go/go-aix-ppc64-bootstrap/src/go/scanner/scanner.go:888 +0x174
go/parser.(*parser).next0(0x700000028237200)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/parser.go:257 +0x1e4
go/parser.(*parser).next(0x700000028237200)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/parser.go:320 +0x84
go/parser.(*parser).expectSemi(0x700000028237200)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/parser.go:436 +0xc8
go/parser.(*parser).parseGenDecl(0x700000028237200, 0x4b, 0x7000000281110f8, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/parser.go:2749 +0x470
go/parser.(*parser).parseFile(0x700000028237200, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/parser.go:2889 +0x820
go/parser.ParseFile(0x700000028484140, 0x700000028489ac0, 0x3c, 0x10050f500, 0x7000000281c3188, 0x6, 0x0, 0x0, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/parser/interface.go:124 +0x140
go/build.readGoInfo(0x100693d40, 0x7000000280a0218, 0x70000002848a870, 0x100693d40, 0x7000000280a0218)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:409 +0x3f4
go/build.(*Context).matchFile(0x180070f40, 0x7000000281ad900, 0x31, 0x7000000283a8772, 0xa, 0x7000000284827e0, 0x70000002829e4e0, 0x700000028484140, 0x700000028316a20, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:1411 +0x9b4
go/build.(*Context).Import(0x180070f40, 0x70000002818c8e1, 0xd, 0x7000000281fc380, 0x18, 0x4, 0x700000028296060, 0x1005310a0, 0x70000002807c570)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:830 +0x1eb0
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x700000028298060, 0x700000028293e80, 0x10056b8a0, 0x70000002829c030)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x70000002818c8e1, 0xd, 0x7000000281fc360, 0x19, 0x7000000281fc380, 0x18, 0x0, 0x0, 0x0, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c8e1, 0xd)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 29 [semacquire]:
sync.runtime_SemacquireMutex(0x70000002838e028, 0x100560720, 0x1)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/sema.go:71 +0x44
sync.(*Mutex).lockSlow(0x70000002838e024)
        /opt/go/go-aix-ppc64-bootstrap/src/sync/mutex.go:138 +0x1a8
sync.(*Mutex).Lock(...)
        /opt/go/go-aix-ppc64-bootstrap/src/sync/mutex.go:81
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x7000000283904d0, 0x700000028253e80, 0x10056b8a0, 0x700000028396d50)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:126 +0x124
cmd/go/internal/load.loadPackageData(0x70000002839a1d1, 0x3, 0x70000002818c8f1, 0x4, 0x70000002840e000, 0x28, 0x700000028024007, 0x1f, 0x100000000000000, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281bb000, 0x100000000000001, 0x700000028189200, 0x70000002839a1d1, 0x3)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 23 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x7000000282e2360, 0x9, 0x9, 0x7000000281bb000)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c8f1, 0x4)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 24 [semacquire]:
bufio.NewReaderSize(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:57
bufio.NewReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:63
go/build.newImportReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:33
go/build.readGoInfo(0x100693d40, 0x70000002829a348, 0x7000000282e2750, 0x100693d40, 0x70000002829a348)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:366 +0xb8
go/build.(*Context).matchFile(0x180070f40, 0x7000000280ae060, 0x27, 0x700000028250028, 0x9, 0x70000002807d440, 0x7000000283980e0, 0x70000002802e840, 0x70000002848a7e0, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:1411 +0x9b4
go/build.(*Context).Import(0x180070f40, 0x70000002818c8f7, 0x3, 0x7000000281fc380, 0x18, 0x4, 0x70000002838e020, 0x1005310a0, 0x70000002807c570)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:830 +0x1eb0
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x700000028390010, 0x70000002828de80, 0x10056b8a0, 0x700000028396000)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x70000002818c8f7, 0x3, 0x7000000281fc360, 0x19, 0x7000000281fc380, 0x18, 0x0, 0x0, 0x0, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c8f7, 0x3)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 26 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x700000028223440, 0x4, 0x4, 0x7000000281bac00)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c901, 0x9)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 27 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x700000028183020, 0x6, 0x6, 0x7000000281bb800)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c8fc, 0x3)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 33 [select]:
cmd/go/internal/load.(*preload).preloadImports(0x700000028189200, 0x70000002829dad0, 0x3, 0x3, 0x7000000284d2000)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:982 +0x188
cmd/go/internal/load.(*preload).preloadImports.func1(0x70000002829e000, 0x100000000000001, 0x700000028189200, 0x70000002839c049, 0x14)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:990 +0x120
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 37 [runnable]:
bufio.NewReaderSize(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:57
bufio.NewReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:63
go/build.newImportReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:33
go/build.readGoInfo(0x100693d40, 0x70000002842e118, 0x7000000282e26c0, 0x100693d40, 0x70000002842e118)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:366 +0xb8
go/build.(*Context).matchFile(0x180070f40, 0x7000000282d4660, 0x2b, 0x70000002840756c, 0x7, 0x70000002840cc00, 0x70000002829e8e0, 0x700000028418540, 0x700000028316900, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:1411 +0x9b4
go/build.(*Context).Import(0x180070f40, 0x70000002818c915, 0x7, 0x7000000281fc380, 0x18, 0x4, 0x700000028297020, 0x1005310a0, 0x70000002807dce0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:830 +0x1eb0
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x700000028298a40, 0x7000000284c7e80, 0x10056b8a0, 0x70000002829d3e0)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x70000002818c915, 0x7, 0x7000000281fc360, 0x19, 0x7000000281fc380, 0x18, 0x0, 0x0, 0x0, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c915, 0x7)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 65 [runnable]:
syscall.Lstat(0x7000000280ae2a0, 0x2b, 0x7000000280e2f38, 0x2, 0x7000000280ae2a0)
        /opt/go/go-aix-ppc64-bootstrap/src/syscall/zsyscall_aix_ppc64.go:988 +0x11c
os.lstatNolog.func1(...)
        /opt/go/go-aix-ppc64-bootstrap/src/os/stat_unix.go:46
os.ignoringEINTR(...)
        /opt/go/go-aix-ppc64-bootstrap/src/os/file_posix.go:246
os.lstatNolog(0x7000000280ae2a0, 0x2b, 0x3, 0x3, 0x7000000280ae2a0, 0x2b)
        /opt/go/go-aix-ppc64-bootstrap/src/os/stat_unix.go:45 +0x64
os.Lstat(0x7000000280ae2a0, 0x2b, 0x26, 0x1005c52ea, 0x1, 0x70000002843eb6c)
        /opt/go/go-aix-ppc64-bootstrap/src/os/stat.go:22 +0x48
os.(*File).readdir(0x70000002842e038, 0xffffffffffffffff, 0x2, 0x1001757c0, 0x18009de00, 0x1001bcd00, 0x10000c27c, 0x70000002840e240, 0x26, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/os/dir_unix.go:116 +0x68c
os.(*File).Readdir(0x70000002842e038, 0xffffffffffffffff, 0x0, 0x55550, 0x70000002842e038, 0x0, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/os/dir.go:41 +0x48
io/ioutil.ReadDir(0x70000002840e240, 0x26, 0x70000002840e240, 0x26, 0x7000000284c30d8, 0x2802a528, 0x100059ed8)
        /opt/go/go-aix-ppc64-bootstrap/src/io/ioutil/ioutil.go:63 +0x6c
cmd/go/internal/fsys.readDir(0x70000002840e240, 0x26, 0x70000002840e240, 0x26, 0x1800a7d20, 0xd, 0x7000000000021d8)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/fsys/fsys.go:246 +0x34
cmd/go/internal/fsys.ReadDir(0x70000002840e240, 0x26, 0x700000028071960, 0x2, 0x2, 0x700000028158380, 0x32)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/fsys/fsys.go:270 +0x2f4
go/build.(*Context).readDir(0x180070f40, 0x70000002840e240, 0x26, 0x2, 0x700000028158380, 0x32, 0x4, 0x1006938a0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:189 +0x58
go/build.(*Context).Import(0x180070f40, 0x70000002818c911, 0x2, 0x7000000281fc380, 0x18, 0x4, 0x700000028312780, 0x1005310a0, 0x70000002807c570)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:793 +0x13f0
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x70000002830e280, 0x7000000284c3e80, 0x10056b8a0, 0x7000000283089f0)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x70000002818c911, 0x2, 0x7000000281fc360, 0x19, 0x7000000281fc380, 0x18, 0x0, 0x0, 0x0, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281ba800, 0x1, 0x700000028189200, 0x70000002818c911, 0x2)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 36 [semacquire]:
bufio.NewReaderSize(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:57
bufio.NewReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/bufio/bufio.go:63
go/build.newImportReader(...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:33
go/build.readGoInfo(0x100693d40, 0x700000028392250, 0x7000000283aa750, 0x100693d40, 0x700000028392250)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/read.go:366 +0xb8
go/build.(*Context).matchFile(0x180070f40, 0x70000002840e3f0, 0x29, 0x7000000282d47da, 0x6, 0x70000002829d740, 0x70000002813a4e0, 0x7000000282bea40, 0x700000028316990, 0x0, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:1411 +0x9b4
go/build.(*Context).Import(0x180070f40, 0x7000000280227a1, 0x5, 0x700000028024c00, 0x2d, 0x4, 0x700000028071820, 0x1005310a0, 0x70000002807dce0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:830 +0x1eb0
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x70000002805e8a0, 0x700000028255e80, 0x10056b8a0, 0x70000002807dcb0)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x7000000280227a1, 0x5, 0x70000002818c901, 0x9, 0x700000028024c00, 0x2d, 0x700000028024007, 0x1f, 0x100000000000000, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281bac00, 0x100000000000001, 0x700000028189200, 0x7000000280227a1, 0x5)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 81 [semacquire]:
strings.(*Builder).grow(...)
        /opt/go/go-aix-ppc64-bootstrap/src/strings/builder.go:68
strings.(*Builder).Grow(...)
        /opt/go/go-aix-ppc64-bootstrap/src/strings/builder.go:82
strings.Join(0x700000028095470, 0x7, 0xa, 0x100687688, 0x1, 0x0, 0x7000000284804d0)
        /opt/go/go-aix-ppc64-bootstrap/src/strings/strings.go:434 +0x114
go/ast.(*CommentGroup).Text(0x7000000284006a8, 0x7000000283064a8, 0x4)
        /opt/go/go-aix-ppc64-bootstrap/src/go/ast/ast.go:159 +0x338
go/build.(*Context).Import(0x180070f40, 0x70000002839a0e1, 0x4, 0x700000028024ea0, 0x26, 0x4, 0x70000002840a280, 0x1005310a0, 0x70000002807dce0)
        /opt/go/go-aix-ppc64-bootstrap/src/go/build/build.go:894 +0x2850
cmd/go/internal/load.loadPackageData.func2(0x1800702e0, 0x100514b20)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:855 +0x324
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x700000028412100, 0x700000028095e80, 0x10056b8a0, 0x70000002840c390)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:128 +0x168
cmd/go/internal/load.loadPackageData(0x70000002839a0e1, 0x4, 0x70000002818cb71, 0x2, 0x700000028024ea0, 0x26, 0x700000028024007, 0x1f, 0x100000000000000, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281bb400, 0x100000000000001, 0x700000028189200, 0x70000002839a0e1, 0x4)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

goroutine 39 [semacquire]:
sync.runtime_SemacquireMutex(0x700000028312788, 0x100560720, 0x1)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/sema.go:71 +0x44
sync.(*Mutex).lockSlow(0x700000028312784)
        /opt/go/go-aix-ppc64-bootstrap/src/sync/mutex.go:138 +0x1a8
sync.(*Mutex).Lock(...)
        /opt/go/go-aix-ppc64-bootstrap/src/sync/mutex.go:81
cmd/go/internal/par.(*Cache).Do(0x1800702e0, 0x100514b20, 0x700000028390720, 0x70000002858be80, 0x10056b8a0, 0x700000028397560)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/par/work.go:126 +0x124
cmd/go/internal/load.loadPackageData(0x7000000280227a9, 0x2, 0x70000002818c901, 0x9, 0x700000028024c00, 0x2d, 0x700000028024007, 0x1f, 0x100000000000000, 0x1, ...)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:829 +0x354
cmd/go/internal/load.(*preload).preloadImports.func1(0x7000000281bac00, 0x100000000000001, 0x700000028189200, 0x7000000280227a9, 0x2)
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:987 +0x88
created by cmd/go/internal/load.(*preload).preloadImports
        /opt/go/go-aix-ppc64-bootstrap/src/cmd/go/internal/load/pkg.go:986 +0x1d4

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x700000028304660)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x700000028323c80, 0x700000028304660, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x700000028323c80)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x70000002839e580)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x700000028383e00, 0x70000002839e580, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x700000028383e00)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x7000000284982c0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x7000000284a0f00, 0x7000000284982c0, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x7000000284a0f00)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x7000000282a4840)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x700000028083980, 0x7000000282a4840, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x700000028083980)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x7000000280289e0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x70000002815a180, 0x7000000280289e0, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x70000002815a180)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

runtime stack:
runtime.throw(0x1005cb8bb, 0xc)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/panic.go:1126 +0x68
runtime.(*lfstack).push(0x1800a6250, 0x70000002808e500)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/lfstack.go:30 +0x180
runtime.gcBgMarkWorker.func1(0x700000028184300, 0x70000002808e500, 0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/mgc.go:1936 +0x8c
runtime.park_m(0x700000028184300)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/proc.go:3298 +0x108
runtime.mcall(0x0)
        /opt/go/go-aix-ppc64-bootstrap/src/runtime/asm_ppc64x.s:192 +0x54

rfx77 avatar Mar 16 '21 13:03 rfx77

It looks like some more part in the runtime needs to be changed. But I don't know where. I don't remember having such issues. @aclements @mknyszek maybe you'll have a better idea about what's wrong here.

Helflym avatar Mar 16 '21 14:03 Helflym

Okay. i also edited malloc at another position and lfstack_64.go. now my webserver seems stable. i will build another bootstrap and give you feedback

rfx77 avatar Mar 16 '21 14:03 rfx77

Now i get an error when with make.bash

GOROOT_BOOTSTRAP=/develop/go-aix-ppc64-bootstrap/ ./make.bash

Building Go cmd/dist using /develop/go-aix-ppc64-bootstrap/. (devel +120b9eb1c3 Tue Mar 16 13:06:17 2021 +0000 aix/ppc64)
# runtime/internal/atomic
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/asm:
System error - error data is: -1 9 (64002005)
# internal/bytealg
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/asm:
System error - error data is: -1 9 (64002005)
go: failed to remove work dir: unlinkat /tmp/go-build3976199215: invalid argument

with more verbosity

GOROOT_BOOTSTRAP=/develop/go-aix-ppc64-bootstrap/ ./make.bash -a -d -v


Building Go cmd/dist using /develop/go-aix-ppc64-bootstrap/. (devel +120b9eb1c3 Tue Mar 16 13:06:17 2021 +0000 aix/ppc64)
go: failed to remove work dir: unlinkat /tmp/go-build1123863472: invalid argument
Building Go toolchain1 using /develop/go-aix-ppc64-bootstrap/.
runtime/internal/sys
internal/abi
internal/race
unicode
internal/unsafeheader
go tool compile: exit status 255
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/compile:
System error - error data is: -1 9 (64002005)
go: failed to remove work dir: unlinkat /tmp/go-build282961040: invalid argument
runtime/internal/math
go tool dist: FAILED: /develop/go-aix-ppc64-bootstrap/bin/go install -gcflags=-l -tags=math_big_pure_go compiler_bootstrap -v bootstrap/cmd/...: exit status 1

rfx77 avatar Mar 16 '21 14:03 rfx77

I did some testing an i cannot get the compiler to work on the ibmi. but all my crosscompiled projets work without a problem. i even tried a simple mongodb client and it runs flawless.

for me it would be sufficient when i can develop on my pc and crosscompile to ibmi. Developing directly on ibmi is not necessary - and no real fun ;-)

would it be possible to add a new cross-compile os like (ibmi or aix-pase). i only changed three lines of code but this kills the aix compatibility and i have no aix to test.


diff --git a/src/runtime/lfstack_64bit.go b/src/runtime/lfstack_64bit.go
index 4812dd1156..bce6450689 100644
--- a/src/runtime/lfstack_64bit.go
+++ b/src/runtime/lfstack_64bit.go
@@ -53,7 +53,7 @@ func lfstackUnpack(val uint64) *lfnode {
                return (*lfnode)(unsafe.Pointer(uintptr(int64(val) >> cntBits << 3)))
        }
        if GOARCH == "ppc64" && GOOS == "aix" {
-               return (*lfnode)(unsafe.Pointer(uintptr((val >> aixCntBits << 3) | 0xa<<56)))
+               return (*lfnode)(unsafe.Pointer(uintptr((val >> aixCntBits << 3) | 0x7<<56)))
        }
        return (*lfnode)(unsafe.Pointer(uintptr(val >> cntBits << 3)))
 }
diff --git a/src/runtime/malloc.go b/src/runtime/malloc.go
index 8435f96532..e3d98382b8 100644
--- a/src/runtime/malloc.go
+++ b/src/runtime/malloc.go
@@ -302,7 +302,7 @@ const (
        //
        // On other platforms, the user address space is contiguous
        // and starts at 0, so no offset is necessary.
-       arenaBaseOffset = 0xffff800000000000*sys.GoarchAmd64 + 0x0a00000000000000*sys.GoosAix
+       arenaBaseOffset = 0xffff800000000000*sys.GoarchAmd64 + 0x0700000000000000*sys.GoosAix
        // A typed version of this constant that will make it into DWARF (for viewcore).
        arenaBaseOffsetUintptr = uintptr(arenaBaseOffset)

@@ -539,7 +539,7 @@ func mallocinit() {
                                        // to avoid collisions with others mmaps done by non-go programs.
                                        continue
                                }
-                               p = uintptr(i)<<40 | uintptrMask&(0xa0<<52)
+                               p = uintptr(i)<<40 | uintptrMask&(0x70<<52)
                        default:
                                p = uintptr(i)<<40 | uintptrMask&(0x00c0<<32)

rfx77 avatar Mar 16 '21 15:03 rfx77

Well, until anyone managed to get a full bootstrap on IBMi, I guess patches related to IBMi won't be accepted in the source code. Moreover, AFAIK any new OS needs a VM in the build farm. I don't know how the Solaris/Illumos separation started but this is kind of the same here.

Helflym avatar Mar 16 '21 17:03 Helflym

Since IBM has a good Open-Source strategy with IBM-i maybe someone from IBM could help. Are there any active contributors from them?

rfx77 avatar Mar 17 '21 08:03 rfx77

@ThePrez maybe you can jump in at this point? Are there any plans for go support on IBM i and besides that also for providing IBM i VMs for open source development?

holterandreasb avatar Mar 17 '21 08:03 holterandreasb

When i use the bootstrap version of go as GOROOT i can compile a simple "Hello World" on IBMi

GOROOT=/develop/go-aix-ppc64-bootstrap/ go build test.go && ./test

But it seems to me that the cgo integration is the problem. When i try to compile a net/http server example i get the following error:

# runtime/cgo
/tmp/go-build1628516446/b085/_cgo_import.go:14:3: usage: //go:cgo_import_dynamic local [remote ["lib.a/object.o"]]

The content of the file is:

package cgo
//go:cgo_import_dynamic errno errno "libc.a/shr_64.o"
//go:cgo_import_dynamic __mod_init __mod_init "libc.a/shr_64.o"
//go:cgo_import_dynamic calloc calloc "libc.a/shr_64.o"
//go:cgo_import_dynamic exit exit "libc.a/shr_64.o"
//go:cgo_import_dynamic __assert __assert "libc.a/shr_64.o"
//go:cgo_import_dynamic __crt0v __crt0v "libc.a/shr_64.o"
//go:cgo_import_dynamic __malloc_user_defined_name __malloc_user_defined_name "libc.a/shr_64.o"
//go:cgo_import_dynamic __n_pthreads __n_pthreads "libpthreads.a/shr_xpg5_64.o"
//go:cgo_import_dynamic __pth_init __pth_init "libpthreads.a/shr_xpg5_64.o"
//go:cgo_import_dynamic __pthread __pthread "libpthreads.a/shr_xpg5_64.o"
//go:cgo_import_dynamic _ _ "libpthreads.a/shr_xpg5_64.o"
//go:cgo_import_dynamic _ _ "libc.a/shr_64.o"
//go:cgo_import_dynamic _ _ "../"

Has anybody seen this and can help me what the problem might be.

Greetings, Franz

Update:

Problem seems to be the last line which fails in lex.go because the split with / returns only one element.

rfx77 avatar Mar 19 '21 14:03 rfx77

Hacked around the problem but now i am back where i was when trying to compile with the bootstrap version:

GOROOT=/develop/go-aix-ppc64-bootstrap/ go build test.go
# vendor/golang.org/x/crypto/curve25519
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/compile:
System error - error data is: -1 9 (64002005)
# net/http/internal
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/compile:
System error - error data is: -1 9 (64002005)
# vendor/golang.org/x/text/secure/bidirule
Could not load program /develop/go-aix-ppc64-bootstrap/pkg/tool/aix_ppc64/compile:
System error - error data is: -1 9 (64002005)
# runtime/cgo
Could not load program /QOpenSys/usr/bin/sh:
System error - error data is: -1 9 (2005)
go: failed to remove work dir: unlinkat /tmp/go-build2796491330: invalid argument

rfx77 avatar Mar 19 '21 15:03 rfx77

We unfortunately don't have any concrete dates for golang yet. IBM should probably implement those *at functions properly in PASE and likely will at some point. One could conceivably implement those in runtime code (in this case a relatively simple wrapper around unlink), but we haven't gotten to that task yet. I'll bounce this thread off some other dev folks

ThePrez avatar Mar 19 '21 18:03 ThePrez

Hi, IBMer here working on IBM i.

While the IBM i PASE runtime (ie userspace layer) is based on AIX, the kernel is distinct. In the case of 1TB Segment Aliasing, that is not a feature that is implemented in the IBM i kernel, so it does not work and we'll need to use the standard mmap region as has been mentioned previously in this thread. (I've been meaning to bring this up with our kernel folks).

I have played around with some Golang stuff a bit on IBM i and have some branches and issues here: https://github.com/kadler/go including https://github.com/kadler/go/tree/pase-go1.15.3. IIRC, this is enough to get some basic Golang programs built in a cross compile manner, but I have not had much luck running the compiler on IBM i itself due to the loader errors System error - error data is: -1 9 (64002005). I have not had time to investigate much further in to these, though Golang cross-compilation is so easy to use I don't find it much of a hindrance to me at this time (not to say it wouldn't be for others, though).

@rfx77 you seem pretty gung-ho on go/IBM i. Maybe we can collaborate to move this forward.

kadler avatar Mar 19 '21 19:03 kadler

@kadler: i have fixed the 64002005. They all come from concurrency problems with the ifs. i have disabled all parallel work and this is solved but thats not the only problem ;-) i will post in the next minutes my findings with a compile output as summary for all interested.

rfx77 avatar Mar 19 '21 19:03 rfx77

So a short summary: Good News and bad news now.

With some work i managed to get to the point where i can build toolchain1, go_bootstrap, toolchain2 and toolchain3 on the ibmi

The compiler compiles much of the libraries but i have a problem with cgo. it seems to require dwarf debug symbols which are not supported with gcc on ibmi. (gcc -gdwarf)

is there any way around this when i want to build golang on the platform.

Here is my output (i removed some verbosity):


Building Go cmd/dist using /develop/go-bs/. (devel +120b9eb1c3 Tue Mar 16 13:06:17 2021 +0000 aix/ppc64)
Building Go toolchain1 using /develop/go-bs/.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for aix/ppc64.
os/user
# os/user
cgo: cannot load DWARF output from $WORK/b038//_cgo_.o: decoding dwarf section info at offset 0x0: too short
net
# net
cgo: cannot load DWARF output from $WORK/b088//_cgo_.o: decoding dwarf section info at offset 0x0: too short
os/signal/internal/pty
# os/signal/internal/pty
cgo: cannot load DWARF output from $WORK/b201//_cgo_.o: decoding dwarf section info at offset 0x0: too short
go tool dist: FAILED: /develop/go/pkg/tool/aix_ppc64/go_bootstrap install -gcflags=all= -ldflags=all= -v -p=1 std cmd: exit status 2

rfx77 avatar Mar 19 '21 19:03 rfx77

You are running on IBM i 7.4? AFAIK, that's the earliest version where DWARF support might be available (I don't recall if it was out of the box in AIX 7.2). Regardless, you'll need a newer version of GCC which understands DWARF support in AIX as well. While we are working on building a newer GCC version, it is not ready for release yet.

kadler avatar Mar 19 '21 19:03 kadler

Yes 7.4. Where can i get this gcc version?

rfx77 avatar Mar 19 '21 19:03 rfx77

As I said, it's not ready for release.

kadler avatar Mar 19 '21 19:03 kadler

The cgo tool does currently require DWARF debugging information. It would be possible but non-trivial to add support for some other form of debugging information.

If you build with CGO_ENABLED=0 in the environment, then cgo will be disabled. That may let you make some forward progress.

ianlancetaylor avatar Mar 19 '21 21:03 ianlancetaylor

YES!! Thanks @ianlancetaylor

Go is now compiled on IBMi and i compiled a little WebServer (REST). Seems to work.

I already run some minutes of jmeter against it and it seem stable.

How should we proceed now??

rfx77 avatar Mar 22 '21 11:03 rfx77