all: move src/device to separate repository

[sago35]

@aykevl @deadprogram refs #2634 Unlike #2634, I was able to build tinygo-org/drivers/examples/ili9341/pyportal_boing. Perhaps we are proceeding correctly.

The entire device repository is over 500MB and cannot go get. For this reason, only samd21 and samd51 can be built at this time.

$ go get tinygo.org/x/device@e0608542069c7beaa72e203e8533de55208ad166
go: downloading tinygo.org/x/device v0.0.0-20220804091950-e0608542069c
go: tinygo.org/x/device@e0608542069c7beaa72e203e8533de55208ad166: create zip: module source tree too large (max size is 524288000 bytes)

Finally, I think directories such as tinygo.org/device/sam or tinygo.org/device/stm32 might be a good place. To place this, we need to associate tinygo.org/device/sam with github.com/tinygo-org/sam.

In that case, only device/asm.go will have trouble finding a place to put it. We need to think about good placement.

[sago35]

CI is likely to be faster, as intended.

CircleCI : test-llvm14-go119

test	before	after
Pull submodules	49 sec	65 sec
make gen-device -j4	186 sec	0 sec

[sago35]

What should I do about this?

=== RUN   TestCompiler
=== PAUSE TestCompiler
=== CONT  TestCompiler
=== RUN   TestCompiler/basic.go
=== RUN   TestCompiler/pointer.go
=== RUN   TestCompiler/slice.go
=== RUN   TestCompiler/string.go
=== RUN   TestCompiler/float.go
=== RUN   TestCompiler/interface.go
=== RUN   TestCompiler/func.go
=== RUN   TestCompiler/defer.go-cortex-m-qemu
Error: ../../../../.cache/tinygo/goroot-e70de3bab50987dd6e72e1df1be7fed9a7f9a954d989c7d2a929c3536bbe1629/src/runtime/interrupt/interrupt_cortexm.go:7:2: cannot find package
FAIL	github.com/tinygo-org/tinygo/compiler	0.929s

 === CONT  TestBuild/EmulatedRISCV/alias.go
Error: ../../../.cache/tinygo/goroot-e70de3bab50987dd6e72e1df1be7fed9a7f9a954d989c7d2a929c3536bbe1629/src/runtime/interrupt/interrupt_tinygoriscv.go:6:8: cannot find package
FAIL	github.com/tinygo-org/tinygo	286.048s

[sago35]

I don't know why, but you are not doing go get tinygo.org/x/device.

Run make ASSERT=1 test
CGO_CPPFLAGS="-I/home/runner/work/tinygo/tinygo/llvm-project/llvm/include -I/home/runner/work/tinygo/tinygo/llvm-build/include -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/runner/work/tinygo/tinygo/llvm-build/tools/clang/include -I/home/runner/work/tinygo/tinygo/llvm-project/clang/include -I/home/runner/work/tinygo/tinygo/llvm-project/lld/include" CGO_CXXFLAGS="-std=c++14" CGO_LDFLAGS="-L/home/runner/work/tinygo/tinygo/llvm-build/lib -lclang -Wl,--start-group -lclangAnalysis -lclangAST -lclangASTMatchers -lclangBasic -lclangCodeGen -lclangCrossTU -lclangDriver -lclangDynamicASTMatchers -lclangEdit -lclangFormat -lclangFrontend -lclangFrontendTool -lclangHandleCXX -lclangHandleLLVM -lclangIndex -lclangLex -lclangParse -lclangRewrite -lclangRewriteFrontend -lclangSema -lclangSerialization -lclangTooling -lclangToolingASTDiff -lclangToolingCore -lclangToolingInclusions -Wl,--end-group -lstdc++ -Wl,--start-group -llldCOFF -llldCommon -llldELF -llldMachO -llldMinGW -llldWasm -Wl,--end-group -L/home/runner/work/tinygo/tinygo/llvm-build/lib  -lLLVMWindowsManifest -lLLVMMCJIT -lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMExtensions -lLLVMLibDriver -lLLVMOption -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMOrcTargetProcess -lLLVMOrcShared -lLLVMDebugInfoPDB -lLLVMCoverage -lLLVMCoroutines -lLLVMXtensaDisassembler -lLLVMXtensaAsmParser -lLLVMXtensaCodeGen -lLLVMXtensaDesc -lLLVMXtensaInfo -lLLVMAVRDisassembler -lLLVMAVRAsmParser -lLLVMAVRCodeGen -lLLVMAVRDesc -lLLVMAVRInfo -lLLVMWebAssemblyDisassembler -lLLVMWebAssemblyAsmParser -lLLVMWebAssemblyCodeGen -lLLVMWebAssemblyDesc -lLLVMWebAssemblyUtils -lLLVMWebAssemblyInfo -lLLVMRISCVDisassembler -lLLVMRISCVAsmParser -lLLVMRISCVCodeGen -lLLVMRISCVDesc -lLLVMRISCVInfo -lLLVMAArch64Disassembler -lLLVMAArch64AsmParser -lLLVMAArch64CodeGen -lLLVMAArch64Desc -lLLVMAArch64Utils -lLLVMAArch64Info -lLLVMARMDisassembler -lLLVMARMAsmParser -lLLVMARMCodeGen -lLLVMipo -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMFrontendOpenMP -lLLVMARMDesc -lLLVMARMUtils -lLLVMARMInfo -lLLVMX86TargetMCA -lLLVMMCA -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMCFGuard -lLLVMGlobalISel -lLLVMX86Desc -lLLVMX86Info -lLLVMMCDisassembler -lLLVMSelectionDAG -lLLVMInstrumentation -lLLVMAsmPrinter -lLLVMDebugInfoMSF -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts -lLLVMInstCombine -lLLVMAggressiveInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMDebugInfoDWARF -lLLVMObject -lLLVMTextAPI -lLLVMMCParser -lLLVMMC -lLLVMDebugInfoCodeView -lLLVMBitReader -lLLVMCore -lLLVMRemarks -lLLVMBitstreamReader -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle -lrt -ldl -lpthread -lm -lstdc++ " go test -v -timeout=20m -buildmode exe -tags "byollvm osusergo" ./builder ./cgo ./compileopts ./compiler ./interp ./transform .
go: downloading github.com/google/shlex v0.0.0-20181106134648-c34317bd91bf
go: downloading golang.org/x/tools v0.1.11
go: downloading tinygo.org/x/go-llvm v0.0.0-20220626113704-45f1e2dbf887
go: downloading github.com/aykevl/go-wasm v0.0.2-0.20220616010729-4a0a888aebdc
go: downloading github.com/blakesmith/ar v0.0.0-20150311145944-8bd4349a67f2
go: downloading github.com/gofrs/flock v0.8.1
go: downloading github.com/marcinbor85/gohex v0.0.0-20200531091804-343a4b548892
go: downloading github.com/mattn/go-colorable v0.1.8
go: downloading go.bug.st/serial v1.1.3
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/mattn/go-isatty v0.0.12
go: downloading golang.org/x/sys v0.0.0-20220114195835-da31bd327af9
go: downloading github.com/creack/goselect v0.1.2

[deadprogram]

Just a few general comments.

As @aykevl mentioned the custom domain package naming would be tinygo.org/x/device/sam. We use that to more easily handle the requirements for custom domains/caching/etc.

So for example tinygo.org/x/device/sam is in its own repository.

Each of the packages tinygo.org/x/device/sam could in fact be a separate package, but still within the main repo device. As long as each subpackage has its own go.mod file, they will be treated as separate, and avoid having to pull down files for families until you actually try to compile some code that needs it.

[aykevl]

As long as each subpackage has its own go.mod file, they will be treated as separate, and avoid having to pull down files for families until you actually try to compile some code that needs it.

I have tried this, and couldn't get it to work. If someone gets it to work, I'd be happy to hear.