go-yara
go-yara copied to clipboard
hillu
Errors cross compiling go-yara
I am trying to cross compile go-yara for x86 and x64 and I am getting some errors. The tool set for cross compiling is xgo.
I can cross compile correctly for darwin or windows, but I am unable to do it for linux.
x64
/usr/local/lib/libyara.a(math.o): In function `string_entropy':
math.c:(.text+0xecc): undefined reference to `log2'
/usr/local/lib/libyara.a(math.o): In function `data_entropy':
math.c:(.text+0x1150): undefined reference to `log2'
collect2: error: ld returned 1 exit status
x86
/usr/bin/ld: skipping incompatible /usr/local/lib/libyara.a when searching for -lyara
/usr/bin/ld: skipping incompatible //usr/local/lib/libyara.a when searching for -lyara
/usr/bin/ld: cannot find -lyara
collect2: error: ld returned 1 exit status
I am not sure that the errors came either from go-yara or yara itself.
If you need more datils, please ask them.
The log2 function needs libm, so I suppose that the compiler switch -lm is missing there. For x86 it seems that the linker is only finding libyara.a for the wrong architecture, presumably your /usr/local/lib/libyara.a is an x64 library.
I have never tried to work with xgo, can you provide the command line you used?
Hi, Here is more detail about the cross compilation process. This pice of code id for x64.
I cut some bits just for shorten it a little bit.
Compiling & Installing Yara
You can see while configuring the Yara sources log2 is availabe.
[...]
Compiling for linux/amd64...
Bootstaping dependency yara-3.7.1 for x86_64-linux...
[...]
checking whether to check for GCC pthread/shared inconsistencies... yes
checking whether -pthread is sufficient with -shared... yes
checking for isnan in -lm... yes
checking for log2 in -lm... yes
[...]
Building dependency yara-3.7.1 for x86_64-linux...
[...]
Making install in libyara
[...]
Libraries have been installed in:
/usr/local/lib`
Compiling the project with go-yara
Do you have any idea why this is happening?
I put the entire log.
WORK=/tmp/go-build672133109
mkdir -p $WORK/github.com/xxx/common/command/_obj/
mkdir -p $WORK/github.com/xxx/common/
cd /ext-go/1/src/github.com/xxx/common/command
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/xxx/common/command.a -trimpath $WORK -p github.com/xxx/common/command -complete -buildid 6de232fd460c689487cd553d492a5b981f9a6a4e -D _/ext-go/1/src/github.com/xxx/common/command -I $WORK -pack ./command.go
mkdir -p $WORK/github.com/xxx/common/errors/_obj/
cd /ext-go/1/src/github.com/xxx/common/errors
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/xxx/common/errors.a -trimpath $WORK -p github.com/xxx/common/errors -complete -buildid 0b88032c86f59d76b47652396d7a97a9baba41d6 -D _/ext-go/1/src/github.com/xxx/common/errors -I $WORK -pack ./error.go
mkdir -p $WORK/github.com/hillu/go-yara/_obj/
mkdir -p $WORK/github.com/hillu/
cd /ext-go/1/src/github.com/hillu/go-yara
pkg-config --cflags yara
mkdir -p $WORK/github.com/go-ini/ini/_obj/
mkdir -p $WORK/github.com/go-ini/
cd /ext-go/1/src/github.com/go-ini/ini
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/go-ini/ini.a -trimpath $WORK -p github.com/go-ini/ini -complete -buildid 474f2dbaf2ddd368243b39907845dc5143ab9520 -D _/ext-go/1/src/github.com/go-ini/ini -I $WORK -pack ./error.go ./file.go ./ini.go ./key.go ./parser.go ./section.go ./struct.go
cd /ext-go/1/src/github.com/hillu/go-yara
pkg-config --libs yara
CGO_LDFLAGS="-g" "-O2" "-L/usr/local/lib" "-lyara" /usr/local/go/pkg/tool/linux_amd64/cgo -objdir $WORK/github.com/hillu/go-yara/_obj/ -importpath github.com/hillu/go-yara -- -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 cgo.go compiler.go compiler_yara37.go error.go error_yara34.go error_yara35.go error_yara36.go error_yara37.go rule.go rule_yara37.go rules.go rules_yara34.go stream.go
mkdir -p $WORK/github.com/sirupsen/logrus/_obj/
mkdir -p $WORK/github.com/sirupsen/
cd /ext-go/1/src/github.com/sirupsen/logrus
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/github.com/sirupsen/logrus.a -trimpath $WORK -p github.com/sirupsen/logrus -complete -buildid 69e4640f7e47be37721fc4a6cace76ae47f6b1d5 -D _/ext-go/1/src/github.com/sirupsen/logrus -I $WORK -pack ./alt_exit.go ./doc.go ./entry.go ./exported.go ./formatter.go ./hooks.go ./json_formatter.go ./logger.go ./logrus.go ./terminal_linux.go ./terminal_notwindows.go ./text_formatter.go ./writer.go
cd $WORK
gcc -fdebug-prefix-map=a=b -c trivial.c
gcc -gno-record-gcc-switches -c trivial.c
cd /ext-go/1/src/github.com/hillu/go-yara
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/_cgo_export.o -c $WORK/github.com/hillu/go-yara/_obj/_cgo_export.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/cgo.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/cgo.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/compiler.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/compiler.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/compiler_yara37.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/compiler_yara37.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/error.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/error.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/error_yara34.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/error_yara34.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/error_yara35.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/error_yara35.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/error_yara36.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/error_yara36.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/error_yara37.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/error_yara37.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/rule.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/rule.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/rule_yara37.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/rule_yara37.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/rules.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/rules.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/rules_yara34.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/rules_yara34.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/stream.cgo2.o -c $WORK/github.com/hillu/go-yara/_obj/stream.cgo2.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/rules-callback.o -c ./rules-callback.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I/usr/local/include -I $WORK/github.com/hillu/go-yara/_obj/ -g -O2 -o $WORK/github.com/hillu/go-yara/_obj/_cgo_main.o -c $WORK/github.com/hillu/go-yara/_obj/_cgo_main.c
gcc -I . -fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/github.com/hillu/go-yara/_obj/_cgo_.o $WORK/github.com/hillu/go-yara/_obj/_cgo_main.o $WORK/github.com/hillu/go-yara/_obj/_cgo_export.o $WORK/github.com/hillu/go-yara/_obj/cgo.cgo2.o $WORK/github.com/hillu/go-yara/_obj/compiler.cgo2.o $WORK/github.com/hillu/go-yara/_obj/compiler_yara37.cgo2.o $WORK/github.com/hillu/go-yara/_obj/error.cgo2.o $WORK/github.com/hillu/go-yara/_obj/error_yara34.cgo2.o $WORK/github.com/hillu/go-yara/_obj/error_yara35.cgo2.o $WORK/github.com/hillu/go-yara/_obj/error_yara36.cgo2.o $WORK/github.com/hillu/go-yara/_obj/error_yara37.cgo2.o $WORK/github.com/hillu/go-yara/_obj/rule.cgo2.o $WORK/github.com/hillu/go-yara/_obj/rule_yara37.cgo2.o $WORK/github.com/hillu/go-yara/_obj/rules.cgo2.o $WORK/github.com/hillu/go-yara/_obj/rules_yara34.cgo2.o $WORK/github.com/hillu/go-yara/_obj/stream.cgo2.o $WORK/github.com/hillu/go-yara/_obj/rules-callback.o -g -O2 -L/usr/local/lib -lyara
# github.com/hillu/go-yara
/usr/local/lib/libyara.a(math.o): In function `string_entropy':
math.c:(.text+0xecc): undefined reference to `log2'
/usr/local/lib/libyara.a(math.o): In function `data_entropy':
math.c:(.text+0x1150): undefined reference to `log2'
collect2: error: ld returned 1 exit status
2018/02/20 19:08:52 Failed to cross compile package: exit status 2.
make: *** [linux-x64] Error 1
@Xumeiquer Please tell me what exact command you used to build your program so I can try to reproduce it.
One thing that strikes me as odd is that even though you seem to have built a custom copy of yara, libyara that has been installed to /usr/local/ is used by the build system.
Hi,
As I already said I am using xgo to cross compile my project. xgo uses docker with several compilers for several architectures. The project xgo allows to compile C-like dependencies as well and the way it does is by running ./configure, make and, make install. As you may know, Yara uses a bootstrap command before calling ./configure so I made a a fix for using xgo with Yara in https://github.com/Xumeiquer/xgo and especially in https://github.com/Xumeiquer/xgo/blob/master/docker/base/build_deps.sh.
What I suggest if you try to reproduce is to download the karalabe/xgo-latest and them modify the build_deps.sh file and generate a new image from the running container.
It may be a little bit tedious but it is a way to cross compile projects easily.
After all of this, the command line I use to build my project is:
DOCKER_IMAGE := "x-go-yara" ## Modified version of karalabe/xgo-latest
DEPS := "https://github.com/VirusTotal/yara/archive/3.7.1.tar.gz"
xgo -x -image $(DOCKER_IMAGE) -dest build -deps $(DEPS) -targets="linux/386" .
The same way to cross compile my project for Linux 64bit I use the following command:
DOCKER_IMAGE := "x-go-yara" ## Modified version of karalabe/xgo-latest
DEPS := "https://github.com/VirusTotal/yara/archive/3.7.1.tar.gz"
xgo -x -image $(DOCKER_IMAGE) -dest build -deps $(DEPS) -targets="linux/amd64" .
Just in case you ask, I can cross compile without any issues for Windows and Darwin.
A thing that seems odd to me is that you are patching specific configure options into the setup script; did --depsargs not work for you?
Could you try if adding -lm to the libs.private entry of yara.pc works for you?
Hi,
Using --depsargs doesn't work because what yara build process needs is to run the bootstrap first.
I also tried to modify the yara.pc, but unfortunately it doesn't work either.
What I have done in terms to "debug" this is running the build process manually step by step. I have discover that exporting the CGO_CFLAGS and CGO_LDFLAGS and adding -lm to CGO_LDFLAGS go-yara compiles well.
# export CGO_CFLAGS="-I${YARA_SRC}/libyara/include"
# export CGO_LDFLAGS="-L${YARA_SRC}/libyara/.libs -lyara -lm"
# go build -tags yara3.7
Do you have any clue about if it would be possible to add -lm as -ldflag or -extldflags basically without exporting the CGO_ variables?
Using --depsargs doesn't work because what yara build process needs is to run the bootstrap first.
Ah, this is probably why I roll my own scripts / Makefiles for cross-building instead of relying on tools like xgo. :-)
I also tried to modify the yara.pc, but unfortunately it doesn't work either.
Ah, I forgot to mention it: You'd have to pass the "yara_static" build tag, so the Go toolchain calls "pkg-config --static yara".
Assuming that the issue has been fixed or become irrelevant at some point in the last 5 years, I'm closing it.