Fail to compile with Clang because of -m64 linker flag

[lucic71]

I'm trying to compile john with Clang 15.0.7 and I get the following error:

/usr/bin/ld.lld mkvcalcproba.o -g -m64  -L/usr/local/lib -L/usr/lib64 -L/lib64   -lm  -o ../run/mkvcalcproba
ld.lld: error: unknown emulation: 64

I ran configure with the following env variables:

LD=/usr/bin/ld.lld CC=/usr/bin/clang CXX=/usr/bin/clang++ ./configure

The configure output shows the following line:

checking if /usr/bin/clang  supports -m64 w/ linking... yes

I greped in src/configure and found this: https://github.com/openwall/john/blob/2f4b2dfee51e1477c5147d5b02ee71dc04b3ce0d/src/configure#L7034

If I understand this right then this line runs the linker to see if -m64 flag is supported. However the linker is called through clang in this case. If you call ld.lld as a standalone program with the -m64 it will complain that 64 is an unknown emulation. The same case happens for GNU gold if called as standalone.

The linkers (both ld.ldd and GNU gold) indeed support some emulation flags, but they take different values (elf_x86_64, etc).

So why set the LDFLAGS to -m64 in the following line? https://github.com/openwall/john/blob/2f4b2dfee51e1477c5147d5b02ee71dc04b3ce0d/src/configure#L7078

Another thing that confused me is this: https://github.com/openwall/john/blob/2f4b2dfee51e1477c5147d5b02ee71dc04b3ce0d/src/configure#L7025

Why set 64 bit emulation flags for 64 bit platforms?

Thanks

[magnumripper]

The autoconf stuff initially mostly tried to mimic the original Makefile (now Makefile.legacy).

Not sure what you mean with emulation flags? As for adding it to LDFLAGS, I think what we do is correct as long as LD = $(CC) which is what we use. Then -m64 generates 64-bit code, not emulation.

We're open to improvements to the config stuff though.

[lucic71]

Sorry for the confusion. I was looking at the -m flag for the linker which is used for the emulation layer.

As for LDFLAGS, even if you call LD through CC you don't need -m64 as it does not affect in any way the linking step. It only affects the compilation step.

[magnumripper]

So maybe we do it wrong, and even the legacy Makefile was wrong in this regard. What's the reason you want to specify LD as ld.lld explicitly?

[lucic71]

I wanted to link with ld.lld but meanwhile I realized that it's safer to call the linker through clang.

[claudioandre-br]

I wanted to link with ld.lld but meanwhile I realized that it's safer to call the linker through clang.

In this case, the best option would be to overwrite settings using the flag [1] -fuse-ld=<linker name>? If so, the issue is fixed/invalid and should be closed (after your final thoughts and tests).

[1] In this way, you want to set -fuse-ld=lld to LDFLAGS when building your programs.

[solardiz]

the issue is fixed/invalid

I think it's valid criticism that we use LDFLAGS with CC rather than with LD.

[lucic71]

@claudioandre-br

In this case, the best option would be to overwrite settings using the flag [1] -fuse-ld=?

Yes. However, as @solardiz points, it's rather awkward to add -m64 in LDFLAGS when we know that it won't be used by the linker.

A last thought, shouldn't the compiler generate 64 bit code by default on a 64 bit host? In this case why bother adding the -m64 flag?

[magnumripper]

A last thought, shouldn't the compiler generate 64 bit code by default on a 64 bit host? In this case why bother adding the -m64 flag?

Nowadays I guess most or all do, but 13 years ago that wasn't necessarily the case. It simply depends on how the build tools were configured when they were built themselves. Same goes eg. for SIMD by the way - IIRC Apple's compilers default to SSE 4.1 (or whatever it was) because all intel Macs had at least that, while the only things that's actually guaranteed by any x86-64 CPU is SSE2.

[lucic71]

Should we close this issue?

[claudioandre-br]

• IMO:

I think you could propose a patch that does what you believe is right for us to test (I, for example, can't imagine what would be best).

At some point I think it's desirable that we forget about old platforms, so changes that impact them are acceptable (probably this moment is after the john 2.0 release, but we can try to do something now).

Anyway, there is no guarantee that your suggestion is really the right one or that the focus on recent compilers and platforms will be accepted as the best solution. Note, the IMO part.

[solardiz]

See also #5288. I think we need a better fix than simply dropping this flag.

[solardiz]

@lucic71 How about we drop x86_64 from the case statement? On this arch, -m64 at link stage was possibly only needed in early cross-compiles. We similarly don't list arm64 / aarch64 in there as a modern arch that presumably didn't need this.

[lucic71]

@solardiz sounds good to me

I'm going to update #5286 with the proposed changes.

[solardiz]

I'm going to update #5286 with the proposed changes.

Yes, please.

[solardiz]

How about we drop x86_64 from the case statement? On this arch, -m64 at link stage was possibly only needed in early cross-compiles.

BTW, besides early cross-compiles, I guess it would also be needed if one wants to build for full x86_64 while running on x32, which was just a few years ago a curious Linux arch target for x86_64 with 32-bit pointers (more memory efficient and faster than full x86_64, at the expense of having only 4 GiB of address space per process). We still support it since 2014 with ac76e8d503dede31cbcb3d3878986954e96245aa, but that support should be unaffected by dropping of -m64 - only cross-compiles to full x86_64 off such systems (producing binaries that would run on those systems?) would be affected. I also added support for x32 to yescrypt in 2019 IIRC after a problem report via libxcrypt that this wasn't working. So maintained distros for x32 existed as recently as 2019. I didn't keep track after that.

Just sharing my thoughts. I don't think this should stop us from making the change.

[claudioandre-br]

How about we drop x86_64 from the case statement?

What worries me is Solaris on X86 but it works fine (tested). I don't expect problems in any other CI environment.

0001-autoconf-discard-m64-from-LDFLAGS-on-X86.txt

[solardiz]

This issue was fixed for x86-64 via #5286 and we decided not to fix it for other archs yet. Closing.