[help] Advice for setting up this custom build

[tmcdonell]

At Haskell compile time (specifically, at the template-haskell phase), my library is generating extra object files (.o files) which need to be linked together with the final library/executable.

My current approach uses this GHC plugin to collect the extra generated .o files and add them to the GHC command line args. A bit of a hack, perhaps, but seems to be working for the following use cases:

• ghci
• ghc --make
• cabal when building an executable (example project)

What doesn't work is cabal building a library. Here is a minimal example project. When cabal builds the final static & dynamic libraries, I need it to also include these extra .o files.

I'm happy to tell my users that they need to use a custom Setup.hs with defaultMain etc. that my library supplies, but do you have any suggestions for the easiest way to pass in these extra files without just duplicating the entire buildHook to add those here?

(If building a library only I might be able to work around this with a post-build hook, but in the attached library+executable example that won't work because the post-build hook runs after building both the library and executable. Unfortunately you never reach that point because the library is missing these extra symbols so the executable link fails.)

Thanks!

---

From the linked accelerate-cabal-test gist:

> stack build --cabal-verbose
...
[1 of 1] Compiling Lib              ( lib/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Lib.o )
Data.Array.Accelerate.LLVM.Native.Plugin: linking module ‘Lib’ with:
  /Users/trevor/.accelerate/accelerate-llvm-1.0.0.0/accelerate-llvm-native-1.0.0.0/x86_64-apple-darwin15.6.0/ivybridge/rel/meepF807D1E151E6A2B5.o

Linking...
...
/usr/bin/ar -r -s .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/objs-26544/libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Lib.o
# Above line needs to include the extra object file(s) when building the final static lib. Similarly for the dynamic lib.
# When using the library, linking of course now fails:
...
[1 of 1] Compiling Main             ( exe/Main.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp/Main.o )
Linking...
/Users/trevor/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc --make -fbuilding-cabal-package -O -static -outputdir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -odir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -hidir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -stubdir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -i -i.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -iexe -i.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen/cabal_macros.h -hide-all-packages -no-user-package-db -package-db /Users/trevor/.stack/snapshots/x86_64-osx/lts-9.0/8.0.2/pkgdb -package-db /Users/trevor/Desktop/accelerate-cabal-test/.stack-work/install/x86_64-osx/lts-9.0/8.0.2/pkgdb -package-db .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/package.conf.inplace -package-id accelerate-1.0.0.0-FhjhkdxKbGpC7beQBhvKOz -package-id accelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV -package-id base-4.9.1.0 -XHaskell2010 exe/Main.hs -o .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test -O2 -threaded -rtsopts -Wall -ddump-hi -ddump-to-file
Linking .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test ...
Undefined symbols for architecture x86_64:
  "_foldAllP1_f807d1e151e6a2b5", referenced from:
      _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllP1zuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)
  "_foldAllP2_f807d1e151e6a2b5", referenced from:
      _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllP2zuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)
  "_foldAllS_f807d1e151e6a2b5", referenced from:
      _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllSzuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)
  "_generate_f807d1e151e6a2b5", referenced from:
      _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezugeneratezuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
`gcc' failed in phase `Linker'. (Exit code: 1)

[angerman]

Hi Trevor,

how about turning your object files into an archive? Say libX.a, and add ghc-options: -L/path/to/libX.a and -lX? You can create an archive of object files using ar q libX.a object.o. This would essentially be like treating your generated object files like a static external library.

Cheers, Moritz

On Aug 8, 2017, at 1:20 PM, Trevor L. McDonell [email protected] wrote:

At Haskell compile time (specifically, at the template-haskell phase), my library is generating extra object files (.o files) which need to be linked together with the final library/executable.

My current approach uses this GHC plugin to collect the extra generated .o files and add them to the GHC command line args. A bit of a hack, perhaps, but seems to be working for the following use cases:

• ghci • ghc --make • cabal when building an executable (example project) What doesn't work is cabal building a library. Here is a minimal example project. When cabal builds the final static & dynamic libraries, I need it to also include these extra .o files.

I'm happy to tell my users that they need to use a custom Setup.hs with defaultMain etc. that my library supplies, but do you have any suggestions for the easiest way to pass in these extra files without just duplicating the entire buildHook to add those here?

(If building a library only I might be able to work around this with a post-build hook, but in the attached library+executable example that won't work because the post-build hook runs after building both the library and executable. Unfortunately you never reach that point because the library is missing these extra symbols so the executable link fails.)

Thanks!

From the linked accelerate-cabal-test gist:

stack build --cabal-verbose ... [1 of 1] Compiling Lib ( lib/Lib.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Lib.o ) Data.Array.Accelerate.LLVM.Native.Plugin: linking module ‘Lib’ with: /Users/trevor/.accelerate/accelerate-llvm-1.0.0.0/accelerate-llvm-native-1.0.0.0/x86_64-apple-darwin15.6.0/ivybridge/rel/meepF807D1E151E6A2B5.o

Linking... ... /usr/bin/ar -r -s .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/objs-26544/libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Lib.o

Above line needs to include the extra object file(s) when building the final static lib. Similarly for the dynamic lib.

When using the library, linking of course now fails:

... [1 of 1] Compiling Main ( exe/Main.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp/Main.o ) Linking... /Users/trevor/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc --make -fbuilding-cabal-package -O -static -outputdir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -odir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -hidir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -stubdir .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -i -i.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -iexe -i.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen -I.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test-tmp -optP-include -optP.stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/autogen/cabal_macros.h -hide-all-packages -no-user-package-db -package-db /Users/trevor/.stack/snapshots/x86_64-osx/lts-9.0/8.0.2/pkgdb -package-db /Users/trevor/Desktop/accelerate-cabal-test/.stack-work/install/x86_64-osx/lts-9.0/8.0.2/pkgdb -package-db .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/package.conf.inplace -package-id accelerate-1.0.0.0-FhjhkdxKbGpC7beQBhvKOz -package-id accelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV -package-id base-4.9.1.0 -XHaskell2010 exe/Main.hs -o .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test -O2 -threaded -rtsopts -Wall -ddump-hi -ddump-to-file Linking .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/accelerate-cabal-test/accelerate-cabal-test ... Undefined symbols for architecture x86_64:

"_foldAllP1_f807d1e151e6a2b5" , referenced from: _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllP1zuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)

"_foldAllP2_f807d1e151e6a2b5" , referenced from: _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllP2zuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)

"_foldAllS_f807d1e151e6a2b5" , referenced from: _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezufoldAllSzuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o)

"_generate_f807d1e151e6a2b5" , referenced from: _acceleratezmcabalzmtestzm0zi1zi0zi0zmLBTQYbTCMWLDsNOhtYaXOV_Lib_zuzuacceleratezullvmzunativezugeneratezuf807d1e151e6a2b5_closure in libHSaccelerate-cabal-test-0.1.0.0-LBTQYbTCMWLDsNOhtYaXOV.a(Lib.o) ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation)

gcc' failed in phase Linker'. (Exit code: 1) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[tmcdonell]

Hi Moritz,

Thanks for the suggestion, I'll see if I can make this work.

Also, just to clarify my initial message, the .o files are temporary files generated from user code.

[angerman]

Hi Trevor, let me see if I understood or misunderstood your question.

You plugin will (via Template Haskell produce object files at compile time), however you want to ensure that those are linked in the final link phase.

Assuming we have a fixed path, say accelerate-dist/libaccelerate.a, we'd need to provide the final linking step with -Laccelerate-dist -laccelerate.

So the plugin would (after creating the object file) invoke:

ar q accelerate-dist/libaccelerate.a object.o

(you might need to do some path magic here to get this all relative to the project root).

Thus you would collect the (temporary) object files at compile time in the libaccellerate.a archive. (Archive files can have the same name multiple times. Once we end up in linking the library, we have collected all the object files in the archive at a known location, and the link should succeed.

Let me know if this sounds plausible, or I am missing something obvious.

Cheers, Moritz

[phadej]

@tmcdonell completely wild guess: check what inline-c -package is doing.

[tmcdonell]

@angerman Yep, that sounds right. I currently have the plugin generating the .a file, and am working to get cabal to use this in the right way/copy to the final install location/etc, but this seems to be making progress. Being able to load the installed library into ghci will be the next hurdle.

@phadej It's a good idea, I looked into this too. Older versions (< 0.6) required that for each Foo.hs which used inline-c, the .cabal file included a c-sources: Foo.c line, and then cabal sorted it out as usual. From 0.6, they have gotten changes into template-haskell-2.12 so this happens automagically without changing the .cabal file, but currently this is only for C-like files, much like the c-sources stanza. Extending that mechanism might be a good direction for me to take for the future though.

[angerman]

@tmcdonell, why do you need to copy the final archive? Shouldn't the linking step pull in all required symbols at link time, and just leave you with the final library/executable? If you build the library with cabal (and hence have the archive linked into the final distibutable), ghci -package ... should be sufficient, no?

Anyway, let me know. If you need some .a merging logic, I have an unreleased package that can read/write GNU and BSD archives, but still requires a ranlib pass to produce the symbol index, which I use for the generic -staticlib in ghc 8.4.

Or just ping on me irc.

[tmcdonell]

It is enough to build the executable, yes. But, the library component is still missing those symbols:

> nm .stack-work/install/x86_64-osx/lts-9.0/8.0.2/lib/x86_64-osx-ghc-8.0.2/accelerate-cabal-test-0.1.0.0-IaFjUzMCIsr6GDzr6kbbzB/libHSaccelerate-cabal-test-0.1.0.0-IaFjUzMCIsr6GDzr6kbbzB.a | grep _foldAll
                 U _foldAllP1_8650db2480dc7309
                 U _foldAllP1_f807d1e151e6a2b5
                 U _foldAllP2_8650db2480dc7309
                 U _foldAllP2_f807d1e151e6a2b5
                 U _foldAllS_8650db2480dc7309
                 U _foldAllS_f807d1e151e6a2b5

similarly, the shared library also contains these undefined symbols.

Updating the .a in a postBuild step should be easy enough (e.g. in the way you suggest), so I think regular static builds will be fine.

I'm currently still investigating ways to update the .so/.dylib so that it finds those symbols when loading into ghci. At least my mac is not happy with ghci -package ... (including when explicitly pointing it to the new .a or original .o files containing the necessary symbols; I guess that is just the way dlopen works). I haven't tried on linux yet.

Do you happen to know if you can add an additional shared library dependency / LD_LOAD_DYLIB entry to an existing .so/.dylib?

[angerman]

@tmcdonell I would have assumed that the -L and -l flags during the link phase of the dynamic library would have pulled in those symbols. However if those symbols are not directly visible to the linker, it might now have pulled them in (could it be that only the executable makes actual use of the generated functions, and the library in itself does not?). The best would be to grab the linker command (by providing ghc with -verbose), and investigate what's happening.

You could experiment with the -all_load and -force_load linker flags:

     -all_load   Loads all members of static archive libraries.
[...]
     -force_load path_to_archive
                 Loads all members of the specified static archive library.  Note: -all_load forces all members of all archives
                 to be loaded.  This option allows you to target a specific archive.

(from man ld)

Then there is also libtool on macOS, which allows creating static and dynamic libraries. I have however stayed away from using libtool as the one that ships with macOS is rather different than the GNU version, and I was aiming for a cross platform solution.

[tmcdonell]

Following up, I ran out of time to get this working for the dynamic libraries build, so for the time being just did the simple thing of copying the build_hook step to add these lines.

[deech]

@tmcdonell Not sure if this helps but when I wanted to link with a static archive produced in my library in my app I had to add the following line to my app's Cabal: https://github.com/deech/fltkhs/blob/master/fltkhs.cabal#L183.

[dfordivam]

@deech This worked for me!

It should be documented properly somewhere. It took me weeks to figures this out. At one point I gave up and started using the ghc command line to build the whole project!

[hsyl20]

For the record, the line is:

ghc-Options: -pgml g++ "-optl-Wl,--allow-multiple-definition" "-optl-Wl,--whole-archive" "-optl-Wl,-Bstatic" "-optl-Wl,-lfltkc" "-optl-Wl,-Bdynamic" "-optl-Wl,--no-whole-archive"

[RyanGlScott]

As far as I can tell, ld on macOS doesn't support -Bstatic, so I don't think the trick in https://github.com/haskell/cabal/issues/4677#issuecomment-625365136 would work on macOS.

[ulysses4ever]

Is it okay to close this rather old user-question issue?

[gbaz]

The answer, insofar as there is one, seems to be much more about ghc and g++ options, not about cabal per se. so i vote close.