build-script location

[awvwgk]

Currently the build-script can specified in the [library] table with:

library.build-script = "build.mk"

For an executable only project, a library table would still be required to select a build script, even if no library sources are actually available.

Adding the build-script to the [build] table feels more intuitive. Either as build-script or to reduce redundancy, just script:

build.build-script = "build.mk"
build.script = "build.mk"

This might be a breaking change for some projects using this fpm feature.

[ivan-pi]

In Cargo, the default behavior is to check for a file named build.rs in the root of the package. A custom build script can be specified as:

[package]
# ...
build = "custom_build_name.rs"

To disable automatic build script detection the setting is simply build = false. The possibility to use other build systems is then delegated to custom user packages which are called from build.rs.

The suggestion from @certik in #94 was to allow any kind of build script, be it a binary executable, shell script, Makefile, or other build system. Since fpm is only allowed to interact with the build script through environment variables and the output of the build script to standard output prepended with fpm:. This seemed like a reasonable idea. Upon further thought, I am worried this might become an obstacle for cross-compatibility between different operating systems. In this case fpm will need to report to the user to install CMake, Meson, Make, bash, etc. or whatever build system necessary.

In any case, do we also want to support such automatic build script detection?

Related to the immediate issue above, I agree that the [library] section does not feel right in case of executable-only projects. The build.script seems more fitting, also considering the fact the link keyword is in the same table.

[awvwgk]

I always wondered why Makefiles take such a special role in fpm, while CMake and meson are standardized enough to give a somewhat predictable behaviour for third-party tools, a Makefile can hide all kind of surprises and caveats inside (I never get tired of a random Makefile suddenly starting to write stuff in my home directory).

Interacting with other build files will always be difficult for fpm, have a look at https://github.com/fortran-lang/webpage/issues/64 for an incomplete list. Preferably we can convince projects to switch to fpm, but most likely we will end up with projects supporting fpm and their original build system. My projects currently all support meson, but many have additional fpm support or CMake support, depending on the communities they find use in.

In any case, do we also want to support such automatic build script detection?

Please don't, those features should be opt-in only, I don't want fpm to start auto-detecting my meson build files and force me to disable another of the automatic detection features in my projects.

[ivan-pi]

I certainly agree we don't want fpm interacting with other build systems aimed to be use independently from fpm.

My proposition/question was only related to the way Cargo automatically looks for a build.rs file. Under this model behavior, and the suggestion from @certik, fpm could look for build.sh, build.mk, build.cmake, etc.

But given that Fortran is not a monolithic/centralized language like Rust, and that there is a large variety of different build systems in everday use, it makes more sense to require package developers to specify a build script explicitly and save us the trouble from having to disable it manually.

(I admit to having a soft spot for auto-detecting a build.f90 program; the idea of writing a Fortran executable to specify a foreign language package build sounds both crazy and very fascinating at the same time.)

[awvwgk]

Okay, let's try to make the manifest syntax a bit more concrete.

I would propose we could have external build scripts (as array of strings), they are invoked by fpm and we just hope they play nicely together with fpm, but guarantee for nothing:

build.script = ["make", "-f", "build.mk"]  # or just ["build.mk"], make extension is detected
build.script = ["sh", "build.sh"]  # or just ["build.sh"], shell extension is detected
build.script = ["python", "build.py"]  # or just ["build.py"], Python extension is detected
build.script = ["ruby", "build.rb"] # or just ["build.rb"], Ruby extension is detected
build.script = ["cmake"]  # we might detect that it is CMake and run multiple steps for this script

And actual build scripts like proposed in fortran-lang/fpm#94 (provided as string), we will allow any format supported by fpm (f90, f, F90, F and c) as well as scripts (we can detect extensions as well and try to call the script with the correct program):

build.script = "build.f90"  # compile and run Fortran executable
build.script = "build.c"  # compile and run C executable
build.script = "build.sh"  # run shell script
build.script = "build.py"  # run Python script
build.script = "build.rb"  # run Ruby script

The script is expected to produce fpm: instructions which fpm parses and uses to build the project.

This might require a new table of build-dependencies at some point, which are usable in the build script.

[ivan-pi]

This might require a new table of build-dependencies at some point, which are usable in the build script.

This only seems to make sense for build scripts which are C or Fortran executables, allowing fpm to resolve (and reuse) their dependencies. Am I right? (I see many signs fpm will ultimately evolve also into a C package manager. )

Is there any intrinsic benefit to supporting build scripts in dynamic languages directly (apart from user convenience)? This could be done after all in a Fortran main program:

! build.f90
write(*,*) "fpm:rerun-if-changed=build.py"
call execute_command_line("python build.py")
end

[LKedward]

I wonder whether we should perhaps separate discussion of build scripts into those with one-way and two-way communication with fpm. The former, implemented experimentally in Haskell version, receive inputs via environment variables and are expected to place libraries in the correct location. The latter have reverse-communication with fpm via stdout (#94) for more advanced behaviour.

My intention for fortran-lang/fortran-lang.org#219 was only to initially implement one-way build scripts to support cases such as fortran-lang/fortran-lang.org#341. In terms of specification I think that we should avoid hard-coding the detection of specific file-extensions or build systems and simply have an array of strings to be invoked at the command line by fpm (I like your proposed syntax for this above @awvwgk).

While I like the idea of build.f90 etc., I'm hesitant to prioritise build scripts with reverse communication at such an early stage in fpm development. IMHO I think we should encourage package maintainers to adopt the native fpm package structure and work with them to improve fpm accordingly.

[ivan-pi]

That is a good way to look at it. I imagine in a one way setup, the script invoked would be responsible to place all the necessary executable files, module files, and binaries into a specified folder.

Would it then be up to the package developer to hard code any flags required for linkage straight into the manifest of the package?

[ivan-pi]

In terms of specification I think that we should avoid hard-coding the detection of specific file-extensions or build systems and simply have an array of strings to be invoked at the command line by fpm (I like your proposed syntax for this above @awvwgk).

Would this also work in the eventual case of a Fortran main program? For example:

build.script = ['$FPM_FC', 'build.f90', '-o', 'build', '&&', './build']

I can imagine this would become unwieldy, if my build program requires some other fpm-sourced modules.

But I agree to prioritise the first case (no reverse communication), as it would already enable a lot more complex projects.

[awvwgk]

Would this also work in the eventual case of a Fortran main program? For example:

build.script = ['$FPM_FC', 'build.f90', '-o', 'build', '&&', './build']

This should not be allowed, because we will guard the command execution against variable expansion and command chaining at some point, see fortran-lang/fpm#166.

I took some inspiration from docker where entry points and commands are defined differently by syntax, of course we can separate the one and two way communication more easily by allowing separate entries which are mutually exclusive. The advantage of only allowing a string for a two way build script instead of a full command line is that we move the complexity from the package manifest (configuration file) to the build script (full programming language).