Support building without binaries in the clone

[eschaton]

It’s inconvenient to have to merge a binary distribution into a cloned repository in order to build it. It should be possible to build CCL without having binaries in the same directory, as long as ccl or ccl64 is in the PATH.

[informatimago]

Also, it would be nice if rebuilding would not overwrite the binaries used to rebuild.

[phoe]

@informatimago It is the Lisp kernel's makefile that overwrites the binaries. It could be parametrized in order to accept a new name for the resulting binary and its matching Lisp image.

[xrme]

More than one thing is at issue here.

The first is that the lisp kernel executable needs to find a heap image to load. By default, the executable looks in its own directory for an image.

Next, we need to know where the sources are. That's the "ccl:" directory (set via CCL_DEFAULT_DIRECTORY or else defaulting to the directory of the lisp kernel binary).

Finally, because the CCL sources use the #_ reader macro and friends, we need to know where the interface databases are. CCL currently expects them to be in some per-backed directory inside the "ccl:" directory.

The interface databases are the troublesome thing, it seems to me. I could imagine a new environment variable like CCL_INTERFACE_DATABASE or something that would specify where to look.

To avoid putting binaries in the clone, we'd be looking at some directory structure like this, probably:

ccl-src/
  compiler/
  level-0/
  level-1/
  tools/
    asdf.lisp
    ...
  ...

ccl-bin/
  darwin-x86-headers64/
  dx86cl64
  dx86cl64.image

Then, you'd set CCL_DEFAULT_DIRECTORY to /path/to/ccl-src (to specify the location of #p"ccl:"), and set CCL_INTERFACE_DATABASE to /path/to/ccl-bin/darwin-x86-headers64. We'd need to change CCL to pay attention to CCL_INTERFACE_DATABASE. Maybe we define "ccl:interfaces;" logical pathname and define the appropriate translations for that?

Anyway, all this is just thinking out loud, so to speak.

If it weren't for the interface database, we could almost do this today: set CCL_DEFAULT_DIRECTORY to /path/to/sources, start CCL from somewhere, and then do (rebuild-ccl :full t). But we need the interfaces to compile CCL.