Project manifests, for codebase-wide imports and exports

[rocketnia]

There are some mixed incentives around importing. If we import two libraries using simple require declarations, they could both add a new export with the same name and cause a conflict. So it's a little safer to use only-in or prefix-in.

This is the same issue brought up in https://github.com/racket/rhombus-brainstorming/issues/159, where a better form of prefixed imports is proposed.

In my projects, rather than prefixes, I've made extensive use of only-in. There are relative benefits between the two, but I'm not that picky, and this is more or less just the decision I made at the time.

My only-in forms can get rather long, with 30 or more names I'm importing from each of several modules. This might be when most people reach for prefix-in. Me, I've started to explore abstracting my imports into a single macro that I implement in one module and invoke from each module that needs it.

Doing it this way seems a lot like a codebase-wide import. And if I want to refer to something the same way throughout a codebase, isn't a codebase-wide import exactly what I want?

Dedicating a file to this also creates an opportunity to use the same file for codebase-wide exports. In current practice with Racket, it's possible to require any module or submodule from a package, even the ones that are supposed to be for internal use only. Currently, putting "private" into the module path is a convention that works out rather well, and I don't mind keeping that up. But if there's any need for more a programmatic representation of these design intentions, this file is where it could go.

Proposal

Modules in a Rhombus codebase could optionally start with a #project line after their usual #lang line:

#lang rhombus
#project example-lib

...

When a file like this is compiled, it looks up "example-lib/info.rkt". That info file can specify the imports and exports directly if they're simple enough, or it can have a line like (define project 'example-lib/project) that redirects to a module that specifies the imports and exports programmatically.

(By visiting an info.rkt file first, we can coalesce this information into the manifest files Racket already has. Only a sufficiently complex project would need additional control.)

The #project line is optional. If it's omitted, then the manifest simply specifies its own import and export information in the traditional way.

The #project line could vary between different files in the same codebase, which would simply cause different parts of the codebase to use different import and export policies for themselves.

The policies are applied by the individual modules

Note that most modules in this kind of codebase will have a compile-time dependency on the project manifest. Racket's module system doesn't permit cyclic module loading, so the project manifest won't be able to have a compile-time dependency on those modules.

The information in the project manifest may nevertheless refer to specific modules in the codebase. For instance, if it contains export information, it might specify certain modules that are exported and certain modules that are not. References like these don't create a compile-time link; instead, they're merely represented by a quoted module path until some other module follows up on the reference.

The actual application of these policies would happen when each module that has a #project line is compiled. During compilation, it looks up the project manifest information, discovers the import and export policies, and adjusts its own behavior based on that information (mainly, inserting imports).

Related issues

This is a small part of an approach I've been considering for a #lang with cyclic dependencies and parameterizable modules. Those features can make dependencies a bit more complex, at which point there's even more that can be abstracted away into a codebase-wide policy like this.

• https://github.com/racket/rhombus-brainstorming/issues/159: As mentioned above, both of these issues try to tip the balance towards backwards-compatible imports rather than the currently more convenient catch-all imports.
• https://github.com/racket/rhombus-brainstorming/issues/9: There's a bit in common between imposing an API specification (in that case contracts or types) on code from the outside and abstracting the API specification (in this case imports and exports) into a central location. However, I think the kind of "compile them in a variety of policies" flexibility @jeapostrophe discusses is likely more tied to parameterizable modules (with compile-time arguments that can carry the policies). Project manifests are, at most, only one part of the picture, and the policies @jeapostrophe imagines would probably be specified by the user of a package, not by a manifest that comes with the package. Still, I suppose defaults of these policies would be right at home in the manifest.

What about info.rkt files?

Yeah! If you see the details above, I'm proposing a technique where individual modules in a codebase look up an info.rkt file to find what they should import. If the needs get more complex, they can spill over into another module with more programmatic capabilities.