shards

shards copied to clipboard

Provide a feature similar to bundler groups.

# Requires all shards listed in shard.yaml.
Shards.require

# Requires a group.
Shards.require :development

One possible way to implement this feature:

1. Have shards generate one .cr per group.
2. Shards.require requires the generated file(s).

Other bundler features may or may not be included.

What's the usecase?

Less work in each application maintaining require statements?

My primary purpose in grouping is multiple apps within 1 repo.

For example I have a web service, command line app and queue consumer. ~50% of the gems and library are shared. Almost all of the application code is shared. The web framework and many supporting shards are unique to the web service. TTY/console and a few misc shards are unique to the command line app. The queue service uses neither. Database, queue, and s3 etc are common between them.

This is impossible to do because require is a compile-time thing while here it seems you need to parse the shard.yml at runtime to know this.

Well, it's possible using macro run, but I don't encourage that. You can always write a shard to do that

Oh, well, if a cr is generated then it can be done using macros... but in general I don't know if requiring all shards at once is good. Specially, some shards might depend on others and the require order you need io use must be very specific, and using the same order in shard.yml might be tricky or not obviously related.

@asterite If shard a depends on b, doesn't a normally require "b" solving most ordering problems?

@didactic-drunk Is this only about compile-time requires in Crystal code? In my experience the main use case for bundler groups is to group dependencies and be able to selectively install them as necessary using bundle install --with group and bundle install --without group. So that would affect how shards installs dependencies.

Another use case it comes to my mind is when an shard provides an extension or an implementation of some kind. For example, database connectors for crystal-db. Currently we must explicitly require the specific connector.

Actually generating the files within a "pseudoshard" would be really simple. Shards knows the dependency graph so the statements could be generated in the right order. And then just use require "_shards/group" to load that file. No need of macros or anything fancy.

Of course this first needs some way of defining shard groups within shard.yml.

My initial use case was for applications not libraries.

Your solution works except for for partial inclusion:

1. Application a uses shard foo and requires "_shards/group".
2. Shard foo uses part of shard bar by requiring bar/baz.
3. Shard foo doesn't want to include all of bar.

Possible fixes are opting in or out from graph inclusion with a flag or two. Preferably having a flag for all shards or per dependency. Or specifying what to require.

dependencies:
  foo:
    github: foo/foo
    require: bar/baz
  bar:
    github: bar/bar
    require: false

Allowing to specify require key would solve the issue of requiring different but equally-named shards in the same time 👍

@Sija , why? As long as the shard key indicates where to install in the CRYSTAL_PATH (.crystal/shards / lib) the scenario you mention will still not be handled.

@bcardiff Oh, right. I forgot that shards are not scoped within the lib directory.