Control which local dependencies are installed dev vs. stable versions

[gossi]

At the moment, all locally found packages are symlinked instead of being installed from composer. This might not be ideal, e.g. stable packages (which exists as dev versions locally) shouldn't be symlinked. There needs to be a mechanism to control this kind of installs.

Possible locations for config options (maybe both?):

• In .composer/config.json: Make a new node under config.localdev.options and another one for config.localdev.paths to separate them accordingly
• In the extra section of each root composer.json can some config for localdev that overwrites a global configuration.

Possible strategies to decide whether the dev version will be symlinked or the latest dev version from packagist will be installed:

1. If the version you install is a dev version (e.g. dev-master) than a symlink will be created, if the version refers to a major version (e.g. ~2) than the latest stable from packagist will be installed
2. require-dev overwrites require section. That would mean, the require section is for production and require-dev for development environment. require can consists of sth like this: "Symfony/Finder": "~2" and will be overwritten in require-dev with "Symfony/Finder": "dev-master". When the install/update happens in dev-mode and the package is available in require-dev section, than a symlink will happen. Anyway normal install follows. Can also be coupled with strategy (1)

Config options can be used to control the strategies.

[mfrieling]

As I commented in #4 I'm working on a Drupal 8 distribution architecture and I will suggest here a way to decide which packages to symlink and which ones to install from composer.

From the both options pointed out in this issue I prefer the second, keeping it local to the package instead of using the global composer.json. This is for a few reasons:

• You have the local modifications in place with the package
• You can use relative paths instead of absolute ones (e. g. ../packages/vendor) and thus have them in your working space which makes working with many IDEs easier.
• You can use the same local package in different projects with different branches, e. g. in your main development project use the dev-dev branch where in another project or another instance of the same project use the dev-hotfix/issuekey-serious-bug branch.

But as I do not like to have local development related stuff in Git I don't like to put it into the extra section of the root composer.json. My suggestion is to use a local override file like Drupal does with it's settings.php file which contains the database credentials and some other secret information besides general system configuration which must be available before a database connection is established. There you can add a settings.local.php file (or settings.dev.php, settings.production.php, you can choose the name free as it is a PHP file and the logic to load the local file must be in the main file). Then for example in the development version of the local file you put a static database connection as security about database user and password is not important there, whereas in the production version you can put logic to read the database credentials from system environment variables or a secure credentials store.

For this package my suggestion therefore is to use a root composer.local.json which must be put beside the root composer.json file and will be automatically loaded if it exists, but is ignored by Git (still you can put an example.composer.local.json file into Git which other developers can easily copy, rename and adapt to their needs). Instead of using config/localdev then I would use the names defined by composer itself with the following logic:

• Items in repositories array of the composer.local.json are evaluated before those in the composer.json. Thus if the local file defines a path repository and it exists, composer symlinks it from there, otherwise from the remote location defined in the root composer file. That can be packagist or an own Satis or Toran proxy server or similar.
• Other simple options which makes sense in the local file override the values in the root composer.json, like replace, minimum-stability or prefer-stable.

[mfrieling]

This is exactly how I mean it: https://www.reddit.com/r/PHP/comments/4lq9z2/whats_your_workflow_for_developing_with_composer/d3q6hjn/

[gossi]

I occasionally run into two situations where I didn't want to have my local package symlinked (I fixed it, by fixing the issue on the referenced package), though my own input for this is very rare. Thanks for describing your situation.

Though the question remains, at which location do I define my local strategy (other than in global ~/.composer/config). And which values need to be "overwritten" at which point. Also the main idea would be, if you strip out this plugin, will composer still work as expected, that's the main functionality to be persist here. I like the idea of having a local file (e.g. composer.local.json) which will overwrite the global config and which can be easily ignored via .gitignore so your local development setup doesn't pollute shared repos.

So, instead of using require and require-dev for the same package (which I somewhere read is bad practice in composer land ?!? I have no source on this) the composer.local.json can be used to define the strategy here. What is easy as a first start is to define sources for additional packages, as it could be merged with the global definition. And second is to define a strategy whether a locally available package will be symlinked or not. Could be whitelisiting or blacklisting, I'm open on this. Playing with stability flags is mostly a bad idea as it might not work as expected 😁 You maybe have an idea how?

[mfrieling]

First of all, I'm not a composer developer but just a user. So I have no knowledge about its internals and how it works etc. So to answer your question

which values need to be "overwritten"

the idea is that everything defined in the composer.local.json overwrites the corresponding value in the composer.json. Of course you need different strategies depending on the individual setting. Simple settings like minimum-stability or prefer-stable are easy, you just overwrite the value from composer.json. But for array structures like repositories, require or require-dev this is different. As far as I understood composer, the repositories array is evaluated in the order of defintion and further in order project root composer.json, global composer.json and then composer defaults (aka the main packagist). This means the items from composer.local.json repositories must be prepended to the merged array of all other sources. Then you can have a a package twig/extensions (original on packagist.org, defined in composer defaults) forked to provide some own extensions for the packages which should be integrated when they are stable (hosted as your own packagist package or on your own Satis instance, defined in composer.json) and for active development you add the path repository to your working clone of your fork in composer.local.json. Then locally you get your local working copy, on your (test) servers where either the plugin is not installed or no composer.local.json exists, it takes the forked package from its remote location. And as soon as your modifications are stable and your merge request is accepted, you remove the forked repository definition from composer.json and use the original package including your code. For bugfixing you just setup the definition to you local clone again and get it symlinked. Herewith we also have answered the question part

at which point

For

is to define a strategy whether a locally available package will be symlinked or not

I've read somewhere that for repository items there exists a symlink = true|false flag, but am not sure if it is in composer core or an extension. Of course this can be defined in composer.json and optionally overriden in composer.local.json (if not the value in composer.json counts). Or it can be defined only in composer.local.json (then overriding its default value). What likely becomes much more tricky – if it is possible at all – is generating the composer.lock file as if no composer.local.json was present, just with information taken from composer.json and the more generic files (global, defaults). That would be needed to always generate a production ready of the file without polluting it with development local stuff.

And yes, playing with stability flags is bad. But sometimes you have to. Think about finding a bug or getting a bug report and your analysis nails it down to one 3rd party contrib module you use. Then you find out that the latest release (which you are using) is quite old but there is a current dev version, and you don't find a corresponding issue. Then you might want to test if the bug is solved by the latest dev version or still exists. If it solves the bug you might get more information by looking on the diff between working and not working version and possibly find the issue with knowledge about the commit fixing it. Then you have to decide to either apply a patch fixing the bug, using the dev version of that module until its next release or do something else. This situation is not that rare as you might expect, especially if your customer needs exotic features which require exotic and less actively maintained modules where it takes lots of months until the next release.

Side information: All work in the organization behind the project which is bringing me here is voluntarily, so I do it in my spare time and might need a bit longer to reply.

[gossi]

Having a composer.local.json is basically a nice idea to overwrite any values of composer.json for your local setup. In that case that would be a feature request to composer itself. Thinking further you can define path repositories in your composer.local.json, that wouldn't be committed to the central repository. That would be a legimitate option, which would abandon the need to for this plugin but some that I'd also appreciate. Keeping composer.lock in sync as if composer.local.json is a tricky task as well, which also would fall into the hand of composer itself.

Well, what I'm looking for is some values that you'd define globally and you might overwrite at a given repo. With this plugin I globally define my local packaes and each of my repos will use that config to symlink to them. So which configuration I would like to overwrite at a repo, which is configured globally. E.g. I configured the package phootwork/collections globally for me but in one of my repos I don't want a symlink there but instead want to install it from packagist instead. That's the kind of research task at hand which I see for this plugin.