drush-ops
Can't specify alias-path values unless the --config argument is used to specify the drush/drush.yml config file.
Describe the bug
I am attempting to specify a directory for scanning and loading of site alias files. I'm using the drush.paths.alias-path config inside the top-level drush/drush.yml file. drush does not seem to be scanning the specified directory unless I add --config=drush/drush.yml to the command line.
When specified as an argument, the test file, drush/local/self.site.yml, is loaded and the site aliases become available. Without the extra argument on the CLI, it appears that the drush.paths.alias-path values are ignored in drush/drush.yml even though that config file is definitely being loaded for other configuration information.
My goal is to be able to specify non-standard directories for site alias files so that I can switch between them based on the server environment. So various options like /drush/local/, /drush/dev/, /drush/prod/, etc. This is going to be a Drupal multisite installation and we want to be able to use the same aliases, for each site of the multisite, no matter which server environment we are operating on. Dynamically altering the directory appears to be the best option for doing that.
I don't know if this is a bug or just my lack of understanding of how site alias files are scanned.
To Reproduce
Inside /var/www/html/drush/drush.yml add:
drush:
paths:
alias-path:
- /var/www/html/drush/local
Create file /var/www/html/drush/local/self.site.yml with contents like:
# File: self.site.yml
test:
root: /var/www/html/
uri: https://test.example.com
Execute drush sa --debug. My results are like:
[preflight] Config paths: /var/www/html/vendor/drush/drush/drush.yml,/var/www/html/drush/drush.yml
[preflight] Alias paths: /var/www/html/web/drush/sites,/var/www/html/drush/sites
[preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/web/drush,/var/www/html/drush
[debug] Starting bootstrap to none [0.06 sec, 3.11 MB]
[debug] Drush bootstrap phase 0 [0.06 sec, 3.11 MB]
[debug] Try to validate bootstrap phase 0 [0.07 sec, 3.11 MB]
[success] No site aliases found. [0.07 sec, 3.12 MB]
As you can see, no site aliases found and no mention of the drush/local directory under "Alias paths".
Execute differently as drush sa --debug --config=drush/drush.yml and results are like:
[preflight] Config paths: /var/www/html/drush/drush.yml,/var/www/html/vendor/drush/drush/drush.yml,/var/www/html/drush/drush.yml
[preflight] Alias paths: /var/www/html/drush/local,/var/www/html/web/drush/sites,/var/www/html/drush/sites
[preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/web/drush,/var/www/html/drush
[debug] Starting bootstrap to none [0.06 sec, 3.13 MB]
[debug] Drush bootstrap phase 0 [0.06 sec, 3.13 MB]
[debug] Try to validate bootstrap phase 0 [0.06 sec, 3.13 MB]
'@local.self.test':
root: /var/www/html/
uri: 'https://test.example.com'
The '@local.self.test' alias is loaded and notice how /var/www/html/drush/local now appears un "Alias paths".
I have verified that drush/drush.yml is loading normally and you can also see it in the "Config paths" portion of the debug log. When the --config argument is used, it is actually listed twice.
I have verified that the drush/drush.yml file is loading by setting options.uri: 'http://example.com/subdir' in the drush/drush.yml file. drush status shows that the URI value is set. It is the same with or without the --config argument.
Expected behavior
I would expect the same alias file to be loaded with or without the --config argument.
Actual behavior
It alias file is not loaded and the directory is not listed under "Alias paths"
System Configuration
| Q | A |
|---|---|
| Drush version? | 10.6.2 |
| Drupal version? | 9.3.0-rc1 |
| PHP version | 7.4 |
| OS? | Linux (ddev) |
Each config file has a very specific priority. The drush/drush.yml configuration file is loaded after the Drupal site has been identified. Configuration from an alias needs to be available in order to select the Drupal site, ergo the specified alias is loaded before drush/drush.yml is loaded, ergo adding to the set of places aliases are loaded from at this stage has no effect.
Okay. I thought all config files get merged (I think I saw that in a file comment somewhere). But also, there is no site yet because drush needs the alias since this is a multisite. So without specifying an alias, the only site that can be identified is the default?
Maybe I have to try something in the /web/sites/all/ directory? I think that drush.yml files do load from inside there. Or at least they did in a previous Drush version.
Drush 9+ is intended to be used as only a site-local Drush, installed alongside Drupal via Composer. If your aliases are for different environments of the same site (e.g. test and live), the recommended solution is to list them in self.drush.yml
If you have a more esoteric need, perhaps a shell alias that maps drush to drush --config=... might meet your requirements.
This is a site-local Drush, so I’m still confused. The site just happens to function as a multisite, so I need to be able to set the URI option by alias. All that works fine with /drush/sites/self.site.yml.
My goal is to be able to load /drush/{$env.ENV_NAME}/self.site.yml aliases and to specify that path in the site-local /drush/drush.yml file via the drush.paths.alias-path option. I must be misunderstanding the purpose of that file or that option.
If specifying the URI option by alias is working fine with /drush/sites/self.site.yml, I am confused about what your issue with paths is. What are you trying to do that isn't working?
Thank you for taking your time to work through this with me.
Drush correctly loads the alias list and config if I put it in drush/sites/self.site.yml.
I am trying to specify alternate paths to scan and load a single alternate self.site.yml alias config file depending on an environment variable. That is not working unless I add the --config argument to the command.
For our local development environments, drush/local/self.site.yml will look something like this:
default:
uri: https://site-default.ddev.site
site_a:
uri: https://site-a.ddev.site
site_b:
uri: https://site-b.ddev.site
For our QA test environment, drush/test/self.site.yml will look something like this:
default:
uri: http://site-default.test.private.cloudvpn
site_a:
uri: http://site-a.test.private.cloudvpn
site_b:
uri: http://site-b.test.private.cloudpvn
drush/prod/self.site.yml will look something like this:
default:
uri: https://www.example.com
site_a:
uri: https://site-a.example.com
site_b:
uri: https://site-b.example.com
(These are just example site names and URIs. In reality, we have more internal sub-domains under private.cloudvpn. These subdomains all happen to be located in AWS and we are required to ssh into them after using a VPN connection)
Then any developer on my team can execute drush @site_b status in any environment and drush will bootstrap the correct multisite settings.php file.
I'm trying to avoid having to deploy different files per environment. I was hoping to be able to use the drush.paths.alias-path option, with an ${env.ENVIRONMENT_NAME} reference, and deploy all the variations of self.site.yml needed for the different environments.
I am testing & debugging by using drush/local instead of drush/${env.ENVIRONMENT_NAME}
So when I have drush/drush.yml configured like:
drush:
paths:
alias-path:
- drush/local
Drush does not load the drush/local/self.site.yml file unless I specify --config=drush/drush.yml on the command line. Doing this also causes drush/drush.yml to be listed twice as one of the paths in the config preflight.
In the final drush/drush.yml that we would commit to the repo, the config will look something like:
drush:
paths:
alias-path:
- '${env.HOME}/.drush/sites'
- /etc/drush/sites
- 'drush/${env.ENVIRONMENT_NAME}'
Incidentally, I also tried creating a drush/local/drush.yml file that looks like:
# Preflight configuration.
drush:
paths:
alias-path:
- drush/local
and then I added this to the main drush/drush.yml file:
drush:
paths:
# Specify config files to load.
config:
# Load any personal config files. Is silently skipped if not found. Filename must be drush.yml
- ${env.HOME}/.drush/config/drush.yml
- drush/local/drush.yml
Executing drush site:alias --debug results in:
[preflight] Config paths: /var/www/html/vendor/drush/drush/drush.yml,/var/www/html/drush/drush.yml,/var/www/html/drush/local/drush.yml
[preflight] Alias paths: /var/www/html/web/drush/sites,/var/www/html/drush/sites
[preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/drush
[debug] Starting bootstrap to none [0.07 sec, 3.11 MB]
[debug] Drush bootstrap phase 0 [0.07 sec, 3.11 MB]
[debug] Try to validate bootstrap phase 0 [0.07 sec, 3.12 MB]
[success] No site aliases found. [0.07 sec, 3.12 MB]
Executing drush site:alias --debug --config=drush/local/drush.yml results in:
[preflight] Config paths: /var/www/html/drush/local/drush.yml,/var/www/html/vendor/drush/drush/drush.yml,/var/www/html/drush/drush.yml
[preflight] Alias paths: drush/local,/var/www/html/web/drush/sites,/var/www/html/drush/sites
[preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/drush
[debug] Starting bootstrap to none [0.07 sec, 3.12 MB]
[debug] Drush bootstrap phase 0 [0.07 sec, 3.12 MB]
[debug] Try to validate bootstrap phase 0 [0.07 sec, 3.12 MB]
'@local.self.default':
uri: 'https://site-default.ddev.site'
'@local.self.site_a':
uri: 'https://site-a.ddev.site'
'@local.self.site_b':
uri: 'https://site-b.ddev.site'
Interestingly, notice how drush/local/drush.yml changes positions in the list of config paths. When it is in the last position, the additional alias-path value do not seem to be considered in preflight when searching for aliases. When drush/local/drush.yml is in the first position (presumably because of the --config argument), then the additional alias-path of drush/local is considered during preflight.
I was tracing through the code (always a dangerous thing) and I found the line in Preflight::preflight where the alias search paths first seem to be used.
I was exploring the very crude idea of changing
$paths = $this->configLocator->getSiteAliasPaths($this->preflightArgs->aliasPaths(), $this->environment);
to
$paths = $this->configLocator->getSiteAliasPaths(array_merge(
$config->get('drush.paths.alias-path'),
$this->preflightArgs->aliasPaths()
), $this->environment);
but then I found that $config->get('drush.paths.alias-path') has different values depending on if I'm running with xdebug enabled (all the paths I'm looking for) or if it's not enabled (empty). So clearly I don't know what I'm doing.
I have finally got myself deeper into this and I think I have both a better understanding of everything that is happening in PreFlight and a potential fix to my issue that I believe would be a good enhancement to drush.
@greg-1-anderson: I'm wondering if I can pull you back into this conversation to see if it's appropriate for this to be submitted as a pull request in the near future.
To recap, I am trying to specify one or more alternate search directories for self.site.yml alias files. Aliases are going to be used for a multisite installation and the utlimate goal is to be able to execute commands like drush @site_a uli and the resulting reset link will have the correct URI for site_a depending on which environment we are executing the command in. Each site of the multisite has it's own URIs and the URIs have to vary between environments (local, dev, test, impl, prod).
I am trying to specify those locations using the site-wide drush/drush.yml file. Files will be organized like:
- drush/local/self.site.yml
- drush/dev/self.site.yml
- drush/test/self.site.yml
The drush/drush.yml file will have config yaml like:
drush:
paths:
alias-path:
- '/var/www/html/drush/${env.ENVIRONMENT_NAME}'
Depending on the ${env.ENVIRONMENT_NAME} value, a different directory will be searched when looking for self.site.yml files. These files are preconfigured with the right URIs for each of the multisites and those URIs are specific to the environment.
Aliases that come out of these self.site.yml files are labeled @local.self.default, @local.self.site_a, @local.self.site_b, @test.self.site_a, etc. This allows drush @site_b status to work and it will use a different multisite URI depending on which site and environment it is being executed in.
From my analysis, it does not appear that the drush.paths.alias-path config values will be utilized unless the config file that contains the values is specified on the drush CLI like --config=drush/drush.yml. This appears to be because of
https://github.com/drush-ops/drush/blob/287400c51a2e3a83cb95440826ce12d2d95956f6/src/Preflight/Preflight.php#L231
From what I can understand, the 'process' config context is updated with empty/default alias-path values because none were provided on the CLI. If you happen to include the config file on the CLI, --config=drush/drush.yml, the recursive nature of the config apply methods will pull the drush.paths.alias-path values out of the specified config file and merge them into the 'process' config context.
Then, in https://github.com/drush-ops/drush/blob/287400c51a2e3a83cb95440826ce12d2d95956f6/src/Preflight/Preflight.php#L240-L246 I think only the 'process' config context is considered, and the 'drupal' confit context data is ignored. I don't pretend to understand why this is. It could simply be a shortcoming of the code or it could be deliberate.
I think this could be a simple fix for the above lines and would provide a general enhancement to drush:
// Look up the locations where alias files may be found.
$paths = $this->configLocator->getSiteAliasPaths(
array_merge(
$this->config()->getContext('drupal')->get('drush.paths.alias-path', []),
$this->preflightArgs->aliasPaths()
), $this->environment);
// Configure alias manager.
$aliasFileLoader = new \Drush\SiteAlias\SiteAliasFileLoader();
$this->aliasManager = (new SiteAliasManager($aliasFileLoader))->addSearchLocations($paths);
$this->aliasManager->setReferenceData($config->export());
So instead of only searching standard paths for self.site.yml, plus those specified on the CLI, and those specified in a drush.yml file specified on the CLI, this would also consider paths specified in drush.paths.alias-path values from the 'drupal' site-wide config context. This config context is already loaded in PreFlight; it is just not passed down into SiteAliasManager::addSearchLocations. Other contexts might be useful here as well.
With this code in place, the $paths array looks like:
array (
0 => '/var/www/html/drush/test',
1 => '/var/www/html/web/drush/sites',
2 => '/var/www/html/drush/sites',
)
Element 1 and 2 are standard drush paths. Element 0 is the one I specified in the drush/drush.yml file.
I don't think this in anyway complicates PreFlight and adds some additional flexibility that would allow for dynamic insertion of alias-path values. I suspect this was the original intention of including drush.paths.alias-path in the drush.yml examples, but I am of course not aware of any history around this.
I suppose that it could be argued that individual sites of a multisite aren't really the use case for drush site aliases, but we have been successfully using them for a while in our local development environments. Now that our project is moving to additional test and staging environments, we want to continue to use drush site aliases there. Even drush site:set works well in this situation.
Am I correct in reading this issue like so (and from our experimentation today)? From https://www.drush.org/12.x/using-drush-configuration/#directories-and-discovery Any alias-path config will only be read from 3 (--config), 4 (~/.drush/drush.yml) or 5 (/etc/drush/drush.yml)?
For our use case - none of these locations are stored in the git repo, and overwriting/appending existing drush.yml seems not right?
Is there any way a generated alias-path can be added to drush.yml inside the projectroot and have it read by drush?
(our use case is an alias generator that writes an alias file in a running container - so therefore only able to write to persistent storage or /tmp as a fallback - and we'd preferably like for a project to be able to define the path to the alias file in a drush.yml inside their projectroot)
For our use case - none of these locations are stored in the git repo, and overwriting/appending existing drush.yml seems not right?
Appending is permissable for service providers (Lagoon, Pantheon, etc) IMO. A bit more info is at https://www.drush.org/12.x/site-aliases/#site-alias-files-for-service-providers
At a minimum you should get self.alias.yml, which can be committed to the site repository. I thought that other alias files could as well, but no time to investigate right now.
we've already got the wildcard aliases - which work ok, but we're implementing a per-cluster ssh endpoint, which means that the wildcard might not always work, so for certain cases (projects that have different environments in different clusters, or even regions), we want to be able to inject a generated alias file over the top of it. Committing the alias isn't feasible, as then it would need updating for every new environment/region added, for all environments. Hence the requirement of being able to commit where the alias file is going to be generated, so Drush can read it.