symfony-console-completion

symfony-console-completion copied to clipboard

Hi there,

we from Nextcloud started using the completion and it works quite fine in general.

The only problem is, that it is impossible to run the stuff with e.g. sudo -u www-data. This is the recommanded easy-default for Nextcloud, to simply have all code and files owned by the www-data user. But I am unable to --generate-hook in this case.

I'm not sure if this is fixable, but would be very cool if so. I tried adding a copy of a generated hook script manually in /etc/bash_completion.d/ but when trying to modify the script to add sudo -u www-data on the calls, it seems to not execute properly anymore.

Any idea/workaround?

Cheers and keep it up

Can you please write exact commands you're using and username you login with, that has hook installed for?

For example:

1. I login using alex username
2. user alex has ~/.bashrc file with these line source <($HOME/web/code-insight/bin/code-insight _completion --generate-hook -p code-insight)
3. after that presuming, that code-insight binary is accessible in PATH I type code-insight TAB TAB and get auto-completion

I've just tested that bash-completion for sudo [args...] mysymfonycommand correctly hands off completion to the handler for mysymfonycommand (at least on Debian stretch). @nickvergessen - are you running these sudo commands from a non-privileged user, or under the root account? Kind of an odd question, but I've experienced problems with bash-completion stopping working under sudo su etc. Breakdown of steps per @aik099's comment would be great

So in my case it is as follows:

local user: nickvergessen script: /var/www/nextcloud/occ

occ (calling console.php) has the following section:

    $user = posix_getpwuid(posix_getuid());
    $configUser = posix_getpwuid(fileowner(OC::$configDir . 'config.php'));
    if ($user['name'] !== $configUser['name']) {
        echo "Console has to be executed with the user that owns the file config/config.php" . PHP_EOL;
        exit(1);
    }

owner of config.php has to be the apache user www-data. So instead of /var/www/nextcloud/occ I need to run: sudo -u www-data /var/www/nextcloud/occ

I guess the only way is to "ignore" the user check for the _complete command? Because it is not possible to add the sudo to the call of your generated script:

RESULT="$(/var/www/nextcloud/occ _completion </dev/null)";

Since ignoring the user check does not work, because we can not get a connection to the database because we can not read the config,... I tried to continue with the other route.

With:

RESULT="$(sudo -u www-data /var/www/nextcloud/occ _completion </dev/null)";

I receive:

  [RuntimeException]                                                               
  Failed to configure from environment; Environment var CMDLINE_CONTENTS not set.  

_completion [-g|--generate-hook] [-p|--program PROGRAM] [-m|--multiple] [--shell-type [SHELL-TYPE]]

I guess this is, because www-data has nologin

I haven't tested, but does sudo ls -la:

1. show only folders that root access
2. just folders that current user invoking sudo command can access

?

If this is the 2nd case, then I don't see possible workaround for sudo. Also the problem with sudo is that it will ask current user's password. We can just ignore that fact and try to run sudoed command somehow.

$ sudo ls -la config/
drwxrwxr-x  2 www-data www-data  4096 Nov 21 16:10 .
drwxr-xr-x 23 nickv    nickv     4096 Nov 21 16:10 ..
-rw-rw----  1 www-data www-data  2176 Nov 21 16:10 config.php

$ tail config/config.php 
tail: cannot open 'config/config.php' for reading: Permission denied

So it lists the config file which I cannot read/open.

It will work if you have entered password previously or you're allowed to run sudo without entering a password at all. Maybe then it's possible to tweak auto-complete as well.

Yeah, so any idea how to fix this? entering sudo password before doesnt seem to help

Not really.

You can create basic bash autocomplete hook using compgen and complete commands and see if it works through sudo.

• If it does, then it means all ENV vars are properly proxied to completion script and it's stecman/symfony-console-completion that doesn't handle it.
• If it doesn't, then bash doesn't support auto-complete over sudo and stecman/symfony-console-completion can't help, because it's working based on ENV vars provided by bash.

I have:

_nhs() 
{
	local cur
	# Pointer to current completion word.
	# By convention, it's named "cur" but this isn't strictly necessary.

	COMPREPLY=()   # Array variable storing the possible completions.
	cur=${COMP_WORDS[COMP_CWORD]}

	case "$COMP_CWORD" in
		1)
		SERVERS='9 10 11 12'
		COMPREPLY=( $( compgen -W "$SERVERS" -- $cur ) );;
	esac

	return 0
}
complete -F _nhs nhs

and can do:

sudo nhs <tab>
10 11 12 9

The problem is the RESULT call in your script that needs sudo as well

RESULT="$(/home/nickv/Nextcloud/12/server/occ _completion </dev/null)";

So I replaced https://github.com/nextcloud/3rdparty/blob/b6ef8a42ae8c7d4d69c6acdf919072a7989648d7/stecman/symfony-console-completion/src/HookFactory.php#L44-L44 with

RESULT="$(sudo -u www-data %%completion_command%% </dev/null)";

but then sudo -u www-data ./occ <tab> brings:

  [RuntimeException]                                                               
  Failed to configure from environment; Environment var CMDLINE_CONTENTS not set.  

_completion [-g|--generate-hook] [-p|--program PROGRAM] [-m|--multiple] [--shell-type [SHELL-TYPE]]

Not fixabled?

I know that /usr/bin/env can pass through env vars. I'm not sure if it can pass env vars from non-sudo to sudo calls.

Does RESULT="$(sudo -u www-data /usr/bin/env %%completion_command%% </dev/null)"; work?

@nickvergessen can you provide any more detail about your environment? I've just tested this on my machine and completion works fine calling with sudo occ [tab] and sudo -u www-data occ [tab] from what I can see:

$ uname -a
Linux 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ echo $BASH_VERSION
4.3.46(1)-release

$ php -v
PHP 5.6.29-1+deb.sury.org~xenial+1 (cli) 
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies

Does RESULT="$(sudo -u www-data /usr/bin/env %%completion_command%% </dev/null)"; work?

Same issue:

$ sudo -u www-data ./occ _completion -g --shell-type bash

function _occ_feacda5c68cc086b_complete {


    local CMDLINE_CONTENTS="$COMP_LINE"
    local CMDLINE_CURSOR_INDEX="$COMP_POINT"
    local CMDLINE_WORDBREAKS="$COMP_WORDBREAKS";

    export CMDLINE_CONTENTS CMDLINE_CURSOR_INDEX CMDLINE_WORDBREAKS

    local RESULT STATUS;

    RESULT="$(sudo -u www-data /usr/bin/env ./occ _completion </dev/null)";

    STATUS=$?;

    local cur mail_check_backup;

    mail_check_backup=$MAILCHECK;
    MAILCHECK=-1;

    _get_comp_words_by_ref -n : cur;


    if [ $STATUS -eq 200 ]; then
        _filedir;
        return 0;

    elif [ $STATUS -ne 0 ]; then
        echo -e "$RESULT";
        return $?;
    fi;

    COMPREPLY=(`compgen -W "$RESULT" -- $cur`);

    __ltrim_colon_completions "$cur";

    MAILCHECK=mail_check_backup;
};

if [ "$(type -t _get_comp_words_by_ref)" == "function" ]; then
    complete -F _occ_feacda5c68cc086b_complete "./occ";
else
    >&2 echo "Completion was not registered for ./occ:";
    >&2 echo "The 'bash-completion' package is required but doesn't appear to be installed.";
fi
  
$ eval $(sudo -u www-data ./occ _completion -g --shell-type bash)

$ sudo -u www-data ./occ 

  [RuntimeException]                                                               
  Failed to configure from environment; Environment var CMDLINE_CONTENTS not set.  

_completion [-g|--generate-hook] [-p|--program PROGRAM] [-m|--multiple] [--shell-type [SHELL-TYPE]]

System info:

$ uname -a
Linux nickv-infinity 4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ echo $BASH_VERSION
4.3.42(1)-release
$ php -v
PHP 7.0.14-2+deb.sury.org~xenial+1 (cli) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.14-2+deb.sury.org~xenial+1, Copyright (c) 1999-2016, by Zend Technologies
    with Xdebug v2.5.0, Copyright (c) 2002-2016, by Derick Rethans

I guess the most important thing is the changed owner ship. I made config.php owned by www-data:www-data, but I guess you have that as well. Not sure what could be of interest other then that.

$ ls -la config/
insgesamt 136
drwxrwxr-x  2 nickv    www-data  4096 Dez 19 15:47 .
drwxr-xr-x 22 nickv    nickv     4096 Dez 19 15:47 ..
-rw-rw----  1 www-data www-data  2059 Dez 19 15:47 config.php
-rw-rw-r--  1 nickv    www-data 41687 Dez 15 16:36 config.sample.php
-rw-rw-r--  1 nickv    www-data   225 Dez 13 17:59 .htaccess

Seeing you ran your eval $(... without sudo -u www-data, your setup must be different because for me that doesn't even work:

$ /home/nickv/Nextcloud/12/server/occ
Cannot write into "config" directory!
This can usually be fixed by giving the webserver write access to the config directory

See https://docs.nextcloud.com/server/11/go.php?to=admin-dir_permissions

Okay, I got it working using the following instead of https://github.com/nextcloud/3rdparty/blob/b6ef8a42ae8c7d4d69c6acdf919072a7989648d7/stecman/symfony-console-completion/src/HookFactory.php#L44-L44:

RESULT="$(sudo -u www-data /usr/bin/env CMDLINE_CONTENTS='$COMP_LINE' CMDLINE_CURSOR_INDEX='$COMP_POINT' CMDLINE_WORDBREAKS='$COMP_WORDBREAKS' %%completion_command%% </dev/null)";

Would be nice if this could make it into your repo in a clean way 😃

I guess we can specify --sudo-username option (new) during hook generation and if set it will:

1. use sudo command for that user
2. transfer all env (better not to hardcode env variables) to sudo

I just noticed, that now only commands can be autocompleted:

$ sudo -u www-data /home/nickv/Nextcloud/12/server/occ _completion 
app:check-code                      encryption:disable                  maintenance:mode
app:disable                         encryption:enable                   maintenance:repair
app:enable                          encryption:encrypt-all              maintenance:singleuser
....

But I guess that is because I only forwarded some of the enviroment variables?

You can do sudo -E (according to http://unix.stackexchange.com/questions/202292/pass-environment-variables-values-to-a-program) to pass all env to sudoed script (I wish I could have found this sooner). This way following code should work as well:

RESULT="$(sudo -E -u www-data %%completion_command%% </dev/null)";

👍 that one works

I guess you can send PR, that will add --sudo-user option with required value and then when specified during hook generation the sudo version of RESULT variable would be used for both BASH and ZSH.

https://github.com/stecman/symfony-console-completion/issues/73

$ /home/nickv/Nextcloud/12/server/occ
Cannot write into "config" directory!
This can usually be fixed by giving the webserver write access to the config directory

See https://docs.nextcloud.com/server/11/go.php?to=admin-dir_permissions

@nickvergessen to be honest this seems like a permissions / design issue with Nextcloud, not the completion system. A few suggestions for working around this:

• Configure file/dir permissions so that www-data and the owning user can both write to those directories:

  $ chown -R application-code-owner:www-data config/ ...
  $ chmod 775 config/
  $ chmod 660 config/*
  
  # or with sticky groups on directories if your program creates nested
  # directories that need the same permissions
  chmod 2775 config/
  

  If you have all of the code owned by an application user like nextcloud, this also potentially solves this from the end of that help page:

  These strong permissions prevent upgrading your Nextcloud server; see Setting Permissions for Updating for a script to quickly change permissions to allow upgrading.

  Since it means you can do something like:

  sudo su application-user
  cd /var/www/nextcloud
  # do upgrading things with all the write permissions needed
  

• Alternatively you could remove the check for write permissions during completion, unless write access is actually needed by the completion in your application.

Well most enviroments don't have an application-user, also all docs and tutorials out there mention www-data to be used, but for that one sudo su www-data is not allowed, because it is a no-login account.

The patch is already done (see #73 ) and we will be easily able to use this, if you don't want to merge it, we will just add the _completion -g command a second time with the fixed version, works for me, but I think it would be cooler if it is all in one place.

Sorry, I was meaning [application-user] as a placeholder for a user that's been added to run Nextcloud (or just the user's account that installed it) - eg.

$ sudo useradd --system nextcloud
$ sudo su nextcloud
# or
$ sudo -u nextcloud

I appreciate that you've made a PR for this, but I think it adds unnecessary complexity for something that's very specific to the way Nextcloud currently works, and can be avoided with a small change to filesystem permissions. If you couldn't fix this any other clean way I wouldn't have a problem with it, but it looks like there are quite a few good solutions that don't require changes to this library.

In addition to the two suggestions above, another relatively clean solution might be extending CompletionCommand and HookFactory in your code to get the functionality you're after. I'm happy to accept a PR that makes it easier to do this, like getting HookFactory from a method in CompletionCommand instead of instantiating it inline.

We think that adding a user is too complicated, because sometimes permissions might change and all the docs would need adjustment and so on.

So if you don't want to merge this because it adds too much parameters ( I get and see that as well), I will try to extend your classes and see how that works. As far as I can see you use protected everywhere (no private) so that should work.

PS: okay the HookFactory can not be extended due to final If you could get rid of that in a future version it would allow things to be easier, but all the template needs to be overwritten anyway, so just the helper functions would be "saved".

Hmm, so it is.. I think I did that with the intention that HookFactory would essentially be a template storage thing and no more. You're welcome to rejigger HookFactory to make it easier to inject or override specific parts of the replacements, and remove the final keyword, though that's straying into depending on implementation details which isn't ideal for you either :/

Is my second suggestion feasible? Seems like completion shouldn't need write access to run:

...remove the check for write permissions during completion, unless write access is actually needed by the completion in your application.

@nickvergessen: were you able to find a reliable procedure to enable autocompletion yet? i am struggeling around ...

Not really