Add ability to call custom GET/POST command via synadm script

[MurzNN]

Will be good to use same synadm tool not only for pre-defined API calls, but for custom too, for not to create custom scripts for each call. For example, allow something like this:

synadm custom-api-call --type POST --path "/_synapse/admin/v1/media/ru-matrix.org/delete?before_ts=1602745639000" --body "{}"

where make default argument value for type as GET, body = {}.

This way will be much more comfortable, than composing full curl command in cli manually and remember your token each time.

Especially this is useful at now, when some of popular API calls are not implemented in build-in commands.

[MurzNN]

Also will be good to show response text in prettified JSON, instead of plaintext like in raw curl command.

[JOJ0]

Did you check the latest upstream version (master branch)? Have you seen the --output/-o option? Actually pretty printed json is supported since day one :-) It was called --json and now is called --output pprint. Just issue synadm config and you can configure your prefererred default output format.

[JOJ0]

Will be good to use same synadm tool not only for pre-defined API calls, but for custom too, for not to create custom scripts for each call. For example, allow something like this:

synadm custom-api-call --type POST --path "/_synapse/admin/v1/media/ru-matrix.org/delete?before_ts=1602745639000" --body "{}"

where make default argument value for type as GET, body = {}.

This way will be much more comfortable, than composing full curl command in cli manually and remember your token each time.

Especially this is useful at now, when some of popular API calls are not implemented in build-in commands.

As for this suggestion: Yes nice idea and I will think about it but actually what do you think about my preferred idea: You seem to have a specific idea of what admin API calls you are missing. It is pretty easy to implement a missing one. So please file a pull-request with the one you are missing most and I will happily try to assist with reviewing your code :-)

The goal of synadm is to at some point support everything the Synapse admin APIs offer. So implementing one available API after the other is the first priority. Any help with that is greatly appreciated!

The second priority is supporting regular matrix calls too. One of the next features will be to let the admin user login by password prompt or token environment variable, so if a user prefers it that way, the token has not to be saved in the configuration. User login is a regular matrix client API and to support it there needs to be some foundational work done first.

At the moment top priority is getting the latest bugs fixed that were introduced by latest PR. So please go ahead and try out the latest version and open issues when you find anything that fails or is not working as before! That would help a lot! Thanks and keep those suggestions and issue reports coming! Appreciated :-)

[amstan]

+1 on this. I was chatting about this earlier on #synapse:matrix.org as well.

I propose raw for the subcommand name, it's shorter. Instead of custom-api-call

It would be a great stopgap when the official cli ui is not implemented for some new feature.

Bonus points if you can make the matrix api documentation writers to adopt the tool instead of curl (though now i can't find examples of curl usage in any such documentation).

[JOJ0]

I will think about it but it seems to be more work and thinking compared to supporting a new regular admin API, which as already mentioned, currently should be my main priority. If any of you would send a PR draft we could work on it together. That would be awesome! :-)

[JOJ0]

+1 on this. I was chatting about this earlier on #synapse:matrix.org as well.

I propose raw for the subcommand name, it's shorter. Instead of custom-api-call

It would be a great stopgap when the official cli ui is not implemented for some new feature.

Bonus points if you can make the matrix api documentation writers to adopt the tool instead of curl (though now i can't find examples of curl usage in any such documentation).

@amstan I like synadm raw as the option name but somehow we'd need to distinguish between "admin API" command and "matrix client API" command.

The first part of the URI is in the configuration already, currently only for the admin api but as mentioned synadm will make use of the regular matrix client api in the future, so it will be a configuration item and should be used. See here, soon there will be an option called "Matrix Client path" or similar:

$ synadm config
Running configurator...
Synapse admin user name [jo]:
Synapse admin user token [asdf]:
Synapse base URL [http://localhost:8008]:
Synapse admin API path [/_synapse/admin]:
Default output format (yaml, json, human, pprint) [yaml]:
Default http timeout [7]:
Restricting access to config file to user only

You have a good suggestion how the command could be constructed? My only idea would be something like:

synadm raw 'admin_api_url'

synadm raw -m 'matrix_client_api_url'

or better make dedicated commands? (but that's kind of even more ugly ;-))

synadm raw-admin 'matrix_client_api_url'

synadm raw-matrix 'matrix_client_api_url'

As mentioned already: I'd welcome a PR implementing this but won't find the time any time soon to code it myself. Thanks in advance if you'd really consider to submit it.

[JOJ0]

@amstan still waiting for your pr draft 😅

[JOJ0]

Part of this has been implemented in branch do_as_user - regular Matrix API calls can be executed using synadm

Note that this is a convenience feature for admins trying to help users and not trying to replace existing Matrix API CLI tools! Since I recently also implemented a command for "user token fetching API": synadm user login (also in do_as_user branch), this felt useful to implement.

Data can be read in from file or stdin:

synadm matrix raw 'client/r0/user_directory/search' -m post -f search.json
cat search.json | synadm matrix raw 'client/r0/user_directory/search' -m post -f -

or from CLI directly:

synadm matrix raw 'client/r0/user_directory/search' -d '{"search_term": "username", "limit":100}' -m post

A Matrix token can be read in interactivly:

synadm matrix raw 'client/r0/user_directory/search' -m post -f search.json -p

Set on CLI:

synadm matrix raw 'client/r0/user_directory/search' -m post -f search.json -t xxxunsecurexxx

Or even read from an environment variable:

read MTOKEN
synadm matrix raw 'client/r0/user_directory/search' -m post -f search.json

The current help of the new command should describe all the possibilities:

jojo@jum ~/git/synadm (do_as_user) $ synadm matrix raw -h
Usage: synadm matrix raw [OPTIONS] ENDPOINT

  Execute a raw request to the Matrix API.

  The endpoint argument is the part of the URL _after_ the configured base
  URL and Matrix path (see `synadm config`). A simple get request would e.g
  like this: `synadm matrix raw client/versions`

  Use either --token or --prompt to provide a user's token and execute
  Matrix commands on their behalf. Respect the privacy of others! Be
  responsible!

  The precedence rules for token reading are: 1. Interactive input using
  --prompt; 2. Set on CLI via --token string; 3. Read from environment
  variable $MTOKEN; 4. Preconfigured admin token set in synadm's config
  file.

Options:
  -m, --method [get|post|put|delete]
                                  The HTTP method used for the request.
                                  [default: get]

  Data input: [mutually_exclusive]
    -d, --data TEXT               The JSON string sent in the body of post,
                                  put and delete requests - provided as a
                                  string. Make sure to escape it from shell
                                  interpretation by using single quotes. E.g
                                  '{"key1": "value1", "key2": 123}'  [default:
                                  {}]

    -f, --data-file FILENAME      Read JSON data from file. To read from stdin
                                  use "-" as the filename argument.

  Matrix token: [mutually_exclusive]
    -t, --token TEXT              Token used for Matrix authentication instead
                                  of the configured admin user's token. If
                                  --token (and --prompt) option is missing,
                                  the token is read from environment variable
                                  $MTOKEN instead. To make sure a user's token
                                  does not show up in system logs, don't
                                  provided it on the shell directly but set
                                  $MTOKEN by using shell command `read
                                  MTOKEN`.

    -p, --prompt                  Prompt for the token used for Matrix
                                  authentication. This option always overrides
                                  $MTOKEN.  [default: False]

  -h, --help                      Show this message and exit.
jojo@jum ~/git/synadm (do_as_user) $

@amstan and @MurzNN: Please help me get this feature stable and do some testing and playing around with it. Also review of the help text's english and clarity of descriptions is appreciated. That would help a lot. Thanks in advance

[JOJ0]

Above mentioned has been released in this version: https://github.com/JOJ0/synadm/releases/tag/v0.30

[Ascurius]

I am closing the issue because the requested feature already has been implemented

[JOJ0]

Sorry, please reopen. It only is implemented for matrix calls but not for admin api calls 😀

some details pn the initial ideas here: https://github.com/JOJ0/synadm/issues/11#issuecomment-774434384