Table of Contents

• GitLab bash API
  • Installation
  • Configuration
  • Usage
    • Generic GET
    • Generic PUT
    • About users
    • About groups
    • About projects (repositories)
    • About branches
  • Audit and backups
    • Backups repositories
    • Audit groups and repositories
  • Samples
  • About GitLab and gitlab-bash-api
  • Related documentations

Created by gh-md-toc

GitLab bash API

Access GitLab CE API or GitLab EE API from bash.

Last version is available on GitHub: https://github.com/cClaude/gitlab-bash-api

Current version is based on GitLab V4 API but some features work on V3. V3 is no more supported except by glGet and glPut commands.

Installation

This tool require bash, curl, jq and git.

sudo apt update
sudo apt upgrade
sudo apt install jq git

git clone https://github.com/cClaude/gitlab-bash-api.git

Configuration

You can create a my-config folder (ignored by git) to configure/customize this application or just copy content of custom-config-sample/. The my-config folder is taken in account by default by the API

You can also use any custom folder for configuration, by setting GITLAB_BASH_API_CONFIG variable with the full path of your custom folder.

In you configuration files:

• You can create any custom file to declare variables (bash format), all theses files will be sourced.
• You can override default values define in config/ folder,
• You need at least define values for GITLAB_PRIVATE_TOKEN and GITLAB_URL_PREFIX.

GITLAB_PRIVATE_TOKEN=__YOUR_GITLAB_TOKEN_HERE__
GITLAB_URL_PREFIX=__YOUR_GITLAB_USER_HERE__

Configuration algorithms :

1. source files in ${GITLAB_BASH_API_PATH}/config
2. source files in ${GITLAB_BASH_API_PATH}/my-config (if folder exists)
3. source files in ${GITLAB_BASH_API_CONFIG} (if variable is define and if folder exists)

Facultative configuration:

You can also configure in you ~/.bashrc file

export GITLAB_BASH_API_PATH='__YOUR_PATH_TO__/gitlab-bash-api'
export GITLAB_BASH_API_CONFIG="__YOUR_PATH_TO__/your-custom-config-folder"

PATH=$PATH:${GITLAB_BASH_API_PATH}/

Hacking

If for any reason you need to customize how curl access to GitLab server you can add some custom configuration in ${GITLAB_BASH_API_PATH}/my-config or in ${GITLAB_BASH_API_CONFIG} folders.

A sample is available in custom-config-sample/customize-curl.sh.

Usage

You can call comment using the full path

${GITLAB_BASH_API_PATH}/listUsers.sh --all

or simply (if ${GITLAB_BASH_API_PATH} is in your path):

listUsers.sh --all

Generic GET

Syntax:

glGet.sh --uri GL_URI [--params 'PARAM1=VALUE1&PARAM2=VALUE2]

glGet.sh --uri /projects | jq .

Generic PUT

Syntax:

glPut.sh --uri GL_URI [--params 'PARAM1=VALUE1&PARAM2=VALUE2]

TODO NEED SAMPLE

About users

• How to create a new user ?

Syntax:

glCreateUser.sh USER_NAME 'USER_FULLNAME' 'USER_EMAIL'

glCreateUser.sh testuser "test user" [email protected]

• How to display all users ?

listUsers.sh --all

• How to display a specific user ?

listUsers.sh testuser

About groups

How to manage groups using glGroups command ?

• Usage: Get groups configuration

    glGroups.sh --config --path GROUP_PATH
    glGroups.sh --config --id GROUP_ID
    glGroups.sh --config --all

• Usage: List groups paths

    glGroups.sh --list-path --path GROUP_PATH
    glGroups.sh --list-path --id GROUP_ID
    glGroups.sh --list-path --all

• Usage: List groups ids

    glGroups.sh --list-id --path GROUP_PATH
    glGroups.sh --list-id --id GROUP_ID
    glGroups.sh --list-id --all

• Usage: Create group

    glGroups.sh --create --path GROUP_PATH
        [--name GROUP_NAME] \
        [--description GROUP_DESCRIPTION] \
        [--lfs_enabled true|false] \
        [--membership_lock true|false]
        [--request_access_enabled true|false] \
        [--share_with_group_lock true|false]]
        [--visibility  private|internal|public] \

• Usage: Edit group configuration

    glGroups.sh --edit --id GROUP_ID --name GROUP_NAME --path GROUP_PATH \
        [--description GROUP_DESCRIPTION] \
        [--visibility  private|internal|public] \
        [--lfs_enabled true|false] \
        [--request_access_enabled true|false]

• Usage: Delete a group

    glGroups.sh --delete --id GROUP_ID

• Sample: Retrieve main configuration on all groups:

glGroups.sh --config --all

• Sample: create a group

glGroups.sh --create --path my_test_group

About projects (repositories)

How to manage groups using glProjects command ?

• Usage: Get projects configuration

    glProjects.sh --config [--compact] --id PROJECT_ID
    glProjects.sh --config [--compact] --group-path GROUP_PATH
    glProjects.sh --config [--compact] --all
    glProjects.sh --config [--compact] --path PROJECT_PATH

• Usage: List projects paths

    glProjects.sh --list-path --id PROJECT_ID
    glProjects.sh --list-path --group-path GROUP_PATH (could return more than one entry)
    glProjects.sh --list-path --all
    glProjects.sh --list-path --path PROJECT_PATH (could return more than one entry)

• Usage: List projects ids

    glProjects.sh --list-id --id PROJECT_ID
    glProjects.sh --list-id --group-path GROUP_PATH (could return more than one entry)
    glProjects.sh --list-id --all
    glProjects.sh --list-id --path PROJECT_PATH

• Usage: Create project

    glProjects.sh --create --group-id GROUP_ID --path PROJECT_PATH \
      [--project-name PROJECT_NAME] \
      [--default-branch DEFAULT_BRANCH] \
      [--project-description PROJECT_DESCRIPTION] \
      [--container-registry-enabled true|false] \
      [--issues-enabled true|false] \
      [--jobs-enabled true|false] \
      [--lfs-enabled true|false] \
      [--merge-requests-enabled true|false] \
      [--only-allow-merge-if-all-discussions-are-resolved true|false] \
      [--only-allow-merge-if-pipeline-succeed true|false] \
      [--printing-merge-request-link-enabled true|false] \
      [--public-jobs true|false] \
      [--request-access-enabled true|false] \
      [--snippets-enabled true|false] \
      [--visibility private|internal|public] \
      [--wiki-enabled true|false]

• Usage: Edit project

    glProjects.sh --edit --id PROJECT_ID --project-name PROJECT_NAME \
      [--path PROJECT_PATH] \
      [--default-branch DEFAULT_BRANCH] \
      [--project-description PROJECT_DESCRIPTION] \
      [--issues-enabled true|false] \
      [--merge-requests-enabled true|false] \
      [--jobs-enabled true|false] \
      [--wiki-enabled true|false] \
      [--snippets-enabled true|false] \
      [--container-registry-enabled true|false] \
      [--visibility private|internal|public] \
      [--public-jobs true|false] \
      [--only-allow-merge-if-pipeline-succeed true|false] \
      [--only-allow-merge-if-all-discussions-are-resolved true|false] \
      [--lfs-enabled true|false] \
      [--request-access-enabled true|false]

• Usage: Delete a project

    glProjects.sh --delete --group-path GROUP_PATH --path PROJECT_PATH
    glProjects.sh --delete --id PROJECT_ID

• Sample: Retrieve main configuration on all projects:

glProjects.sh --config --all

• Sample: Retrieve only path with name space:

glProjects.sh --config --all | jq -r ' .[] | .path_with_namespace'

• Sample: List of all projects id of a group

glProjects.sh --list-id --group-path GROUP_PATH

• Sample: To delete a project

glProjects.sh --delete --id PROJECT_ID

About branches

• List remote branch

Syntax:

listBranches.sh PROJECT_ID

• To have all information about existing branches:

listBranches.sh 82

• To have just branches name list of project with id=10:

listBranches.sh 10 | jq -r ' .[] | .name'

(glBranches.sh command is still in alpha version)

Audit and backups

Backups repositories

glCloneAllProjects allow you to backup all repositories (GitLab projects only) using gitlab-bash-api.

It is not a backup for everything, backup of users, groups, merge-requests, snippets, jobs, ... are not covered by glCloneAllProjects. But it keep full history of your projects, this a good practice to keep a such copy before a GitLab migration.

• To clone all projects you have access

Syntax:

glCloneAllProjects.sh --http|--ssh [--bare] --destination OUTPUT_FOLDER

• Complete example cloning throw ssh

mkdir -p tests-result

glCloneAllProjects.sh --ssh --bare --destination "tests-result/$(date +'%Y-%m-%d.%H-%M').clones"

If you need a custom key to handle this, create the key using

ssh-keygen -t rsa -C "clone-process" -b 4096 -f ~/.ssh/gitlab_root_id_rsa

Add this key on GitLab root account. root should be at least developper of all repositories but for other action you probably need that this account is owner of all repositories.

Then you can run glCloneAllProjects using

GIT_SSH_COMMAND="ssh -i ${HOME}/.ssh/gitlab_root_id_rsa" ./glCloneAllProjects.sh --ssh --bare --destination tests-result/$(date +'%Y-%m-%d.%H-%M').clones

Audit groups and repositories

./glAudit.sh --directory tests-result/$(date +'%Y-%m-%d.%H-%M').audit

This will generate a folder YYYY-MM-DD.HH-MM.audit with these sub-folders

• groups_by_id : for all groups configuration (file 1.json contain configuration of group id=1)
• groups_by_path : contain links (links name are based on group path)
• projects_by_id :for all repositories configuration (file 1.json contain configuration of project id=1)
• projects_by_path : contain links (links name are based on project path name)
• projects_by_path_with_namespace : contain folder (based on group path) then link based on project path.

Samples

Retrieve id of all projects into a group.

./glProjects.sh --config --group-path puppet | jq '[.[] | {
id: .id,
path_with_namespace: .path_with_namespace
}]'

Retrieve id of all projects into a group but format output

./glProjects.sh --config --group-path puppet |
  jq -r '.[] | (.id|tostring) + ":" + (.path_with_namespace)'

Retrieve id of all projects do something with this id

./glProjects.sh --config --group-path puppet |
  jq -r '.[] | (.id|tostring) + ":" + (.path_with_namespace)' |
  while read line; do
    echo "Handle ${line}"
    PROJECT_ID=$(echo "${line}" | cut -d ':' -f 1)

    echo "do something with ${PROJECT_ID}"

  done

Full sample

function enable_key_for_group {
  local group_name=$1
  local deploy_key_id=$2

  "${GITLAB_BASH_API_PATH}/glProjects.sh" --config --group-path "${group_name}" \
  | jq -r '.[] | (.id|tostring) + ":" + (.path_with_namespace)' \
  | while read line; do
      echo "Handle ${line}"
      local project_id=$(echo "${line}" | cut -d ':' -f 1)

      "${GITLAB_BASH_API_PATH}/glDeployKeys.sh" --enable --project-id "${project_id}" --key-id "${deploy_key_id}" || exit 1
  done
}

 # let say you have a deploy code id define in
 # You can use 'glDeployKeys.sh' to have this
 DEPLOY_KEY_ID=56
 GROUP_NAME=puppet

 # Then you want to enable this key on all project of a group
 # Basically it will use
 #   glDeployKeys.sh --enable --project-id PROJECT_ID --key-id DEPLOY_KEY_ID

 enable_key_for_group "${GROUP_NAME}" "${DEPLOY_KEY_ID}"

About GitLab and gitlab-bash-api

If you really need this API you probably need to consider moving to another git server.

GitLab is the best SVN server ever... but for git needs consider to move to something else.

• gitea is complete, it is free and a true OpenSource solution.
• bitbucket from Atlassian is proprietary software but probably the most mature solution.

Related documentations

• How to get your GitLab API key