vlang
Gitly CI/CD Considerations
Gitly CI/CD design
Hi, I'm here to talk about CI/CD integration for Gitly.
I use daily CI/CD on both GitLab and GitHub, and I see the power of the both technology.
GitLab has highly customisable and powerful pipelines. It also has parallels runs.
GitHub as simple workflow and highly community oriented workflow.
So I try to keep the powerful GitLab pipelines with the comminuty based workflow of GitHub Actions.
I think It's powerful enough to do anything.
Here my proposal of a YAML for Gitly CI/CD file.
# filename -> .gitly/flow/
# Name of the flow pipeline
name: Build MyApp
# Define the differents steps of the pipeline flow.
## Steps are launched in the order of the list.
## Names can be customized by developer.
steps:
- prepare
- build
- test
- publish
# Define each jobs run by the flow
jobs:
# Set the id of the job to be selected after.
docker-prepare:
# Set the display name of the jobs. (default will be the id)
name: Prepare the Docker environment
# Set the contraints of the job
on:
# Link this job into the `prepare` steps.
## This will display it in the first column of the flow view. (because prepare is the first)
steps:
- prepare
# run this job only on push event.
events:
- push
# Run this jobs only on event triggered on `main` and `staging` branch...
branches:
- main
- staging
# ...or Run it on tags like `v1.0`
tags:
- v(.*)
# Define all actions that job will run.
actions:
# Get the source code of the repository.
- name: Checkout
## This can be the same like this : gitly.org/gitly/flow/checkout@master
## We can use @<branch_name> or @<tag_name>.
## Any @<tag_name> must be cached to not recompile it at any use.
## So using @<tag_name> will speedup your flow.
use: checkout@v1
args:
# (Optional)
ref: ${gitly.event.ref}
# define another action that run a user custom flow by selecting his public repository. (Can be on other git server)
- name: Get environment docker image
use: gitly.org/newuser/myflow-pull.git@master
# define another action that run a user custom flow by selecting his public repository. (Can be on other git server)
- name: Publish Environment Docker image
use: gitly.org/newuser/myflow-push.git@master
args:
repo: ${gitly.server.docker_repository}
tag: ${gitly.repository.url}/android:${gitly.event.branch-or-tag}
- name: Display image size
# This `use` is optional, it allow to choose a prefered runner and/or a docker image.
use: ubuntu
scripts:
# Ouput is defined by the flow repository (Here by gitly.org/newuser/myflow-push.git@master)
- echo "${gitly.jobs.docker-prepare.output.docker_image_size}"
# Run another job (Other job will not contain the same folder, but can get it from )
display-env:
on:
steps:
- prepare
# Without this, jobs are in parallels. You can access to output of jobs only if his id are listed here.
after:
- docker-prepare
actions:
- name: Display android sdkmanager version
# This use docker image
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- sdkmanager --version
display-env:
on:
steps:
- prepare
actions:
- name: Display flutter version
# This use docker image
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- flutter --version
build_job:
# Link to docker-prepare to get output
## But I didn't define any steps, so it will not be runned.
## That allow to use it as template and shortcut other actions or jobs.
on:
after:
- docker-prepare
actions:
- id: build-setup
use: ${gitly.server.docker_repository}/${gitly.repository.url}
before_script:
- echo "${gitly.secrets.signing_jks_file_hex}" | xxd -r -p - > android-signing-keystore.jks
- "export VERSION_CODE=${gitly.flow.id} && echo $VERSION_CODE"
- "export VERSION_SHA=`echo ${gitly.event.commit_sha:0:8}` && echo $VERSION_SHA"
- cp ./fastlane/metadata/android/en-GB/changelogs/CURRENT_VERSION.txt "./fastlane/metadata/android/en-GB/changelogs/$VERSION_CODE.txt"
- chmod +x ./fastlane/create-changelog-mr.sh
- ./fastlane/create-changelog-mr.sh || true
- git checkout "${gitly.event.commit_sha}"
after_script:
- rm -f android-signing-keystore.jks || true
- id: upload-artifact
name: Upload Artifact
use: set_artifact@v1
args:
id: app_bundle
name: App Bundle
paths: app/build/outputs
# Run job in build steps
build-debug:
on:
steps:
- build
actions:
- name: Checkout
use: checkout@v1
# This extends will keep everything in the actions build_job.build-setup (scripts, before-scripts, after-scripts, env, ...)
## But you can append those elements here.
## If you fix an id on the action, you can replace the action by re-set it here with the fixed id too.
- extends: build_job.build-setup
name: Build debug
# This use docker image
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- bundle exec fastlane buildDebug
build-release:
on:
steps:
- build
actions:
- name: Checkout
use: checkout@v1
# Like you can see, when you use a docker image, the current workspace will be copied into the image.
## So if you use checkout here, the docker image will get the repository.
- extends: build_job.build-setup
name: Build release
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- bundle exec fastlane buildRelease
test-debug:
on:
after:
- build-debug
steps:
- test
actions:
- name: Checkout
use: checkout@v1
- name: Test Debug
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- bundle exec fastlane testDebug
test-release:
on:
after:
- build-release
steps:
- test
actions:
- name: Checkout
use: checkout@v1
- name: Test Release
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- bundle exec fastlane testDebug
publish-internal:
on:
after:
# Here it will wait for test to finish, because it run on steps publish, and all `publish` steps will always be run after `test` steps
- build-debug
steps:
- publish
actions:
- name: Downlaod Artifact
use: get_artifact@v1
args:
artifact: ${gitly.jobs.build-release.output.app_bundle.artifact_url}
path: app/build/outputs
- name: Publish to Playstore in Internal
use: ${gitly.server.docker_repository}/${gitly.repository.url}
scripts:
- bundle exec fastlane internal
It allow to get a Flow like the GitLab Pipelines with the power of Actions like GitHub Action.
If this is taken seriously caching should be mandatory and prio #1. GitLab is very bad in caching and GitHub makes it kinda ez. Would love to see some MinIO (or V native Object Storage) or Redis built-in solution. No file caching for security and enhanced methods to do dind builds and caching artifacts.
Especially with projects having node_modules or Angular prjects (which stores compiled files in node_modules) caching is quite a pain to setup and use properly.
Does CI need to be a YAML document? I know it's fairly standard in other forges, but I always struggle to follow what's happening in these documents and the syntax seems needlessly complicated to me. We already have v run and $tmpl files. I feel like it should be possible to express CI pipeline scripts purely with V.
You're 100% right @Cons-Cat.
Gitly will be using V scripts for CI.
YAML is terrible.
@medvednikov YAML is kind of cool because it is ez to read and write, but missing schema and weird alias/extending syntax makes it hard for this type of target solution.
I think TOML would be a good choice, or even JSON (but that even harder to read); both have schema support.
@medvednikov YAML is kind of cool because it is ez to read and write, but missing schema and weird alias/extending syntax makes it hard for this type of target solution.
YAML is "ez" to read and write... and just as "ez" to get wrong.
