spectacles-ci
Spectacles CI with Github Action is not running on the merged branch
We're working on integrating Spectacles into our CI workflow using GitHub Actions. After reviewing the Spectacles CLI docs and running tests using both the --branch and --commit-ref flags, we've observed that validations (e.g., content validation) are executed against the tip of the incoming branch (1)
This behavior doesn't align with standard CI expectations, where validation typically runs against the merged state—i.e., the result of merging the incoming (head) branch into the base (target) branch. This is critical for catching integration issues.
We were expecting behavior similar to how actions/checkout handles PRs in GitHub Actions—checking out the merge commit between the PR branch and the base branch for validation.
Is there a recommended pattern for achieving this with Spectacles, or is this currently a limitation?
Please find below the example GHA config we used for our tests.
.github/workflows/spectacles_ci.yml
name: Looker_CI
# Controls when the workflow will run
on:
pull_request:
branches: [ "main", "release" ]
workflow_dispatch:
env:
LOOKER_BASE_URL: <REDACTED> #enter your project link here instead
LOOKER_CLIENT_ID: ${{ secrets.LOOKER_API3_CLIENT_ID }} #should save this in secrets first
LOOKER_CLIENT_SECRET: ${{ secrets.LOOKER_API3_CLIENT_SECRET }} #should save this in secrets first
LOOKER_PROJECT: <REDACTED> #replace your project model name here
LOOKER_GIT_BRANCH: ${{ github.head_ref }}
SPECTACLES_REMOTE_RESET:
true
jobs:
must_pass:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: pip install spectacles
- name: Connect to Looker
run: spectacles connect
- name: LookML validator
if: ${{ success() || failure() }}
run: spectacles lookml
- name: SQL validator
if: ${{ success() || failure() }}
run: spectacles sql --ignore-hidden --incremental
- name: Test validation
if: ${{ success() || failure() }}
run: |
if spectacles assert 2>&1 >/dev/null | grep -q 'No data tests found'
then exit 0
else spectacles assert
fi
- name: Content validator
if: ${{ success() || failure() }}
run: spectacles content --incremental
*(1) example log we used to make the observation. The runner validated the tip ofrelease branch instead of creating a merged commit state.
➜ repo git:(feature-XXX) spectacles content --incremental -v
Authenticating to the Looker as client ID '[REDACTED]'
Checking Looker instance release version
Connected to Looker version 25.6.38 using Looker API 4.0
Setting up branch manager in project '[REDACTED_PROJECT]'
Setting Git state for project @ release:
.. Getting current workspace and branch
.. The active branch is '__LOOKER_PROD_[REDACTED_HASH]'
.. Switching to dev workspace
.. Setting project branch to 'release'
.. Resetting to remote HEAD
.. Branch 'release' @ HEAD now active in dev workspace
.. Manifest: no imported projects
Building LookML project @ release
Getting all models and explores from Looker
========== Validating 7 explores [incremental mode] ==========
Validating content in Looker...
Cleaning up Git state
.. Restoring branch to '__LOOKER_PROD_[REDACTED_HASH]'
.. Returning to production workspace
Starting content validation against target ref...
Setting Git state @ production:
.. Active branch: '__LOOKER_PROD_[REDACTED_HASH]'
.. Using commit [REDACTED_HASH] in production workspace
.. Manifest: no imported projects
Building LookML project @ [REDACTED_HASH]
Fetching models and explores from Looker...
Validating all content in Looker...
Cleaning up Git state...
.. Restoring to branch '__LOOKER_PROD_[REDACTED_HASH]'
✓ account_manager passed
✗ exposure failed
✓ functional_location passed
✓ history passed
✓ movements passed
✓ outstanding_repair passed
✓ repair_cost passed
✓ temp_geocity passed
Spectacles only has access to branches and commits available within Looker so unless the merge commit (a) can be passed to Spectacles and (b) is accessible in the remote repo i.e. GitHub, I'm not sure what you are describing would be possible.
You can set in a branch protection rule "Require branches to be up to date before merging" which will effectively accomplish the same thing for you I believe.
