spectacles
spectacles copied to clipboard
spectacles-ci
Redundant git checkouts are occuring in runner.py
This issue arises in two different scenarios:
- In incremental mode, if no explores are found to be tested, we still enter the branch manager here
- In non-fail-fast mode, if no explores error the top-level tests, we still enter the branch manager a number of times here
With a bit of conditional logic, we could probably save some run-time.
Here are some logs for when no explores are found to be tested. You can see that we run the branch manager when more than needed.
Authenticating Looker API credentials
Checking Looker instance release version
Connected to Looker version 22.4.29 using Looker API 3.1
Setting up branch manager in project 'spectacles'
Getting the workspace in use by this session
Getting active branch for project 'spectacles'
The active branch is 'master'
Setting Git state for project 'spectacles' @ no-changes:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'no-changes'
.. Branching 'tmp_spectacles_c06279827b' off 'origin/no-changes'. Afterwards, restoring
to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_c06279827b' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_c06279827b' on project 'spectacles' to ref
'origin/no-changes'
.. Set project 'spectacles' to branch 'tmp_spectacles_c06279827b' @ HEAD in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building explore tests for the desired ref
Getting all models and explores from https://spectacles.looker.com
Getting all dimensions from explore open_source/fct_invocations
Creating async query for open_source/fct_invocations/*
Query for open_source/fct_invocations/* created as query 4332
Retrieving the SQL for query ID 4332
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_c06279827b'
.. Setting project 'spectacles' branch to 'no-changes'
.. Deleting branch 'tmp_spectacles_c06279827b' in project 'spectacles'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ production:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'no-changes'
.. Branching 'tmp_spectacles_994232dccb' off 'b64f625d972c2c4299330f3104650519fe7320fd'.
Afterwards, restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_994232dccb' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_994232dccb' on project 'spectacles' to ref
'b64f625d972c2c4299330f3104650519fe7320fd'
.. Set project 'spectacles' to branch 'tmp_spectacles_994232dccb' @ b64f62 in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building explore tests for the target ref
Getting all models and explores from https://spectacles.looker.com
Getting all dimensions from explore open_source/fct_invocations
Creating async query for open_source/fct_invocations/*
Query for open_source/fct_invocations/* created as query 4332
Retrieving the SQL for query ID 4332
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_994232dccb'
.. Setting project 'spectacles' branch to 'no-changes'
.. Deleting branch 'tmp_spectacles_994232dccb' in project 'spectacles'
.. Updating session to use the production workspace
Found 0 explore tests with unique SQL
==================== Testing 0 explores [concurrency = 10] =====================
Setting Git state for project 'spectacles' @ no-changes:
.. Updating session to use the dev workspace
.. Setting project 'spectacles' branch to 'no-changes'
.. Resetting branch to remote.
.. Set project 'spectacles' to branch 'no-changes' @ HEAD in dev workspace [ephemeral =
False]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ no-changes:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'no-changes'
.. Branching 'tmp_spectacles_5783c1e128' off 'origin/no-changes'. Afterwards, restoring
to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_5783c1e128' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_5783c1e128' on project 'spectacles' to ref
'origin/no-changes'
.. Set project 'spectacles' to branch 'tmp_spectacles_5783c1e128' @ HEAD in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building dimension tests for the desired ref
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_5783c1e128'
.. Setting project 'spectacles' branch to 'no-changes'
.. Deleting branch 'tmp_spectacles_5783c1e128' in project 'spectacles'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ production:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'no-changes'
.. Branching 'tmp_spectacles_0d0c202397' off 'b64f625d972c2c4299330f3104650519fe7320fd'.
Afterwards, restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_0d0c202397' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_0d0c202397' on project 'spectacles' to ref
'b64f625d972c2c4299330f3104650519fe7320fd'
.. Set project 'spectacles' to branch 'tmp_spectacles_0d0c202397' @ b64f62 in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building dimension tests for the target ref
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_0d0c202397'
.. Setting project 'spectacles' branch to 'no-changes'
.. Deleting branch 'tmp_spectacles_0d0c202397' in project 'spectacles'
.. Updating session to use the production workspace
Removing errors that would exist in project @ production
✓ open_source.fct_invocations skipped
Completed SQL validation in 20 seconds.
Here are logs when the explores do differ incrementally, but the explore level check passes. Again, you can see redundant checkout calls.
Authenticating Looker API credentials
Checking Looker instance release version
Connected to Looker version 22.4.29 using Looker API 3.1
Setting up branch manager in project 'spectacles'
Getting the workspace in use by this session
Getting active branch for project 'spectacles'
The active branch is 'master'
Setting Git state for project 'spectacles' @ hidden-dimensions:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'tmp_spectacles_7d7ee974ec'
.. Branching 'tmp_spectacles_92ff558c75' off 'origin/hidden-dimensions'. Afterwards,
restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_92ff558c75' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_92ff558c75' on project 'spectacles' to ref
'origin/hidden-dimensions'
.. Set project 'spectacles' to branch 'tmp_spectacles_92ff558c75' @ HEAD in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building explore tests for the desired ref
Getting all models and explores from https://spectacles.looker.com
Getting all dimensions from explore open_source/fct_invocations
Creating async query for open_source/fct_invocations/*
Query for open_source/fct_invocations/* created as query 11718
Retrieving the SQL for query ID 11718
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_92ff558c75'
.. Setting project 'spectacles' branch to 'tmp_spectacles_7d7ee974ec'
.. Deleting branch 'tmp_spectacles_92ff558c75' in project 'spectacles'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ production:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'tmp_spectacles_7d7ee974ec'
.. Branching 'tmp_spectacles_1d39b099e8' off 'b64f625d972c2c4299330f3104650519fe7320fd'.
Afterwards, restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_1d39b099e8' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_1d39b099e8' on project 'spectacles' to ref
'b64f625d972c2c4299330f3104650519fe7320fd'
.. Set project 'spectacles' to branch 'tmp_spectacles_1d39b099e8' @ b64f62 in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building explore tests for the target ref
Getting all models and explores from https://spectacles.looker.com
Getting all dimensions from explore open_source/fct_invocations
Creating async query for open_source/fct_invocations/*
Query for open_source/fct_invocations/* created as query 9623
Retrieving the SQL for query ID 9623
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_1d39b099e8'
.. Setting project 'spectacles' branch to 'tmp_spectacles_7d7ee974ec'
.. Deleting branch 'tmp_spectacles_1d39b099e8' in project 'spectacles'
.. Updating session to use the production workspace
Found 1 explore tests with unique SQL
===================== Testing 1 explore [concurrency = 10] =====================
Setting Git state for project 'spectacles' @ hidden-dimensions:
.. Updating session to use the dev workspace
.. Setting project 'spectacles' branch to 'hidden-dimensions'
.. Resetting branch to remote.
.. Set project 'spectacles' to branch 'hidden-dimensions' @ HEAD in dev workspace
[ephemeral = False]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Starting a new loop, 1 tests queued
10 available query slots, creating query task
Starting query 11718
Query 11718 is running under query task 5d4a80bbbc6121d8621759124f9f9483
Checking for results of 1 query tasks
Attempting to get results for 1 query tasks
Query task 5d4a80bbbc6121d8621759124f9f9483 status is: added
Checking for results of 1 query tasks
Attempting to get results for 1 query tasks
Query task 5d4a80bbbc6121d8621759124f9f9483 status is: running
Checking for results of 1 query tasks
Attempting to get results for 1 query tasks
Query task 5d4a80bbbc6121d8621759124f9f9483 status is: complete
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ hidden-dimensions:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'hidden-dimensions'
.. Branching 'tmp_spectacles_ae84f26c3a' off 'origin/hidden-dimensions'. Afterwards,
restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_ae84f26c3a' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_ae84f26c3a' on project 'spectacles' to ref
'origin/hidden-dimensions'
.. Set project 'spectacles' to branch 'tmp_spectacles_ae84f26c3a' @ HEAD in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building dimension tests for the desired ref
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_ae84f26c3a'
.. Setting project 'spectacles' branch to 'hidden-dimensions'
.. Deleting branch 'tmp_spectacles_ae84f26c3a' in project 'spectacles'
.. Updating session to use the production workspace
Setting Git state for project 'spectacles' @ production:
.. Updating session to use the dev workspace
.. Getting the workspace in use by this session
.. Getting active branch for project 'spectacles'
.. The active branch is 'hidden-dimensions'
.. Branching 'tmp_spectacles_d89c64b3dc' off 'b64f625d972c2c4299330f3104650519fe7320fd'.
Afterwards, restoring to branch 'master' in project 'spectacles'
.. Creating branch 'tmp_spectacles_d89c64b3dc' on project 'spectacles'
.. Hard resetting branch 'tmp_spectacles_d89c64b3dc' on project 'spectacles' to ref
'b64f625d972c2c4299330f3104650519fe7320fd'
.. Set project 'spectacles' to branch 'tmp_spectacles_d89c64b3dc' @ b64f62 in dev
workspace [ephemeral = True]
.. Getting manifest details
.. Project 'spectacles' imports the following projects: []
.. Project 'spectacles' doesn't import any other projects
Building dimension tests for the target ref
Cleaning up Git state in 'spectacles'
.. Restoring project 'spectacles' to branch 'master' and deleting temporary branch
'tmp_spectacles_d89c64b3dc'
.. Setting project 'spectacles' branch to 'hidden-dimensions'
.. Deleting branch 'tmp_spectacles_d89c64b3dc' in project 'spectacles'
.. Updating session to use the production workspace
Removing errors that would exist in project @ production
✓ open_source.fct_invocations passed
Completed SQL validation in 23 seconds.
