code-dot-org icon indicating copy to clipboard operation
code-dot-org copied to clipboard

ci.rake: `pdm install` before `rake lint` is run

Open snickell opened this issue 1 year ago • 2 comments

When doing the chef rollout for the PyCall PR #60048, we discovered that staging calls rake lint from rake ci from aws/ci_build. Its called before rake install is run, which means that the Python linter pdm run ruff check has not been installed, and thus the build fails (build log).

This PR:

  • Manually calls pdm install from rake ci:build just after running rake ci:chef_update and just prior to running rake lint.

I did not make it conditional on CDO.lint, because it felt better to have the install done at a relatively predictable/stable point as this step can involved installing a new python interpreter when version change, etc.

snickell avatar Aug 09 '24 05:08 snickell

This shows where it will go in the build process for staging:

git fetch
git pull --ff-only origin staging
GetSecretValue: staging/cdo/slack_endpoint
sudo bundle config set --local without development production adhoc test levelbuilder integration
sudo bundle install --quiet --jobs 16
GetSecretValue: staging/cdo/dashboard_honeybadger_api_key
No value provided for required options '--repository'
RAILS_ENV=staging RACK_ENV=staging bundle exec rake ci
GetSecretValue: staging/cdo/slack_token
GetSecretValue: staging/cdo/slack_bot_token
GetSecretValue: staging/cdo/slack_endpoint
sudo curl -o /opt/chef/embedded/ssl/certs/cacert.pem https://raw.githubusercontent.com/code-dot-org/code-dot-org/staging/cookbooks/cacert.pem
sudo bundle config set --local without development production adhoc test levelbuilder integration
sudo bundle install --quiet --jobs 16
RAILS_ENV=staging RACK_ENV=staging bundle exec ./update_cookbook_versions
RAILS_ENV=staging RACK_ENV=staging bundle exec berks install
git add .
git commit -m "Updated cookbook versions"
git push origin staging
RAILS_ENV=staging RACK_ENV=staging bundle exec berks upload --no-freeze
RAILS_ENV=staging RACK_ENV=staging bundle exec berks apply staging
sudo /opt/chef/bin/chef-client --chef-license accept-silent
Finished ci:chef_update (2 minutes)

>>>>.  new `pdm install` would go here   <<<<<
WARNING: Project requires a python version of ==3.12.*, The virtualenv is being created for you as it cannot be matched to the right version.
INFO: python.use_venv is on, creating a virtualenv for this project...
STATUS: Downloading [email protected]
STATUS: Installing [email protected]
Virtualenv is created successfully at /home/ubuntu/staging/.venv
_SNIP_


RAILS_ENV=staging RACK_ENV=staging bundle exec rubocop --parallel
Finished lint:ruby (2 minutes)
RAILS_ENV=staging RACK_ENV=staging bundle exec haml-lint dashboard pegasus shared
Finished lint:haml (1 minute)
RAILS_ENV=staging RACK_ENV=staging bundle exec scss-lint
Finished lint:scss (less than a minute)
yarn 
npm run lint
../../apps/node_modules/eslint/bin/eslint.js *.js
Finished lint:javascript (2 minutes)
pdm run ruff check

snickell avatar Aug 09 '24 05:08 snickell

@code-dot-org/infrastructure bump for review, this is a one-line change and will be necessary to have again if/when we rebuild staging from scratch.

snickell avatar Aug 23 '24 09:08 snickell