justice40-tool icon indicating copy to clipboard operation
justice40-tool copied to clipboard

Update create-score-version Github Action

Open travis-newby opened this issue 1 year ago • 5 comments

This PR updates the create-score-version Github Action to,

  • Separate the action into a few logical steps, and organize the action around those logical steps. This fixes a potential issue where the backend score could be deployed then fail a smoke test, resulting in no tiles being built or uploaded.
  • Use the test input option to upload everything to a new test folder and run smoke tests from there. That way we can test this script without deploying version 1.0 again.
  • Separate out the "extra" uploaded files (from data_pipeline/files) to be in their own folder, and name that folder data_pipeline/files/static to indicate that these files don't change (often). Files in this folder will get automatically uploaded during the build process.

travis-newby avatar Mar 08 '23 19:03 travis-newby

** Score Deployed! ** Find it here:

  • Score Full usa.csv: https://justice40-data.s3.amazonaws.com/data-pipeline-staging/2194/85d591412757cf2922d6974820a644ed5d547935/data/score/csv/full/usa.csv
  • Download Zip Packet: https://justice40-data.s3.amazonaws.com/data-pipeline-staging/2194/85d591412757cf2922d6974820a644ed5d547935/data/score/downloadable/Screening_Tool_Data.zip

github-actions[bot] avatar Mar 08 '23 23:03 github-actions[bot]

Score Comparison Summary

Hi! I'm the Score Comparator. I compared the score in production (version 1.0) to the locally calculated score. Here are the results.

Columns

I compared the columns. Here's what I found.

  • There are no differences in the column names.

Scores

I compared the scores, too. Here's what I found.

  • The production score has 74,134 census tracts, and the freshly calculated score has 74,134. They match!
  • The total population in all census tracts in the production score is 328,267,709.0. The total population in all census tracts locally is 328,267,709.0. They match!
  • There are 27,248 disadvantaged tracts in the production score representing 33.3% of the total population, and 27,248 in the locally generated score representing 33.3% of the total population. The number of tracts match!
  • There are 0 tract(s) marked as disadvantaged in the production score that are not disadvantaged in the locally generated score (i.e. disadvantaged tracts that were removed by the new score). There are 0 tract(s) marked as disadvantaged in the locally generated score that are not disadvantaged in the production score (i.e. disadvantaged tracts that were added by the new score).
  • I compared all values across all census tracts. There are 0 tracts with at least one difference. Please examine the logs or run the score comparison locally to view them all.

github-actions[bot] avatar Mar 08 '23 23:03 github-actions[bot]

** Map Deployed! ** Map with Staging Backend: https://screeningtool.geoplatform.gov/en?flags=stage_hash=2194/85d591412757cf2922d6974820a644ed5d547935 Find tiles here: https://justice40-data.s3.amazonaws.com/data-pipeline-staging/2194/85d591412757cf2922d6974820a644ed5d547935/data/score/tiles

github-actions[bot] avatar Mar 09 '23 00:03 github-actions[bot]

@travis-newby - I don't believe the cache invalidation is in the script. We could place it in another ticket if you think that's better.

Just set this up here.

As an FYI, the BE distribution ID is E1324VDMNCO97N. All other env vars would be the same as above.

vim-usds avatar Mar 10 '23 22:03 vim-usds

Also a fast follow will be to update the FE links once these files are on S3 :)

vim-usds avatar Mar 10 '23 22:03 vim-usds