woocommerce-android icon indicating copy to clipboard operation
woocommerce-android copied to clipboard

[AINFRA-1533] Adopt `git-conceal` in this repo

Open AliSoftware opened this issue 1 month ago • 10 comments

Closes: AINFRA-1533

What

This PR migrates the repo to use git-conceal instead of configure_apply, so that the usage of secrets in the repo gets easier and more transparent for developers.

How

  • 🗑️ Delete .configure and .configure-files/*.enc files
  • Add files that were listed in .configure as encrypted files in the repo instead, by also adding their path as entries in the .gitattributes with filter=git-conceal diff=git-conceal—so they get encrypted by git-conceal on push
  • Update .gradle files and Fastfile to account for new file paths
  • Update .gradle code to account for the fact that now the files are present (but encrypted) after cloning the repo—as opposed to being absent when we didn't run configure_apply in the old way.
  • Replace calls to bundle exec fastlane run configure_apply in .buildkite/* commands—and calls to configure_apply(force: true) in Fastfile—with calls to git-conceal-unlock, and bump ci-toolkit plugin to point to https://github.com/Automattic/a8c-ci-toolkit-buildkite-plugin/pull/195 where this helper lives
  • 📚 Updated documentation

[!NOTE]

ℹ️ About the git-conceal-helper helper from ci-toolkit

The way this currently works is that:

  • The git-conceal GitHub repo builds new binaries for macOS, Linux and Windows when we do a new release, and attach them as assets to the GitHub Release
  • The git-conceal GitHub repo also provides an install.sh script to make it easy to install git-conceal for your current platform (the script takes care of detecting if you're on a Linux, macOS or Windows agent, downloads the right asset from the latest GitHub release for that platform, chmod +x it and install it)
  • The git-conceal-helper wrapper from our a8c-ci-toolkit plugin is then just a small bootstrap/helper script that checks if git-conceal is installed in the CI agent, if so calls git-conceal unlock "env:${1:-GIT_CONCEAL_SECRET_KEY}" and exit; otherwise calls the install.sh script of git-conceal to install it first before calling git-conceal unlock

The way this might work in the future:

  • We will probably ship git-conceal in most of our self-hosted CI agents and custom AMIs to avoid having to download it on each job… even if in practice it takes less than 1s to download and unlock the repo
  • We will still keep that wrapper script in ci-toolkit that does the "tests if installed already and install it if not" logic, because not all our CI agents are custom agents (e.g. we sometimes use the default queue which uses EC2 instances with the official Buildkite AMI, not customized by us), so that would still be useful for those.

Merge timing

[!IMPORTANT] Do not merge this PR until (1) we have all the documentation / FAQ ready for all devs to follow in FG and (2) we have validated similar PRs in other repos—especially ones using macOS/Windows and Tumblr CI agents—work too..

Test Steps

General check

  • Verify CI is green
  • Checkout this branch and validate that all the secret files added to the repo in this PR are encrypted and unreadable:
    • WooCommerce/google-services.json
    • WooCommerce/upload.jks
    • debug.keystore
    • firebase.secrets.json
    • google-upload-credentials.json
    • secrets.properties
    • sentry.properties
  • Review the documentation updates
  • Review and ensure all the changes necessary in *.gradle files to reflect the new setup are covered by this PR (path updates to .properties or keystore files, code updates around logic that was based on if a secret file exists or not, …)

Follow the README.md instructions as an Automattician

  • Clone the repo from scratch in a new directory on your Mac, then checkout this PR's branch
  • Copy the git-conceal encryption key from our Secret Store in your clipboard, then run pbpaste | base64 -d | git-conceal unlock - at the root of your new clone's directory
  • Validate that after this step, you are able to read the content of secret files like secrets.properties in clear now
  • Build and Run the project, validating everything works as expected (including e.g. Google login to validate google-services.json is taken into account, etc)

[!NOTE]

ℹ️ The real migration won't require devs to clone the repo from scratch; this is just for the testing steps of this PR

Above I suggest to test this scenario in a fresh clone of the repo rather than your everyday working copy, especially to avoid risking to leave your git repo in a state that is setup for git-conceal while you were just testing and the PR is not merged yet.

In practice, once the PR is merged in trunk, devs won't need to clone the repo fresh to migrate to git-conceal though. They will just have to run pbpaste | base64 -d | git-conceal unlock - on their existing working copy once, and probably never run a git-conceal command again afterwards (except maybe git-conceal status if they want to validate a new secret file they're about to add is properly gonna be encrypted as they expect).

At some point I'd still highly recommend for them also get rid of the old files that were installed by their previous setup with configure (rm -rf ~/.configure/woocommerce-android and git clean -dxi -e .DS_Store -e .idea -e local.properties), to avoid confusion and risking relying on obsolete files.

Follow the README.md instructions as an external contributor

  • Clone the repo from scratch in a new directory on your Mac, then checkout this PR's branch
  • Validate files like secrets.properties show as encrypted garbage
  • Edit defaults.properties to add the relevant values for wp.oauth.*
  • Overwrite WooCommerce/google-services.json encrypted file with WooCommerce/google-services.json-example
  • Build and Run the project, validating everything works as expected (except Google login)

AliSoftware avatar Nov 14 '25 19:11 AliSoftware