pgcentralfoundation
Document CI caching strategy
When running PGRX in CI, it has to download every PG version, taking considerable time and resources. Many CI platforms including Github's offer various local caching systems, e.g. Swatinem/rust-cache action. Please document how CI steps should be modified in order to speed up CI builds (e.g. which dirs can be safely cached, but will be reliably updated when newer version becomes available, etc)
Should probably be added in #1694 too
uhh our current caching strategy is to not do the thing you describe.
@workingjubilee I am mostly concerned for the users of pgrx, not pgrx repo itself (although I am sure it can also benefit from it by reducing the CI run time). In order to help, I need to understand what gets re-downloaded on every CI run (especially for the users of pgrx), and to what dir, as well as the next steps (extracting to another dir, etc). If any of those dirs have a more or less stable state, we could use dir caching.
If someone does the very naive thing of running cargo pgrx init in CI, they would have to cache the $PGRX_HOME dir, as you mentioned. We do that in our CI, but we are fine with it not being meaningfully cached because we are in fact testing that we can create a new extension directory, execute the cargo pgrx init process for an arbitrarily picked Postgres version, and then build and test that extension.
Otherwise we build and test against a Postgres version we install from the system. This has the disadvantage that we don't test against a Postgres built with assertions.
@workingjubilee thx, does cargo pgrx init take into account the current state of the $PGRX_HOME? I.e. would it skip download/uncompress/compile/install steps if everything is already up to date? If not, is there a way to (semi-)reliably detect if pgrx init step should be skipped?
I tried running cargo pgrx init twice in a row locally, and it re-downloaded and re-compiled all PG versions. So it seems the ideal case would be to add some magical flag to the init command to skip if already initialized. Eventually I would even recommend to make the default behavior unless --force is specified.
@workingjubilee
If someone does the very naive thing of running
cargo pgrx initin CI
how else are you expected to use any pgrx command?
I'm trying to package the crate (cargo pgrx package) in CI, and the command returns an error unless I init first (which downloads every single Postgres version, etc), which is very annoying.
Changing things so intelligent non-overwriting behavior is used seems fine to me.