chore: replace docker shell script with js CLI commands and use for dev and test environments

[yusefnapora]

This closes #2121 by replacing the run-with-dependencies.sh script I added with a runWithServices function in scripts/cmds/services.js.

This does essentially the same thing as the shell script, which is to run docker compose up to bring up the services, get the local ports for the running services and override some config variables, then run the action you give it. It also registers a cleanup hook to run docker compose down whenever the process exits (including abnormal exit, Ctrl+C, etc).

I replaced the existing services start and services stop commands with services exec, which does the runWithServices thing to an arbitrary shell command you give it. The motivation here is to allow using a unique docker compose project name for each run (except in persistent mode, where the project name is always nft-storage-dev-persistent). With separate start and stop commands, you would need to write the project name to a file or something to keep track of it between invocations, but with the start and cleanup commands in the same process you can just keep the project name in memory. If you want something like the old behavior (because you want to manually poke at the db, say), you can run cli.js services exec bash to run a shell, and then run whatever you want inside the shell.

I also wired this up to yarn dev and added a new yarn dev:persist command, which I believe meets the requirements we were trying to hit in #1888, but with less boilerplate and without putting miniflare or the test runner into a container.

The persistent mode works by creating named docker volumes on the host, and using the docker-compose-volumes.yml file to point the services at those long-lived volumes instead of the ephemeral volumes specified in the main docker-compose.yml.

[cloudflare-pages[bot]]

Deploying with    Cloudflare Pages

Latest commit:	df7a91e
Status:	✅  Deploy successful!
Preview URL:	https://2c004b07.nft-storage-1at.pages.dev
Branch Preview URL:	https://chore-refactor-docker-test-s.nft-storage-1at.pages.dev

View logs

[yusefnapora]

cc @funkytek - would be great if you guys could play with this branch a bit and see if there are any rough spots I missed :)