Add cache to save installation time

[LexanRed]

Hello there,

currently the execution of this action takes about 30s, most of that is for installation and setup of python, pip, dependencies. Since setup-python now supports caching, could we add that functionality to cut down the execution time of this action?

Installing it every time again does not make sense to me in this particular case.

https://github.blog/changelog/2021-11-23-github-actions-setup-python-now-supports-dependency-caching/

If you do not want to add caching, could you tell me which directories should be cached in order to skip the installation the next time? I will then setup a cache manually.

Thanks a lot!

[sondrelg]

Good question :thinking: Is it the installation of dependencies that are taking a long time, do you think? 30 seconds definitely sound like too much.

The cache example you showed seems worth implementing regardless. Would you like to create a PR?

[LexanRed]

Yes, it is the installation.

It varies between 20 and 30 seconds most of the time. You can see clearly, the second action is done in significantly less time than the first run.

As soon as I find time to investigate this further and get to a solution, I will let you know. At the moment unfortunately I will not have time for that. Sorry.

[shimonelbaz]

another good option would be change the action to be running from a container. then use a python slim as base image so we could have a very small container to use..

[sondrelg]

It has been a while since I looked at container actions. How do they work again? Will action runs then pull pre-built container images from somewhere, or will they be built on-demand?

[till]

@sondrelg 👋🏼 I came here to request the same.

To answer your question: if you build on demand, you just add another layer of things (Docker) that need to happen on top of the the pip install. I would suggest to build a ready image.

I think to do it, the steps are as follows:

1. Adjust the code to be runable in a container, e.g. provide the right inputs as environment variables.
2. Build and publish a public image (when you create a release), e.g. in the registry of this organization.
3. Adjust the action.yml to use the image.

If you take a look here, then instead of Dockerfile, you use the image you published: https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action#creating-an-action-metadata-file

I think unless you provide a fluent image tag like :latest (which also has downsides), it will require updating the action.yml before you publish a new release — so it's included in the tag.

Advantage of this is to no longer rely on other extensions in the run — and also save time on setup: https://github.com/snok/container-retention-policy/blob/main/action.yml#L67-L79

So, instead of action/setup-python you select a suitable base image with the version you need, and the pip install would move into a RUN statement.

Haven't tested it, but to build your action with a Dockerfile in your repository root:

FROM python:3.11-bookworm
RUN mkdir -p /app
WORKDIR /app
COPY . .
RUN pip --disable-pip-version-check install \
          regex==2022.3.2 \
          httpx \
          dateparser \
          pydantic

ENTRYPOINT ["/usr/bin/python", "/app/main.py"]

(Or add #!/usr/bin/python to main.py and make it executable.)

The above doesn't account for the inputs yet — my guess would be environment variables for the inputs would be the easiest to map them from the yaml to the container.

I did something similar here: https://github.com/hostwithquantum/github-org-sync-action/blob/main/action.yml#L29-L38

But it's been a while.

Happy to help, I didn't want to come on here and make demands. 😃

[sondrelg]

I'm considering rewriting the action as a Rust app, which should result in a ~20mb final docker image.

Think a v3 release would also be a good opportunity to fix most outstanding issues and get rid of a bit of cruft that has built up over time in the configuration options.

[sondrelg]

Just want to give a quick update here: the v3 release is almost ready, and I've ended up with a 2.4Mi docker image. The start up time for my test runs are now < 1 second, and the total runtime of a run deleting 30 images is 1.2 seconds.

[sondrelg]

The v3 release is now out. The runtime for deleting up to 180 package versions (that's the rate limit per minute) is ~2 seconds, and initialization is now less than a second 🥳

The migration guide for v3 is included in the release post 👍 If you run into any issues, please share them in the issue opened for tracking the v3 release ☺️