unstract icon indicating copy to clipboard operation
unstract copied to clipboard

refactor: Dockerfile optimization to build faster

Open chandrasekharan-zipstack opened this issue 10 months ago • 5 comments

What

  • Refactored prompt-service dockerfile
    • Reordered instructions to cache install of gunicorn and other libs
  • Refactored all other service's Dockerfiles in a similar fashion to cache layers better

Why

  • Better caching results in lesser Dockerfile build times (11.67% faster for prompt-service, didn't measure for others)

How

  • Installing libs that don't change often earlier to leverage Docker's caching

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No, only reordered instructions

Related Issues or PRs

  • #1125 - Similar optimization

Notes on Testing

  • Tried building locally and ran some prompts from Prompt studio
  • TODO: Test behaviour with tool - easier to do this in staging

Screenshots

[for prompt-service alone]

  1. Before optimization (252.7s) image
  2. After optimization (223.2s) image
  3. Tool run in a workflow image

Checklist

I have read and understood the Contribution Guidelines.

@chandrasekharan-zipstack I see 3 new sonar issues. Please check it.

Deepak-Kesavan avatar Feb 11 '25 11:02 Deepak-Kesavan

@ritwik-g my main motive was to reduce the time spent on builds on the dev machine. Will look into caching our builds with the github runner as well and maybe raise it as a separate PR if the changes grow in size.

@chandrasekharan-zipstack Overall great initiative! Any boost in DX is a huge welcome.

Also CI runner cache can definitely be looked at next as @ritwik-g mentioned.

hari-kuriakose avatar Feb 17 '25 02:02 hari-kuriakose

filepath function $$\textcolor{#23d18b}{\tt{passed}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_logs}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup\_skip}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_client\_init}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image\_exists}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config\_without\_mount}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_run\_container}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{9}}$$ $$\textcolor{#23d18b}{\tt{9}}$$

github-actions[bot] avatar Feb 19 '25 05:02 github-actions[bot]