s2i-python-container icon indicating copy to clipboard operation
s2i-python-container copied to clipboard

Add support for incremental builds

Open rarguello opened this issue 6 years ago • 8 comments

rarguello avatar Jan 27 '19 14:01 rarguello

Could you please provide more info? What do you mean by incremental builds and why is this important?

frenzymadness avatar Mar 19 '19 10:03 frenzymadness

@omron93 Do we use incremental builds in other containers?

torsava avatar Mar 19 '19 10:03 torsava

@torsava soon in nodejs: https://github.com/sclorg/s2i-nodejs-container/pull/201

pkubatrh avatar Mar 19 '19 10:03 pkubatrh

@pkubatrh Ah, thanks. I thought the incremental build feature of s2i was kinda buggy/unfinished. So I would probably wait to see how it works out for nodejs, see if they can work out the kinks before implementing it here.

torsava avatar Mar 19 '19 14:03 torsava

@pkubatrh @torsava https://github.com/sclorg/golang-container supports incremental builds for some time already. I'm not aware of any problems with it. @jcajka Are you?

omron93 avatar Mar 23 '19 19:03 omron93

I think this would really help adoption. Right now, using s2i means waiting for a really slow build vs. a Dockerfile where requirements.txt would be cached.

jtratner avatar Jun 27 '19 05:06 jtratner

I just found this article about how to make incremental builds work using s2i python container. https://developpaper.com/using-s2i-to-build-images-from-source-code/

wjhrdy avatar Aug 20 '21 14:08 wjhrdy

So, If I understand it correctly from the node and go containers, it should work for Python container like this:

  • Users prepare their own Dockerfile, where they call the save-artifacts script which basically creates a tarball from the main virtual environment with all app dependencies already installed.
  • Then, during a second build, if the tarball is provided in /tmp/artifacts, it's extracted to the original location (virtual environment).
  • Assemble script still tries to install all dependencies but because some or ideally all of them are already there, the second build is much faster.

Is that correct?

I don't use s2i because it does not support podman so is there any significant difference when s2i is used and a user does not have to prepare its own dockerfile manually?

frenzymadness avatar Aug 23 '21 11:08 frenzymadness