stable-diffusion-webui-docker
stable-diffusion-webui-docker copied to clipboard
AbdBarho
[upgrade] Getting rid of unnecesary git clone of other repositories
Currently service "download" and other services are using sequence of commands:
git clone REPOSITORY_URL && cd REPO_DIR && git reset --hard COMMIT_SHA
This unnecesary downloads entire repository history only to change it's state to specific commit.
It would be better not to download entire repository, but zip file from specific commit instead. Download link format is https://github.com/{username}/{projectname}/archive/{sha}.zip - then there is only a matter of unzipping and renaming directory.
in the case of downloads using git, the depth flag exists to make shallow copies:
--depth <depth> create a shallow clone of that depth
EDIT: Usage example would then be:
mkdir -p repositories/CodeFormer && cd repositories/CodeFormer && git init && git remote add origin https://github.com/sczhou/CodeFormer && git fetch origin c5b4593074ba6214284d6acd5f1719b6c5d739af --depth=1 && git reset --hard c5b4593074ba6214284d6acd5f1719b6c5d739af
@AbdBarho sure, but I am running only AUTOMATIC1111 version. Is that ok, to improve only it's Dockerfile or I have to do rework of other ones (I am not sure whether I've got enough disk space to build all images).
Also those changes will surely bust docker cache forcing users to rebuild entire image almost from scratch, so it would be better to merge it later with some bigger changes also involving cache bust, as for now it is small gain only, but gain nonetheless.
@karpfediem that is also neat, maybe verbose, but still neat! is it faster than cloning the entire repo?
@DevilaN I would not worry too much about re-download, the git repositories themselves are rather small (with the exception of taming-transformers), the change in build time would be like + 2min, which is tolerable.
As you said, we can bundle that with a version update that will require a re-build anyway, and we are good to go. Don't stress about it if you can't, I could also implement it after finishing a prototypical implementation of #128
@karpfediem that is also neat, maybe verbose, but still neat! is it faster than cloning the entire repo?
Yes, it will only download that specific commit with git fetch.
I'd argue against archive downloads since they are very much github specific, not that i've seen any other repositories being used though to be fair :sweat_smile:
Remove .git directory afterwards. It is not necessary and takes half of space inside directory.