straight.el
straight.el copied to clipboard
Published
20 hours ago •
radian-software
radian-software
Large disk space overhead
trafficstars
What's wrong
straight.el hasn't installed a single package yet, and it's already consuming north of 300M on disk:
Directions to reproduce
rm -rf ~/.emacs.d/straight- Apply the installation instructions
- If you are vacationing out in the sticks like I am with turn-of-the-century Internet connectivity, come back after lunch
Version information
- Emacs version: gccemacs 28.1 (built on June 10th, 2022)
- Operating system: MacOS Monterey 12.4 (21F79)
I was able to get it under 100M by explicitly adding :depth 1 to the recipe repositories in bootstrap.el.
Test Case
(straight-bug-report
:user-dir "disk-usage.straight"
:pre-bootstrap
(setq straight-repository-user "progfolio" straight-repository-branch "fix/nongnu-depth")
:post-bootstrap
(message "%S"
(straight--process-output "du" "-hSc" "/tmp/disk-usage.straight")))
- Test run at:
2022-07-29 14:30:32 - system-type:
gnu/linux - straight-version:
prerelease (HEAD -> fix/nongnu-depth, fork/fix/nongnu-depth) e3469e3 2022-07-29 - emacs-version:
GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6) of 2022-07-28
Output
Bootstrapping straight.el...
Bootstrapping straight.el...done
Looking for gnu-elpa-mirror recipe → Cloning melpa...
Looking for gnu-elpa-mirror recipe → Cloning melpa...done
Looking for nongnu-elpa recipe → Cloning gnu-elpa-mirror...
Looking for nongnu-elpa recipe → Cloning gnu-elpa-mirror...done
Looking for emacsmirror-mirror recipe → Cloning nongnu-elpa...
Looking for emacsmirror-mirror recipe → Cloning nongnu-elpa...done
Looking for emacsmirror-mirror recipe → Cloning el-get...
Looking for emacsmirror-mirror recipe → Cloning el-get...done
Looking for straight recipe → Cloning emacsmirror-mirror...
Looking for straight recipe → Cloning emacsmirror-mirror...done
Building straight...
Building straight...done
Test run with version: prerelease (HEAD -> fix/nongnu-depth, origin/fix/nongnu-depth) e3469e3 2022-07-29
"284K /tmp/disk-usage.straight/straight/build/straight
0 /tmp/disk-usage.straight/straight/build
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/logs
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/objects/info
44K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/refs
60K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git/info
24K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror/.git
92K /tmp/disk-usage.straight/straight/repos/emacsmirror-mirror
4.0K /tmp/disk-usage.straight/straight/repos/el-get/test/pkgs
248K /tmp/disk-usage.straight/straight/repos/el-get/test/issues
60K /tmp/disk-usage.straight/straight/repos/el-get/test
7.3M /tmp/disk-usage.straight/straight/repos/el-get/recipes
136K /tmp/disk-usage.straight/straight/repos/el-get/methods
188K /tmp/disk-usage.straight/straight/repos/el-get/logo
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.github/workflows
0 /tmp/disk-usage.straight/straight/repos/el-get/.github
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/logs
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/objects/info
1020K /tmp/disk-usage.straight/straight/repos/el-get/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/refs
60K /tmp/disk-usage.straight/straight/repos/el-get/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/el-get/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/el-get/.git/info
196K /tmp/disk-usage.straight/straight/repos/el-get/.git
360K /tmp/disk-usage.straight/straight/repos/el-get
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/packages
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/logs
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/objects/info
62M /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/refs
60K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git/info
40K /tmp/disk-usage.straight/straight/repos/nongnu-elpa/.git
88K /tmp/disk-usage.straight/straight/repos/nongnu-elpa
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/logs
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/objects/info
8.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/refs
60K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git/info
48K /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror/.git
0 /tmp/disk-usage.straight/straight/repos/gnu-elpa-mirror
0 /tmp/disk-usage.straight/straight/repos/melpa/working
4.0K /tmp/disk-usage.straight/straight/repos/melpa/service
12K /tmp/disk-usage.straight/straight/repos/melpa/scripts
21M /tmp/disk-usage.straight/straight/repos/melpa/recipes
4.0K /tmp/disk-usage.straight/straight/repos/melpa/packages
0 /tmp/disk-usage.straight/straight/repos/melpa/packages-stable
4.0K /tmp/disk-usage.straight/straight/repos/melpa/package-build/.github/workflows
0 /tmp/disk-usage.straight/straight/repos/melpa/package-build/.github
76K /tmp/disk-usage.straight/straight/repos/melpa/package-build
52K /tmp/disk-usage.straight/straight/repos/melpa/logo
8.0K /tmp/disk-usage.straight/straight/repos/melpa/html/partials
28K /tmp/disk-usage.straight/straight/repos/melpa/html/js
4.0K /tmp/disk-usage.straight/straight/repos/melpa/html/css
48K /tmp/disk-usage.straight/straight/repos/melpa/html
0 /tmp/disk-usage.straight/straight/repos/melpa/html-stable
8.0K /tmp/disk-usage.straight/straight/repos/melpa/docker/syncer
8.0K /tmp/disk-usage.straight/straight/repos/melpa/docker/rsyncd
20K /tmp/disk-usage.straight/straight/repos/melpa/docker/nginx
16K /tmp/disk-usage.straight/straight/repos/melpa/docker/logprocessor
12K /tmp/disk-usage.straight/straight/repos/melpa/docker/builder
4.0K /tmp/disk-usage.straight/straight/repos/melpa/docker
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.github/workflows
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.github
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/logs
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/objects/info
844K /tmp/disk-usage.straight/straight/repos/melpa/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/refs
60K /tmp/disk-usage.straight/straight/repos/melpa/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/melpa/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/melpa/.git/info
468K /tmp/disk-usage.straight/straight/repos/melpa/.git
80K /tmp/disk-usage.straight/straight/repos/melpa
20K /tmp/disk-usage.straight/straight/repos/straight.el/watcher
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight/repos/straight-mock-repo/subdir
8.0K /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight/repos/straight-mock-repo
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight/repos
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight/build/straight-mock-repo
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight/build
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d/straight
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks/.emacs.d
0 /tmp/disk-usage.straight/straight/repos/straight.el/tests/mocks
24K /tmp/disk-usage.straight/straight/repos/straight.el/tests
16K /tmp/disk-usage.straight/straight/repos/straight.el/scripts
12K /tmp/disk-usage.straight/straight/repos/straight.el/benchmark
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.github/workflows
12K /tmp/disk-usage.straight/straight/repos/straight.el/.github/ISSUE_TEMPLATE
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.github
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs/refs/heads/fix
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs/refs/heads
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs/refs
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/logs
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/objects/info
3.0M /tmp/disk-usage.straight/straight/repos/straight.el/.git/objects/pack
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/objects
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs/remotes/origin
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs/remotes
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs/tags
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs/heads/fix
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs/heads
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/refs
60K /tmp/disk-usage.straight/straight/repos/straight.el/.git/hooks
0 /tmp/disk-usage.straight/straight/repos/straight.el/.git/branches
4.0K /tmp/disk-usage.straight/straight/repos/straight.el/.git/info
20K /tmp/disk-usage.straight/straight/repos/straight.el/.git
792K /tmp/disk-usage.straight/straight/repos/straight.el
0 /tmp/disk-usage.straight/straight/repos
4.0K /tmp/disk-usage.straight/straight
0 /tmp/disk-usage.straight
98M total"
Packages:
"straight" n/a fix/nongnu-depth e3469e3 2022-07-29
"org-elpa" n/a n/a
"melpa" n/a master b33e8f7 2022-07-28
"gnu-elpa-mirror" n/a master 55b898c 2022-07-26
"nongnu-elpa" n/a main baad781 2022-07-24
"el-get" melpa master bf3dba4 2022-06-02
"emacsmirror-mirror" n/a master bc49945 2022-07-27
Don't have time to make a proper patch of this right now, but that may help.
Haven't made a patch for this because I don't think it's as simple as just defaulting to a shallow clone. We have to consider the case where a user may have a lockfile which needs to access the recipe repositories history. I'll have to think on a clean way to implement this and may not be able to get to it any time soon. If anyone has any ideas, I am willing to help review any pull requests for such a feature.
I think https://github.com/radian-software/straight.el/issues/149 is the future for low disk space consumption. Unfortunately, I do not believe it is possible to use shallow clones when the user has a lockfile, because you can't shallow-clone a specific ("unadvertised") commit from many Git repositories, you can only supply the name of a branch.
Agreed 300-400MB disk space is silly and the long setup time bothers me as well.
One mitigation we could do is default to shallow clones for the recipe repos, and just fallback to a full clone when we need a specific commit.