rules_apple icon indicating copy to clipboard operation
rules_apple copied to clipboard

Reduce disk space usage of the rules

Open luispadron opened this issue 10 months ago • 5 comments

This is a general issue to describe the current disk space usage issues of some of the rules.

The current default is to not use the use_tree_artifacts_output flag, this mostly seems to be because of potentially now fixed issues in Bazel like: https://github.com/bazelbuild/bazel/issues/16361. See also: https://github.com/bazelbuild/rules_apple/pull/2545

With the current default the disk space usage for a large-ish application target is >3x the size when compared to the tree artifact ouputs.

Without usage of the use_tree_artifacts_outputs flag (default):

bin/Code/Apps/App/App_archive-root/Payload/App.app (732 MB)
bin/Code/Apps/App/App-intermediates/unprocessed_archive.zip (732 MB)
bin/Code/Apps/App/App.ipa (740 MB)

With use_tree_artifacts_outputs flag:

bin/Code/Apps/App/App_archive-root/Payload/App.app (732 MB)

For large projects this can add several hundreds of gigabytes to the output-base which means engineers need to frequently bazel clean --expunge to clear up disk space (or worse run into failed builds when they run out of disk space).

This has also been a topic of concern for other users, see: https://github.com/bazelbuild/rules_apple/discussions/2638

luispadron avatar Jan 25 '25 00:01 luispadron

To start, do we need unprocessed_archive.zip, at least going off this comment it seems related to Tulsi which is AFAIK deprecated:

https://github.com/bazelbuild/rules_apple/blob/c5a3307c481c8fb5f9d8eefe53a62e1902b80410/apple/internal/processor.bzl#L570-L576

luispadron avatar Jan 25 '25 00:01 luispadron

I don't think we need it.

brentleyjones avatar Feb 10 '25 16:02 brentleyjones

Turns out the comment saying this was for Tulsi only is incorrect, its the archive used when not using tree artifact outputs.

luispadron avatar Feb 10 '25 16:02 luispadron

FWIW i think the unprocessed archive predates the quality tree artifact support in bazel in general. back in 2018 those just didn't work at all. so i imagine there is some way to eliminate it at this point.

keith avatar Feb 10 '25 17:02 keith

Yeah id love to make tree artifacts the default, theres this blocker right now: https://github.com/bazelbuild/rules_apple/pull/2545

luispadron avatar Feb 10 '25 18:02 luispadron