Git history is bloated, uses a lot of disk space

[sam-w]

Summary

When checked out, the .git repo history uses ~0.5GB of disk space. Cloning takes a relatively long time, but the main issue is encountered when attempting to cache dependency checkouts between CI builds.

e.g. On CircleCI, I want to:

• Resolve my SwiftPM dependencies, including stripe-ios
• Cache the resulting objects, including: checkouts/stripe-ios and repositories/stripe-ios
• Reuse that cache for other build jobs, to avoid having to resolve my SwiftPM dependencies again

The total on-disk checkout size for stripe-ios is > 1GB. Obviously zipping reduces this down, but even that costs time on my CI server.

Impact

• CI time
• CI disk usage
• Costs us money

Proposed solution

Taking a wild guess: is this bloated history the result of many changes to the libs in InternalFrameworks/static stored in the git history? Could these files be stored using git-lfs instead? Obviously you'd need to rewrite your entire git history to make this happen, so I don't suggest it lightly. Maybe a new repo entirely?

iOS version

n/a

Installation method

SwiftPM

SDK version

n/a

[yuki-stripe]

Hi @sam-w,

Thanks for raising this issue, we definitely appreciate the pain here. We're investigating options now and will update this thread (cc @davidme-stripe)

[mihaicris-adoreme]

Switching from CocoaPods (90 MB) to SPM (500 MB). Stripe became the longest dependency to fetch, by far. Not happy!

[davidme-stripe]

We don't have many good options. We've considered rewriting our previous releases to replace the InternalFrameworks binaries with our now-open-source 3DS2 library, but that would be very unusual and could cause unexpected side effects.

@sam-w Could you try a shallow clone when building your cache? For example, git clone --depth 1 -b 21.4.0 https://github.com/stripe/stripe-ios will give you a 14MB .git folder and a checkout of version 21.4.0.

We also plan to support the Swift Package Registry once available, which will offer individual versions as ZIP archives.

[danpalmer]

I'm not sure if this solution would work for Stripe, but Klarna have separated their SPM package from the code repo like this: https://github.com/klarna/klarna-mobile-sdk-spm. It took their package checkout from ~gigabytes down to essentially nothing.

[mthole]

@davidme-stripe Any thoughts on the separate repo for the SPM package, per previous comment? My team has to fetch 700MB of Stripe repo history (as SPM will fetch the full --mirror style git repo; a shallow clone is not possible).

[davidme-stripe]

@mthole I understand the pain here, sorry that we haven't solved this yet! A year ago I was optimistic about Swift Package Registry, but I haven't seen much public information about it recently. (It looks like an experimental version might land in Swift 5.7.) We're hesitant to make a new repository just for SPM, as we'd need to support it ~forever.

Here's our plan: When Apple shares their 2022 roadmap on June 6th, we'll make a decision. If the announced version of Xcode includes Swift Package Registry support, we'll adopt that as soon as we can. If not, we'll create a lightweight mirror repo for SPM.

[davidme-stripe]

We've pushed version 22.5.0 to a new repo at https://github.com/stripe/stripe-ios-spm. All future versions will automatically be mirrored to this repo.

We'll update our docs to point to this repository soon.