checkout icon indicating copy to clipboard operation
checkout copied to clipboard

How to use `lfs` with external storage?

Open retzero opened this issue 11 months ago • 2 comments

Hello. I'd like to use our own s3-compatible storage for LFS blobs. It works fine in command line interface. How to use this actions/checkout action for the same? Example usage is very welcome!

Here is my current .lfsconfig;

[lfs]
   url = "https://{ID}:{TOKEN}@my-external-s3.com"

retzero avatar Feb 06 '25 07:02 retzero

Hello @retzero! I just ran into this problem. We using Gitea as LFS server, ssh_known_hosts, ssh_configuration and ssh private key For Accessing Gitea Over SSH have been added to the REPOSITORY Secrets. That Secrets is Correctly Appended to `/.ssh/config, ~/.ssh/known_hosts and ~/.ssh/privatekey files, but Fetching of LFS objects does not work. That Action seems to simply ignore .lfsconfig with the right URL, and is trying to get lfs objects From Github. So the solution is as follows:

  1. Configure ssh using repository / organization secrets.
  2. Use actions/checkout action to checkout repository, but keep lfs false by default.
  3. Fetch lfs objects manually!
      - name: Setup LFS
        shell: bash
        run: |
          mkdir -p ~/.ssh
          echo '${{ secrets.LFS_SSH_HOST_CONFIGURATION }}' >> ~/.ssh/config
          echo '${{ secrets.LFS_SSH_KNOWN_HOSTS }}' >> ~/.ssh/known_hosts
          echo '${{ secrets.LFS_SSH_PRIVATE_KEY }}' > ~/.ssh/private_key
          chmod 600 ~/.ssh/private_key

      - name: Checkout
        uses: actions/checkout@v4

      - name: Checkout lfs objects
        run: |
          git lfs install
          git lfs fetch
          git lfs checkout

Danstiv avatar Feb 22 '25 14:02 Danstiv

I use lfs-s3 to store my LFS objects.

For me, I just need to set the necessary environment variables add the following step before checking out code.

For example:

jobs:
  pull-lfs-objects:
    name: Pull LFS Objects
    runs-on: ubuntu
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_REGION: ${{ secrets.AWS_REGION }}
      S3_BUCKET: ${{ secrets.S3_BUCKET }}

    steps:
      - name: Install Git
        run: |
          apt update
          apt install -y git git-lfs

      - name: Set up lfs-s3
        run: |
          wget -O /usr/local/bin/lfs-s3 https://github.com/nicolas-graves/lfs-s3/releases/download/0.1.7/lfs-s3-linux
          chmod +x /usr/local/bin/lfs-s3
          git config --global --add lfs.customtransfer.lfs-s3.path lfs-s3
          git config --global --add lfs.standalonetransferagent lfs-s3

      - name: Checkout code
        uses: actions/checkout@v4
        with:
          lfs: true

l0rem1psum avatar May 28 '25 09:05 l0rem1psum