js-client icon indicating copy to clipboard operation
js-client copied to clipboard

Published 20 hours ago • verified publisher icon netlify

Support Netlify Large Media

Open calavera opened this issue 6 years ago • 6 comments

- Do you want to request a feature or report a bug?

There are a couple of things we need to add to support Netlify Large Media in the CLI.

- What is the current behavior?

We don't support Netlify LM

- What is the expected behavior?

When a site has LM enabled, we need to read information from LFS pointer files to add them to the deploy requests. This logic is already implemented in the Go client:

https://github.com/netlify/open-api/blob/master/go/porcelain/deploy.go#L512-L532

The original file size needs to be sent as a parameter when we upload a file:

https://github.com/netlify/open-api/blob/master/go/porcelain/deploy.go#L424-L431

There is a caveat to keep in mind in this implementation:

https://github.com/netlify/open-api/commit/2af0664603e56943a2c0db43175405270f4792b6

Maybe @keiko713 can say more about that special hashed key.

calavera avatar Feb 26 '19 21:02 calavera

We did a code walkthrough together the other day. I think we can get this added fairly soon. I'll take a stab at it when I have time.

bcomnes avatar Feb 26 '19 21:02 bcomnes

I need to go and ask for Keiko Sept 2018.

keiko713 avatar Feb 26 '19 22:02 keiko713

actually what do I need to do here? is there any specific question?

keiko713 avatar Feb 27 '19 18:02 keiko713

@keiko713 there is, but we can maybe pair and talk about it when I (or whoever does this) needs the information and form a specific question.

bcomnes avatar Feb 27 '19 18:02 bcomnes

Ok, catching up on this.

A few caveats/edge cases:

  • The buildbot clones the LFS backed repo down, without the LFS media swapped in place for the file pointers.
  • A build process then moves those pointer file (names the same as the actual large files) into a build directory.
  • The go-client deploy algorithm treats these pointer files slightly differently
  • When working locally with the CLI, it is presumed that your have the LFS client installed, and your pointer files are swapped out.
  • That means if you deploy from the CLI, any assets deployed can't take advantage of the transformation api
  • We should warn in the CLI when you deploy a site that has LFS enabled due to this subtle difference.
  • We should still handle LFS pointer files the sale way so that you could, in theory, use this to deploy the same way the go client does in the buildbot.

bcomnes avatar May 02 '19 22:05 bcomnes

So solve the above mentioned edge case we would need to:

  • Swap LFS files during CLI deploys for LFS pointer files
  • Upload new assets that haven't made it into LFS storage

This would be challenging however. Other solutions might exist.

bcomnes avatar May 02 '19 22:05 bcomnes