athens
athens copied to clipboard
gomods
Content-Length Support is required to make local pkgsite working
Is your feature request related to a problem? Please describe.
I'm trying to setup internal copy of pkgsite for private modules. It requires go proxy to be set up to get module content. Trying to use Athens for this purpose.
To read modules pkgsite relies on Content-Length header to be delivered by go proxy alongside with actual content.
See GetZipSize function. And its usage in fetch module. Which is quite extensive and intended to protect over oversized zip files.
Describe the solution you'd like
Athens to send Content-Length header in it's responses
Describe alternatives you've considered No ideas yet
Did a quick hack that makes this work: https://github.com/getoutreach/athens/commit/0a01d905e09e54bf439ddc562b1057a7fe11f0bc
@jaredallard great - would you be ok with submitting a PR with that change?
Works ok. The only caveat - from source code it looks a bit memory-consumption worrying as it keeps zip file in memory.
As of temporal solution looks fine and works.
@arschles So sorry, didn't see this. And yeah, my only concern with submitting it is the memory usage. I'll see if I can get something better out by end of the month!
@AlexeyUzhva if memory is a concern, one thing we can do is store the content size in storage when the module is first stored.
@jaredallard sounds good! can you do me a favor and @ me on a PR if/when you submit one?
@AlexeyUzhva if memory is a concern, one thing we can do is store the content size in storage when the module is first stored.
That would be awesome!
Any chance that could get to the release? Any plans to add so?
@AlexeyUzhva @jaredallard so we do have https://github.com/gomods/athens/pull/1681 (which is in the new v0.11.0 release). iirc, in most cases it's done in memory but we could probably use that abstraction to pre-compute the sizes and store them. Would having that feature (which presumably would eliminate an OOM issue) fix something for you?
From what pkgsite does, it looks like they just fail if the size is too big.