stickler icon indicating copy to clipboard operation
stickler copied to clipboard

Add in mirroring proxy.

Open copiousfreetime opened this issue 14 years ago • 14 comments
trafficstars

Add in an ability for a the mirror middleware to also do intermediate proxying when a gem is requested. Just pass through the request and the put the gem that was downloaded into the mirror repo.

copiousfreetime avatar Mar 27 '11 03:03 copiousfreetime

Great idea!

schlick avatar Feb 07 '12 06:02 schlick

This is what I was hoping stickler config --add --server http://localhost:6789 --upstream https://rubygems.org would accomplish.

I'd like to run stickler as a caching proxy, something we do for Java projects using Maven and Archiva.

spilth avatar Aug 16 '12 21:08 spilth

Yes, that is exactly what this feature will be. When I get to implementing it.

copiousfreetime avatar Aug 17 '12 19:08 copiousfreetime

Hi, I also need this feature. Was there any progress with it? Anything that I can help with?

jbarreneche avatar Apr 22 '13 18:04 jbarreneche

Hi @jbarreneche I'll say, I would love to have a pull request submitted for this feature :-). Feel free to work on it. I can't promise that I'll be getting to this in the near future.

copiousfreetime avatar Apr 25 '13 04:04 copiousfreetime

I may take a stab at this.

johnbellone avatar Aug 27 '13 22:08 johnbellone

@johnbellone Have at it!

copiousfreetime avatar Aug 27 '13 22:08 copiousfreetime

Actually the more that I think about this we could probably solve this using Rack::Cache middleware, no? I am not necessarily certain that all gems should be mirrored, and it'd be difficult, especially in a heavily utilized corporate installation, to limit the subset here.

With that said I think I may have a patch later today to get this working. I am still testing and I figured I would bring this up.

johnbellone avatar Aug 28 '13 10:08 johnbellone

Sounds like a plausible solution. Looking forward to seeing what you come up with!

copiousfreetime avatar Aug 28 '13 16:08 copiousfreetime

@johnbellone I'd like to put something that does this into the next version. Do you have something worked on yet, or shall I start on my own implementation. If you have something I would like to see it so that it may influence what the end solution is.

copiousfreetime avatar Sep 19 '13 03:09 copiousfreetime

Yeah, I don't have anything yet but I'd be willing to work on it next week if we want to sketch out some ideas? I can probably throw together a Rack::Cache middleware example in about an hour or two. We actually were originally looking to proxy communication to the upstream to grab the gem, but now the more we go down the road we actually would like to utilize the existing mechanism to greenlight individual ones.

johnbellone avatar Sep 19 '13 12:09 johnbellone

I have an idea how to implement this. I'll test it out tonight or later this week. @copiousfreetime When do you need it by?

johnbellone avatar Sep 24 '13 21:09 johnbellone

What's the expected behavior if there is more than one upstream?

johnbellone avatar Sep 25 '13 00:09 johnbellone

@johnbellone A week or so is what I'm thinking. As for more than one upstream? I would hope that it wouldn't matter and that the same version of a gem would be the same bytewise no matter what upstream.

Each upstream should at least be a different mirror repository so that we know what gems came from which upstream.

Can you explain the scenario you are considering in more detail?

copiousfreetime avatar Sep 25 '13 16:09 copiousfreetime