fsqio icon indicating copy to clipboard operation
fsqio copied to clipboard

hadoop-lzo jar resolution

Open jglesner opened this issue 5 years ago • 4 comments

Hello,

I'm running into an issue using the latest fsqio master where ivy is consistently unable to resolve hadoop-lzo and fails to run serve.py.

[NOT FOUND ] com.hadoop.gplcompression#hadoop-lzo;0.4.19!hadoop-lzo.jar (0ms)
==== maven.twttr.com-maven: tried
https://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.19/hadoop-lzo-0.4.19.jar

I'm not all too familiar with the dependency management aspects of twofishes. I'm running this from an ubuntu 14.04 docker container, and everything has worked historically ... until now. Any recommendations on how to figure out what's going on / solve for this? I would greatly appreciate it.

Strangely, when I git clone the fsqio repo, and run serve.py from the command line on an Amazon Linux 1 EC2 instance it works perfectly. No dependency issues. I need to run it from a docker container, though preferably.

jglesner avatar Nov 11 '19 12:11 jglesner

Not sure if this is the correct fix, but I was able to get past this issue by adding a new global exclusion to Ivy by editing the pants.ini file under [resolve.ivy]: {"org": "com.hadoop.gplcompression", "name": "hadoop-lzo"},

jglesner avatar Nov 12 '19 12:11 jglesner

I think that your fix is likely the right answer. We are getting ready to update fsq.io, would you be interested in sending a PR?

mateor avatar Nov 14 '19 19:11 mateor

Sure, I can submit it tonight (if that fits with your timeline).

jglesner avatar Nov 14 '19 19:11 jglesner

That sounds great! The update is addressing unrelated bug in our dependencies, so we may need to land that first in order to get the build green anyway

mateor avatar Nov 14 '19 19:11 mateor