Make it work with Hadoop2

[ykk1431]

Hi ,

Any plan to make it work with Hadoop2. I am using cdh4.2 with MR1 and white-elephant is not working for me and i think the issue is during parsing logs. I wish i am a developer to fix this myself.

-Thanks

[matthayes]

Yea we will add support for Hadoop2 at some point. What is your issue with MR1 though? Can you include any error messages you're seeing?

[bilsch]

Just curious, saying it does not work on hadoop2 - does that mean no support for yarn mode but running under mr1 mode is supported/ok? ( since its the same api, I would have thought this to be the case )

[matthayes]

The log parsing jobs are written using MR1, which I presume is still available. So you're right that if the same api is available it should be possible to run the jobs. I suspect there may be a problem parsing the logs though. I think in Hadoop 2 with yarn the log format may have changed. They may not even be stored the same way. I haven't looked into this but this is what I heard.

On Mon, Nov 11, 2013 at 6:07 PM, Bill Schwanitz [email protected]:

Just curious, saying it does not work on hadoop2 - does that mean no support for yarn mode but running under mr1 mode is supported/ok? ( since its the same api, I would have thought this to be the case )

— Reply to this email directly or view it on GitHubhttps://github.com/linkedin/white-elephant/issues/15#issuecomment-28262077 .