webhdfs
webhdfs copied to clipboard
Adds Knox Support
We are using WebHDFS through a Knox gateway.
This requires:
- Specifying the gateway and the cluster to the path
- Support for basic auth
Let us know if you have any questions/ feedback about this PR.
Thanks!
I'm sorry for late reply. Are there any other Knox gateway users? I cannot find anyone around me. I'll re-consider to merge if any others adds comments for merge.
About patch, I think that we should change API of initialize
like below, and change username/doas/proxy_address/proxy_port as deprecated:
def initialize(host='localhost', port=50070, username=nil, doas=nil, proxy_address=nil, proxy_port=nil, options={})
# check options[:username], options[:doas], ... , options[:knox_path], ...
# ...
end
Knox is obviously relatively new, but is definitely shaping up to be the standard way to connect to a secure Kerberized cluster. Both Hortonworks and Cloudera now have it available.
Regarding the initialize method this seems like a decent approach. Though it does seem arbitrary what goes in options and what goes as positioned arguments. Ruby's 2.0 named arguments would solve this but guessing you want your library to support as many Ruby versions as possible.