webhdfs
webhdfs copied to clipboard
Published
20 hours ago •
kzk
kzk
Error 403 GSSException: No valid credentials provided
when i use webhdfs with kerberos hadoop cluster
[hdfs@nfjd-hadoop02-node27 ~]$ irb
irb(main):001:0> require 'webhdfs'
=> true
irb(main):002:0> require 'gssapi'
=> true
irb(main):003:0> client = WebHDFS::Client.new('nfjd-hadoop02-node169', 50070)
=> #<WebHDFS::Client:0xfd0e5b6 @ssl_ca_file=nil, @ssl_version=nil, @kerberos_keytab=nil, @doas=nil, @retry_times=1, @httpfs_mode=false, @proxy_port=nil, @ssl_verify_mode=nil, @kerberos=false, @ssl=false, @username=nil, @ssl_cert=nil, @http_headers={}, @proxy_address=nil, @retry_known_errors=false, @port=50070, @host="nfjd-hadoop02-node169", @retry_interval=1, @ssl_key=nil>
irb(main):004:0> client.kerberos = true
=> true
irb(main):005:0> client.kerberos_keytab = "/home/hdfs/keytab/hdfs.keytab"
=> "/home/hdfs/keytab/hdfs.keytab"
irb(main):006:0>
irb(main):007:0*
irb(main):008:0* client.list('/user/dengsc')
WebHDFS::IOError: <html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/><title>Error 403 GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)</title></head><body><h2>HTTP ERROR 403</h2><p>Problem accessing /webhdfs/v1/user/dengsc. Reason:<pre> GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body></html>
from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:401:in `request'
from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:275:in `operate_requests'
from /home/hdfs/logstash-5.6.3/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:138:in `list'
from (irb):8:in `evaluate'
from org/jruby/RubyKernel.java:1079:in `eval'
from org/jruby/RubyKernel.java:1479:in `loop'
from org/jruby/RubyKernel.java:1242:in `catch'
from org/jruby/RubyKernel.java:1242:in `catch'
from /home/hdfs/logstash/vendor/jruby/bin/irb:13:in `(root)'
I am facing the same issue. Providing a keytab file shows the same error.
However, if I do a kinit -kt <MYKEYTAB> before I use this gem, everything works fine (even If don't provide the keytab file later)
