seatunnel
seatunnel copied to clipboard
apache
[Bug] [HdfsFile] HdfsFile sink connector will throw "SIMPLE authentication is not enabled"
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
When running HdfsFile sink, the following authentication error is occurring.
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
SeaTunnel Version
2.3.4
SeaTunnel Config
env {
execution.parallelism = 2
job.mode = "BATCH"
checkpoint.interval = 10000
}
source {
Jdbc {
url = "jdbc:mysql://10.123.123.123:13306/db_0"
driver = "com.mysql.cj.jdbc.Driver"
user = "user"
password = "password"
query = "select * from test_table limit 5"
}
}
sink {
HdfsFile {
path = "/user/abcd/local/test/raw"
file_format_type = "orc"
fs.defaultFS = "hdfs://pct"
kerberos_principal = "[email protected]"
kerberos_keytab_path = "/c3s_keytabs/c3s.abcd.keytab"
compress_codec = "lz4"
hdfs_site_path = "/etc/hadoop/conf/hdfs-site.xml"
}
}
Running Command
/opt/seatunnel/bin/seatunnel.sh --config /data/seatunnel-batch.conf
Error Exception
java.lang.RuntimeException: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[5b6f0e0b878c2c04e54fcafb, ALL, [{"rating": 0.026002461, "target": "60daf34b41f0eedb360e96b3"}, {"rating": 0.025383532, "target": "5d75af3c3f02311115c77f6b"}, {"rating": 0.024924304, "target": "61af06f841f0ee705ff188bb"}, {"rating": 0.024292462, "target": "6193520f5b43c247a470612b"}, {"rating": 0.024183467, "target": "5fb09981f6b2e40b19a2912a"}, {"rating": 0.023957543, "target": "61f7eb9a5b43c228e2b5ac13"}, {"rating": 0.023779351, "target": "5bfbd479a8c16addea476775"}, {"rating": 0.023599662, "target": "61db9fb1f8bbe81253ce7df4"}, {"rating": 0.023265313, "target": "5f360d7974231e0f3505b1ae"}, {"rating": 0.0232619, "target": "6306184231d8fe0fee7bd7d6"}, {"rating": 0.022837332, "target": "5bdd58caef98f463fea46fde"}, {"rating": 0.022483498, "target": "5c1bab7eb7ad8dd60a3e2ecb"}, {"rating": 0.022443114, "target": "619dbe7b734c7728e3c286a5"}, {"rating": 0.022110818, "target": "630b7709f8bbe897ed32fd6e"}, {"rating": 0.021631876, "target": "5f47113241f0eee65c43f72e"}, {"rating": 0.019302145, "target": "600beab1d568eda0dccc47bd"}, {"rating": 0.018735824, "target": "5f47a2945b43c2ea85acf4b3"}, {"rating": 0.0169863, "target": "6032499dd568ed7185befdb2"}, {"rating": 0.016443873, "target": "5ea0dfa23bf76e143eb50ccf"}, {"rating": 0.016276632, "target": "5f9e855cf6b2e4b98de9e5a1"}, {"rating": 0.016219616, "target": "5c015fb377e686a374a4fe03"}, {"rating": 0.016104022, "target": "621fd3a08ca3be644ec90687"}, {"rating": 0.015920438, "target": "62316f7a41f0eedb8b233c7d"}, {"rating": 0.015685346, "target": "61a5791a8d897ecfc4a7bca6"}, {"rating": 0.014869792, "target": "616244c641f0ee145c23d644"}, {"rating": 0.014427128, "target": "61068450734c77758faddfc6"}, {"rating": 0.014221562, "target": "637192ddf6b2e4b98b8b9840"}, {"rating": 0.01381126, "target": "5dd96c723020783885b64d3f"}, {"rating": 0.013561504, "target": "62234d57734c77a00cb3f42d"}, {"rating": 0.0135063175, "target": "6049e8005b43c225ab713362"}, {"rating": 0.013452409, "target": "604d940fd568ed37b92925d0"}, {"rating": 0.0134058865, "target": "63969e6d55ad5241287d4ce2"}, {"rating": 0.012967562, "target": "5daf100074231e4aaca42198"}, {"rating": 0.01286277, "target": "5fa118da734c7727af6482cf"}, {"rating": 0.012862502, "target": "6381ea8bcc799c4b970b4062"}, {"rating": 0.012695316, "target": "5bb1c9ef1f6786ca8ce289e1"}, {"rating": 0.012659408, "target": "5c2ff46db7ad8dd60ab3b608"}, {"rating": 0.012576638, "target": "5f7fc82331acb3697c34e4bb"}, {"rating": 0.012524918, "target": "61b6329cf6b2e4452683c8de"}, {"rating": 0.012303311, "target": "5deb44d353e4a91bd26478e5"}, {"rating": 0.012303206, "target": "5cea81733f02310be5be6421"}, {"rating": 0.012217641, "target": "5e7ae99da049c40134f7b94a"}, {"rating": 0.012123918, "target": "5de9131274231e8e73f19a45"}, {"rating": 0.012087173, "target": "5f61cc8bf6b2e479ec4d900b"}, {"rating": 0.012013882, "target": "6145fee031d8fe4b5510d35b"}, {"rating": 0.01200705, "target": "61ae87e431d8fea4417d7067"}, {"rating": 0.011979027, "target": "614d6602f6b2e4234ca6b09d"}, {"rating": 0.011943467, "target": "61debc5431d8fea619bd754d"}, {"rating": 0.01187698, "target": "616325b6734c771b08c48ecd"}, {"rating": 0.011742279, "target": "600e8ddb5b43c228b900735e"}, {"rating": 0.011716185, "target": "5c4345d600eeaf33e39f803e"}, {"rating": 0.011671308, "target": "616bf88c8d897eb7e1308b4b"}, {"rating": 0.0115066655, "target": "5e0f1c77d3da7a813b8db8f0"}, {"rating": 0.011225588, "target": "5e1a6cc374231e7936cdcb1b"}, {"rating": 0.011217727, "target": "5e9aa8643020781c59781280"}, {"rating": 0.01118096, "target": "616680498d897eb7e15aecd4"}, {"rating": 0.011168212, "target": "6187ae668d897eda4f7f5479"}, {"rating": 0.011156079, "target": "6251a1f98ca3be72ef96a898"}, {"rating": 0.011105674, "target": "628892a5f6b2e429c7d922e6"}, {"rating": 0.011083827, "target": "5f8793e95b43c21e7d06295b"}, {"rating": 0.011004955, "target": "62d13875857d0f9531680e7f"}, {"rating": 0.010825905, "target": "60f8f22a31d8fe68b6698479"}, {"rating": 0.010813702, "target": "5f30abe053e4a9525929d73e"}, {"rating": 0.010810402, "target": "5dbd404fa049c47079c5d12f"}, {"rating": 0.010690353, "target": "61a628b3f6b2e4fbe639f43a"}, {"rating": 0.010673437, "target": "5be7ad2b0baf81b5640af958"}, {"rating": 0.010657913, "target": "60af5edf5b43c27de71060e9"}, {"rating": 0.010627309, "target": "635c36205b43c2d62ae83a46"}, {"rating": 0.010479885, "target": "62501b2e341c0143eda8d21a"}, {"rating": 0.0103837475, "target": "60a652ddf6b2e4d8b03e8185"}, {"rating": 0.010200929, "target": "5dafed58278d305038b9b8b1"}, {"rating": 0.010184529, "target": "5defb1237eacfbd79c00a1cb"}, {"rating": 0.010164691, "target": "5ff95c9bf6b2e4090ecc564a"}, {"rating": 0.010158137, "target": "60a5a995f6b2e4c0becab615"}, {"rating": 0.010060087, "target": "5eb76d4a278d3087871cea48"}, {"rating": 0.010036934, "target": "6198b3dcf6b2e4ac8e398847"}, {"rating": 0.010028836, "target": "60a77a59734c7752b5669881"}, {"rating": 0.010003384, "target": "60998b23734c770a34a43b6b"}, {"rating": 0.009963704, "target": "60559c54f6b2e432b8d94e91"}, {"rating": 0.009960496, "target": "60a3d57541f0eedc7ed55216"}, {"rating": 0.009902597, "target": "6157221d8d897e62af6d8bf7"}, {"rating": 0.009897488, "target": "61a837e4734c777f3a56a60d"}, {"rating": 0.009853107, "target": "5f1ec17553e4a90ef0a6889f"}, {"rating": 0.009790014, "target": "5fc203745b43c27113913832"}, {"rating": 0.009789292, "target": "62ade471734c770e1f12f26a"}, {"rating": 0.00975097, "target": "5ec267487eacfb957062c104"}, {"rating": 0.009725538, "target": "5e5123e07eacfba162eedf91"}, {"rating": 0.009713324, "target": "5c0d3067d154a5c8398f35c2"}, {"rating": 0.009701602, "target": "5e63d5417eacfb7ea49a3ff5"}, {"rating": 0.009691264, "target": "6294690ef6b2e41818dac987"}, {"rating": 0.009635303, "target": "62728ec341f0ee35d3915d11"}, {"rating": 0.009610111, "target": "604e6484734c77762b118afc"}, {"rating": 0.0095940735, "target": "5e06ef1c3f0231475a648fa6"}, {"rating": 0.009583052, "target": "637d21fc951c8e3261dd6c16"}, {"rating": 0.009576711, "target": "624e46355b43c2f8190cf3bd"}, {"rating": 0.009536556, "target": "631d4b4af8bbe8fdb2b8b551"}, {"rating": 0.009426424, "target": "5c0290d977e686a374c829e9"}, {"rating": 0.009420803, "target": "6190928941f0ee271cde7bc3"}, {"rating": 0.009413466, "target": "5fbdc12d55ad52a8001ec272"}, {"rating": 0.009411947, "target": "61ec0a4b857d0f381cd03eba"}], 2023-01-22T03:26:01, 2023-01-22T03:26:01]}'.]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:257) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:66) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:643) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:944) ~[seatunnel-starter.jar:2.3.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_392]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_392]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_392]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_392]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_392]
Caused by: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[5b6f0e0b878c2c04e54fcafb, ALL, [{"rating": 0.026002461, "target": "60daf34b41f0eedb360e96b3"}, {"rating": 0.025383532, "target": "5d75af3c3f02311115c77f6b"}, {"rating": 0.024924304, "target": "61af06f841f0ee705ff188bb"}, {"rating": 0.024292462, "target": "6193520f5b43c247a470612b"}, {"rating": 0.024183467, "target": "5fb09981f6b2e40b19a2912a"}, {"rating": 0.023957543, "target": "61f7eb9a5b43c228e2b5ac13"}, {"rating": 0.023779351, "target": "5bfbd479a8c16addea476775"}, {"rating": 0.023599662, "target": "61db9fb1f8bbe81253ce7df4"}, {"rating": 0.023265313, "target": "5f360d7974231e0f3505b1ae"}, {"rating": 0.0232619, "target": "6306184231d8fe0fee7bd7d6"}, {"rating": 0.022837332, "target": "5bdd58caef98f463fea46fde"}, {"rating": 0.022483498, "target": "5c1bab7eb7ad8dd60a3e2ecb"}, {"rating": 0.022443114, "target": "619dbe7b734c7728e3c286a5"}, {"rating": 0.022110818, "target": "630b7709f8bbe897ed32fd6e"}, {"rating": 0.021631876, "target": "5f47113241f0eee65c43f72e"}, {"rating": 0.019302145, "target": "600beab1d568eda0dccc47bd"}, {"rating": 0.018735824, "target": "5f47a2945b43c2ea85acf4b3"}, {"rating": 0.0169863, "target": "6032499dd568ed7185befdb2"}, {"rating": 0.016443873, "target": "5ea0dfa23bf76e143eb50ccf"}, {"rating": 0.016276632, "target": "5f9e855cf6b2e4b98de9e5a1"}, {"rating": 0.016219616, "target": "5c015fb377e686a374a4fe03"}, {"rating": 0.016104022, "target": "621fd3a08ca3be644ec90687"}, {"rating": 0.015920438, "target": "62316f7a41f0eedb8b233c7d"}, {"rating": 0.015685346, "target": "61a5791a8d897ecfc4a7bca6"}, {"rating": 0.014869792, "target": "616244c641f0ee145c23d644"}, {"rating": 0.014427128, "target": "61068450734c77758faddfc6"}, {"rating": 0.014221562, "target": "637192ddf6b2e4b98b8b9840"}, {"rating": 0.01381126, "target": "5dd96c723020783885b64d3f"}, {"rating": 0.013561504, "target": "62234d57734c77a00cb3f42d"}, {"rating": 0.0135063175, "target": "6049e8005b43c225ab713362"}, {"rating": 0.013452409, "target": "604d940fd568ed37b92925d0"}, {"rating": 0.0134058865, "target": "63969e6d55ad5241287d4ce2"}, {"rating": 0.012967562, "target": "5daf100074231e4aaca42198"}, {"rating": 0.01286277, "target": "5fa118da734c7727af6482cf"}, {"rating": 0.012862502, "target": "6381ea8bcc799c4b970b4062"}, {"rating": 0.012695316, "target": "5bb1c9ef1f6786ca8ce289e1"}, {"rating": 0.012659408, "target": "5c2ff46db7ad8dd60ab3b608"}, {"rating": 0.012576638, "target": "5f7fc82331acb3697c34e4bb"}, {"rating": 0.012524918, "target": "61b6329cf6b2e4452683c8de"}, {"rating": 0.012303311, "target": "5deb44d353e4a91bd26478e5"}, {"rating": 0.012303206, "target": "5cea81733f02310be5be6421"}, {"rating": 0.012217641, "target": "5e7ae99da049c40134f7b94a"}, {"rating": 0.012123918, "target": "5de9131274231e8e73f19a45"}, {"rating": 0.012087173, "target": "5f61cc8bf6b2e479ec4d900b"}, {"rating": 0.012013882, "target": "6145fee031d8fe4b5510d35b"}, {"rating": 0.01200705, "target": "61ae87e431d8fea4417d7067"}, {"rating": 0.011979027, "target": "614d6602f6b2e4234ca6b09d"}, {"rating": 0.011943467, "target": "61debc5431d8fea619bd754d"}, {"rating": 0.01187698, "target": "616325b6734c771b08c48ecd"}, {"rating": 0.011742279, "target": "600e8ddb5b43c228b900735e"}, {"rating": 0.011716185, "target": "5c4345d600eeaf33e39f803e"}, {"rating": 0.011671308, "target": "616bf88c8d897eb7e1308b4b"}, {"rating": 0.0115066655, "target": "5e0f1c77d3da7a813b8db8f0"}, {"rating": 0.011225588, "target": "5e1a6cc374231e7936cdcb1b"}, {"rating": 0.011217727, "target": "5e9aa8643020781c59781280"}, {"rating": 0.01118096, "target": "616680498d897eb7e15aecd4"}, {"rating": 0.011168212, "target": "6187ae668d897eda4f7f5479"}, {"rating": 0.011156079, "target": "6251a1f98ca3be72ef96a898"}, {"rating": 0.011105674, "target": "628892a5f6b2e429c7d922e6"}, {"rating": 0.011083827, "target": "5f8793e95b43c21e7d06295b"}, {"rating": 0.011004955, "target": "62d13875857d0f9531680e7f"}, {"rating": 0.010825905, "target": "60f8f22a31d8fe68b6698479"}, {"rating": 0.010813702, "target": "5f30abe053e4a9525929d73e"}, {"rating": 0.010810402, "target": "5dbd404fa049c47079c5d12f"}, {"rating": 0.010690353, "target": "61a628b3f6b2e4fbe639f43a"}, {"rating": 0.010673437, "target": "5be7ad2b0baf81b5640af958"}, {"rating": 0.010657913, "target": "60af5edf5b43c27de71060e9"}, {"rating": 0.010627309, "target": "635c36205b43c2d62ae83a46"}, {"rating": 0.010479885, "target": "62501b2e341c0143eda8d21a"}, {"rating": 0.0103837475, "target": "60a652ddf6b2e4d8b03e8185"}, {"rating": 0.010200929, "target": "5dafed58278d305038b9b8b1"}, {"rating": 0.010184529, "target": "5defb1237eacfbd79c00a1cb"}, {"rating": 0.010164691, "target": "5ff95c9bf6b2e4090ecc564a"}, {"rating": 0.010158137, "target": "60a5a995f6b2e4c0becab615"}, {"rating": 0.010060087, "target": "5eb76d4a278d3087871cea48"}, {"rating": 0.010036934, "target": "6198b3dcf6b2e4ac8e398847"}, {"rating": 0.010028836, "target": "60a77a59734c7752b5669881"}, {"rating": 0.010003384, "target": "60998b23734c770a34a43b6b"}, {"rating": 0.009963704, "target": "60559c54f6b2e432b8d94e91"}, {"rating": 0.009960496, "target": "60a3d57541f0eedc7ed55216"}, {"rating": 0.009902597, "target": "6157221d8d897e62af6d8bf7"}, {"rating": 0.009897488, "target": "61a837e4734c777f3a56a60d"}, {"rating": 0.009853107, "target": "5f1ec17553e4a90ef0a6889f"}, {"rating": 0.009790014, "target": "5fc203745b43c27113913832"}, {"rating": 0.009789292, "target": "62ade471734c770e1f12f26a"}, {"rating": 0.00975097, "target": "5ec267487eacfb957062c104"}, {"rating": 0.009725538, "target": "5e5123e07eacfba162eedf91"}, {"rating": 0.009713324, "target": "5c0d3067d154a5c8398f35c2"}, {"rating": 0.009701602, "target": "5e63d5417eacfb7ea49a3ff5"}, {"rating": 0.009691264, "target": "6294690ef6b2e41818dac987"}, {"rating": 0.009635303, "target": "62728ec341f0ee35d3915d11"}, {"rating": 0.009610111, "target": "604e6484734c77762b118afc"}, {"rating": 0.0095940735, "target": "5e06ef1c3f0231475a648fa6"}, {"rating": 0.009583052, "target": "637d21fc951c8e3261dd6c16"}, {"rating": 0.009576711, "target": "624e46355b43c2f8190cf3bd"}, {"rating": 0.009536556, "target": "631d4b4af8bbe8fdb2b8b551"}, {"rating": 0.009426424, "target": "5c0290d977e686a374c829e9"}, {"rating": 0.009420803, "target": "6190928941f0ee271cde7bc3"}, {"rating": 0.009413466, "target": "5fbdc12d55ad52a8001ec272"}, {"rating": 0.009411947, "target": "61ec0a4b857d0f381cd03eba"}], 2023-01-22T03:26:01, 2023-01-22T03:26:01]}'.]
at org.apache.seatunnel.common.exception.CommonError.writeSeaTunnelRowFailed(CommonError.java:83) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:136) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-11], ErrorDescription:[Sink writer operation failed, such as (open, close) etc...] - Get orc writer for file [/tmp/seatunnel/seatunnel/822661860468719617/6c58715d18/T_822661860468719617_6c58715d18_0_1/NON_PARTITION/T_822661860468719617_6c58715d18_0_1_0.lz4.orc] error
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:131) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_392]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_392]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_392]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_392]
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:281) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1217) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.seatunnel.common.exception.CommonError.writeSeaTunnelRowFailed(CommonError.java:83) ~[seatunnel-starter.jar:2.3.4]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:136) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-11], ErrorDescription:[Sink writer operation failed, such as (open, close) etc...] - Get orc writer for file [/tmp/seatunnel/seatunnel/822661860468719617/6c58715d18/T_822661860468719617_6c58715d18_0_1/NON_PARTITION/T_822661860468719617_6c58715d18_0_1_0.lz4.orc] error
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:131) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_392]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_392]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_392]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_392]
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:281) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1217) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:532) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:529) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:543) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:470) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1118) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1098) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.impl.PhysicalFsWriter.<init>(PhysicalFsWriter.java:95) ~[?:?]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:177) ~[?:?]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.OrcFile.createWriter(OrcFile.java:861) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:126) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Caused by: org.apache.hadoop.ipc.RemoteException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1562) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.Client.call(Client.java:1508) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.Client.call(Client.java:1405) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at com.sun.proxy.$Proxy34.create(Unknown Source) ~[?:?]
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:366) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_392]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_392]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_392]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_392]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at com.sun.proxy.$Proxy35.create(Unknown Source) ~[?:?]
at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:276) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1217) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:532) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:529) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:543) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:470) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1118) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1098) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.4]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.impl.PhysicalFsWriter.<init>(PhysicalFsWriter.java:95) ~[?:?]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:177) ~[?:?]
at org.apache.seatunnel.shade.connector.file.org.apache.orc.OrcFile.createWriter(OrcFile.java:861) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:126) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ~[?:?]
at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) ~[?:?]
at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ~[seatunnel-starter.jar:2.3.4]
... 16 more
Zeta or Flink or Spark Version
Zeta
Java or Scala Version
1.8
Screenshots
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
