Aditya Goenka
Aditya Goenka
cc @danny0405
@shubhamn21 Did you tried setting `fs.s3a.connection.maximum` to a higher value ?
@shubhamn21 Were you able to resolve this issue?
@shubhamn21 Sorry for the delay here. After increasing the value of `fs.s3a.connection.maximum` also, did you faced the same exception? We also wanted to understand the root cause for this. By...
@shubhamn21 Thanks for the update. Yes, having multiple writer without lock provider can cause inconsistent behaviour and create this kind of issues.
@JoshuaZhuCN Do you have any idea/design how can that be supported or have any thoughts in your mind? Adding @nsivabalan @yihua @xushiyan @danny0405 @codope
@zyclove can you let us know more about your data size ( existing and incremental) , table and writer configuration.
@khajaasmath786 Just to understand why it happened, Can you confirm if multiple writers are writing to same table concurrently without the lock provider configurations?
@khajaasmath786 Any updates on this?
@zyclove Have you started seeing this issue after hudi upgrade only? Not sure if it's related to hudi.