Steve Loughran

sorry, can't help. as well as not doing hdfs, i'm cutting back on all coding/review because of rsi issues

tests in progress

tested, s3 london -Dscale

+ @mukund-thakur @mehakmeet @sunchao @dongjoon-hyun this is not anything anyone has shipped *yet*

afraid i'm taking a break from all non critical PR review; keeping my typing to a minimum. sorry

rebase to/merge in trunk to see if that fixes the build

hadoop 3.3.4 cuts jax.rs from the dependency graph

1. whose s3 client was used for testing here -if the s3a one, which hadoop release? 2. the azure abfs and gcs connectors do async prefetching of the next block,...

> I was working with s3a > Spark 3.2.1 > Hadoop (Hadoop-aws) 3.3.2 > AWS SDK 1.11.655 thanks., that means you are current with all shipping improvments. the main one...

> At this point the bottlenecks in parquet begin to move towards decompression and decoding but IO remains the slowest link in the chain. Latency is the killer; in an...