Improved experience when remote object store URL does not end in /

[xiedeyantu]

Which issue does this PR close?

• Closes #16302

Rationale for this change

It would be automatically add a / to the path if the first one was not found and try again.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[xiedeyantu]

@alamb Could you help reivew this PR?

[alamb]

@alamb Could you help reivew this PR?

Thanks @xiedeyantu ! We normally need to add tests as part of any code PR -- could you look into adding some tests and documentation about how this change works?

[xiedeyantu]

@alamb Could you help reivew this PR?

Thanks @xiedeyantu ! We normally need to add tests as part of any code PR -- could you look into adding some tests and documentation about how this change works?

@alamb Is there any example I can refer to for testing involving s3, because real access requires ak/sk and the real path?

[alamb]

@alamb Could you help reivew this PR?

Thanks @xiedeyantu ! We normally need to add tests as part of any code PR -- could you look into adding some tests and documentation about how this change works?

@alamb Is there any example I can refer to for testing involving s3, because real access requires ak/sk and the real path?

I think you can follow the one here: https://datafusion.apache.org/user-guide/cli/datasources.html#s3

To read from a public S3 bucket without signatures, use the aws.SKIP_SIGNATURE option:

CREATE EXTERNAL TABLE nyc_taxi_rides
STORED AS PARQUET LOCATION 's3://altinity-clickhouse-data/nyc_taxi_rides/data/tripdata_parquet/'
OPTIONS(aws.SKIP_SIGNATURE true);

[alamb]

@blaginin also did an end to end test with S3 in the CI tests. The instructions are here:

• https://github.com/apache/datafusion/blob/main/datafusion-cli/CONTRIBUTING.md#L30-L29

[github-actions[bot]]

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.