arroyo icon indicating copy to clipboard operation
arroyo copied to clipboard

Invalid URL like s3::https://my-custom-s3/my-bucket/key/path

Open huaxk opened this issue 11 months ago • 0 comments

When starting Worker when aws_endpoint is set like s3::https://my-custom-s3/my-bucket/key/path, throws error: thread 'main' panicked at copy-artifacts/src/main.rs:31:26: Failed to download s3::https://my-custom-s3/my-bucket/key/path/pipeline: InvalidUrl

It should be that the aws_endpoint url without port cannot pass the verification.

The S3_ENDPOINT_URL regular expression: ^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+):(?<port>\d+)/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$ should be changed to: ^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+)(:(?<port>\d+))?/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$ to support this form of aws_endpoint url.

huaxk avatar Mar 15 '24 08:03 huaxk