arroyo
arroyo copied to clipboard
Invalid URL like s3::https://my-custom-s3/my-bucket/key/path
When starting Worker when aws_endpoint is set like s3::https://my-custom-s3/my-bucket/key/path, throws error: thread 'main' panicked at copy-artifacts/src/main.rs:31:26: Failed to download s3::https://my-custom-s3/my-bucket/key/path/pipeline: InvalidUrl
It should be that the aws_endpoint url without port cannot pass the verification.
The S3_ENDPOINT_URL regular expression:
^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+):(?<port>\d+)/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$
should be changed to:
^[sS]3[aA]?::(?<protocol>https?)://(?P<endpoint>[^:/]+)(:(?<port>\d+))?/(?P<bucket>[a-z0-9\-\.]+)(/(?P<key>.+))?$
to support this form of aws_endpoint url.