Fetch API is inconsistent with the browser/web API for URL starting with whitespaces

[SamyPesse]

trafficstars

I'm opening this issue to report a difference in implementation between fetch in the browser and in Cloudflare for an edge case. We are not really expecting a fix, as I'm not sure the browser is doing the right thing there; but I've suggested one at the end.

If you fetch a URL that starts with a whitespace, like:

const res = await fetch(' https://raw.githubusercontent.com/cloudflare/workerd/main/README.md');
console.log(await res.text());

It will throw an error TypeError: Fetch API cannot load in Cloudflare Workers, while this code will work in the browser (at least Chrome).

We encountered a situation in which we needed to fetch a user-inputted URL, but our validation for the URL was not great.

A workaround we implemented was to wrap the url in a new URL(): fetch(new URL(' https://raw.githubusercontent.com/cloudflare/workerd/main/README.md')) works in Cloudflare.

A solution in workerd could be to normalize / trim these urls in the fetch function, the same way new URL is doing it.

[jasnell]

Currently, our implementation of fetch(...) does not use the whatwg URL parsing rules for backwards compatibility reasons. It's been on my todo list for a while to get that fixed but it will require a compatibility flag as it'll be a breaking change. Will see about upping the priority of making that change

/cc @irvinebroque @kentonv