workers-sdk icon indicating copy to clipboard operation
workers-sdk copied to clipboard

`env.ASSETS.fetch(request)` throws ` TypeError: Failed to parse URL from [object Object]`

Open rkusa opened this issue 2 years ago • 8 comments

With the latest beta and alpha release, when fetching assets via:

env.ASSETS.fetch(request)

I am receiving the following error for each static asset when using the built-in proxy:

Could not serve static asset: TypeError: Failed to parse URL from [object Object]

(env.ASSETS.fetch(request.url) works)

I think this was introduced with #105 and is caused by this line: https://github.com/cloudflare/wrangler2/blob/main/packages/wrangler/src/pages.tsx#L601

The line looks as follows in the transpiled code:

const request = new import_undici.Request(input, init);

When providing a Request (and not a URL as string), undici checks instanceof Request.

I've modified the package to:

+ console.log(input instanceof import_undici.Request);
 const request = new import_undici.Request(input, init);

This logs false, confirming that the Request I am receiving in my onRequest handler is apparently not an instance of Request object undici is expecting.

That is, I believe the error is caused by onRequest and undici using different Request prototypes.

rkusa avatar Dec 23 '21 13:12 rkusa

await env.ASSETS.fetch(url); also returns [object Response3].

A workaround I have found:

const fRes = await env.ASSETS.fetch(url);

return new Response(fRes.body, {
  headers: fRes.headers,
});

@GregBrimble ~~I hope this workaround is not required in actual Cloudflare Pages?~~

Edit: env.ASSETS.fetch(request.url) doesn't work at all on Cloudflare Pages, returns 406 - Not Acceptable HTTP Code.

Another thing that I have noticed not working is .dot-folders/ (maybe files too) are not served. Feature or bug? Although it works locally using wrangler2.

amithm7 avatar Jan 01 '22 09:01 amithm7

Edit: env.ASSETS.fetch(request.url) doesn't work at all on Cloudflare Pages, returns 406 - Not Acceptable HTTP Code.

Can confirm.

wrangler@pagesCloudflare Pages
env.ASSETS.fetch(request)
env.ASSETS.fetch(request.url)

Another thing that I have noticed not working is .dot-folders/ (maybe files too) are not served. Feature or bug? Although it works locally using wrangler2.

Something I'll have to check too, as I am a heavy user of .well-known directories.

rkusa avatar Jan 03 '22 18:01 rkusa

With #186, wrangler@alpha now has the following behavior:

env.ASSETS.fetch('http://fakehost/myasset.jpg') // works
env.ASSETS.fetch(new Request('/myasset.jpg')) // works
env.ASSETS.fetch('/myasset.jpg') // doesn't work

I'll re-open this ticket to track behavior on Cloudflare Pages production, and also to see if env.ASSETS.fetch('/myasset.jpg') should work (I think, probably yes).

Please open a separate issue for dotfiles/directories if you do believe there is a bug there.

GregBrimble avatar Jan 04 '22 15:01 GregBrimble

I needed to fetch my index.html asset.

This doesn't works on Cloudflare Pages:

env.ASSETS.fetch('http://fakehost/')
env.ASSETS.fetch(new Request('/'))
env.ASSETS.fetch(new Request('https://fakehost/'))

It works locally with wrangler@beta, wrangler@pages or wrangler@alpha, but after pushing to Cloudflare Pages it fails with "Error 1101" screen.

Then I changed it to:

env.ASSETS.fetch(new Request('https://fakehost/', request))

where request is incoming context.

Now it started work on Cloudflare Pages. But then I occurred new problem: HTTP 304. On second request Google Chrome adds If-None-Match header which was returned by CF on first request. request contains this header, so env.ASSETS.fetch will return HTTP 304, which is fully expected, but not for my case because I need to modify this asset every time.

So I wonder what exactly mandatory properties here. Now I'm passing only request.cf for new request, and it seems to work on Cloudflare Pages. So, request.cf is mandatory?

Here is my full code which works locally and on Cloudflare Pages:

const assetURL = new URL('/', request.url).toString();
const assetReq = new Request(assetURL, {
    cf: request.cf
});
const asset = await env.ASSETS.fetch(assetReq);

Amaimersion avatar Jan 12 '22 09:01 Amaimersion

We've got an internal ticket filed for improving the handling of env.ASSETS.fetch in production. I think we probably want

env.ASSETS.fetch('http://fakehost/')
env.ASSETS.fetch(new Request('http://fakehost/'))

to both respond correctly. Not sure about env.ASSETS.fetch(new Request('/')) yet—we'll need to think about that a bit more. I'll update this GitHub Issue once a fix is in place.

If we can remove the requirement for the cf part of the Request (thanks for the investigation!), that should simplify your particular problem so you can just fetch the index.html asset normally.

But in the meantime, something like this might be what you're after:

const assetRequest = new Request(request.clone())
assetRequest.headers.delete('if-none-match')
env.ASSETS.fetch(assetRequest)

Sorry it's so cumbersome at the moment. We'll definitely try and improve here.

GregBrimble avatar Jan 12 '22 09:01 GregBrimble

I spent several hours on this issue last night as well. Everything I tried works locally with wrangler@beta, and does not work when pushed to Cloudflare Pages. I tried all of the mentioned variations above but in some cases it's still failing.

I get all kinds of errors, from 406, to 500, and sometimes even flaky behavior fetching the right asset once and failing with 500 on the second and so on.

I really suggest putting on the website for pages functions a proper section on how to use env.ASSETS.fetch(...) with 4-5 example calls (e.g. modifying the request, the response, etc.) since at the moment there is only one example for the Advanced mode.

Also, is there any way to check the logs of failing functions on Pages? Even if it's just the active ones, and there is no persistence of the logs. Since wrangler works with anything locally it's close to impossible to debug what's happening once pushed and I am into a guess-trial-and-error loop.

Thanks :)

lambrospetrou avatar Jan 23 '22 10:01 lambrospetrou

I'd really appreciate a section in the docs about accessing static assets from functions as well. I was able to figure it out with the help of this thread, but it still took hours.

This is what I ended up with (only tested using wrangler pages dev . wrangler@beta version 0.0.16):

This responds with the static asset that would normally be sent if there were no function catching the request.

// functions/[[catchall]].js
export async function onRequest( context ){
  const { request, env } = context
  const asset = await env.ASSETS.fetch(request.url)
  if( asset.status === 200 )
    return asset
  else
      return new Response(asset.status)
}

This responds with hello.html. If the asset is a .html file, the filePath must not include the file extension. If it has a different extension (e.g. .js, .jpg), filePath must include the file extension.

// functions/[[catchall]].js
export async function onRequest( context ){
  const { request, env } = context
  const origin = new URL( request.url ).origin
  const filePath = '/hello'
  const asset = await env.ASSETS.fetch( `${origin}/${filePath}` )
  if( asset.status === 200 )
    return asset
  else
    return new Response(asset.status)
}

rubidot avatar Feb 10 '22 20:02 rubidot

I'm the latest person to get stuck on this dev vs. prod parity) bug for hours before finding this thread.

+1 to documenting env.ASSETS.fetch in the Functions docs beyond the current limited blurb. Especially the failure cases!

In case it's useful to the next person to run into this, after trying the various suggestions here, the only combination that works reliably for me both locally (wrangler 2.0.16) and on Cloudflare is:

export function render(path?: string): PagesFunction {
  const handler: PagesFunction = async ({ env, request }) => {
    if (path == null) {
      return env.ASSETS.fetch(request);
    }
    const url = new URL(path, request.url).toString();
    return env.ASSETS.fetch(
      new Request(url, {
        // @ts-ignore
        cf: request.cf,
      })
    );
  };
  return handler;
}

Used like:

export const onRequestGet = [ifUser(false, redirect('/sessions/new')), render('/campaigns/[id]')];

hunterloftis avatar Jul 08 '22 03:07 hunterloftis

env.ASSETS.fetch() should now be consistent in production and in wrangler pages dev.

It has the typical "fetcher" signature, taking either a Request or a URL string. The URL must be fully formed with a hostname (e.g. http://fakehost/image.png). You can construct one either with a fake hostname (https://fakehost${pathname}) or you can use the incoming request URL (new URL(pathname, request.url).toString().

next() is a function available on the Functions request context object which delegates to the next Functions handler (eventually automatically falling to env.ASSETS.fetch()). You can use this next() function as a shorthand for accessing assets when you are happy for the rest of your Functions to continue executing first.

next() also optionally takes "fetcher" params. You can pass it a relative URL if you wish to redirect your request when it hits the env.ASSETS.fetch(). For example, you can next("/image.png") (no host required).

Note that for both env.ASSETS.fetch("https://fakehost/image.png") and next("/image.png") requests, you may wish to pass through additional incoming request properties (e.g. its headers or its method). This doesn't happen automatically. For example, env.ASSETS.fetch("https://fakehost/image.png", request) or explicitly, env.ASSETS.fetch("https://fakehost/image.png", { method: request.method, headers: request.headers }). Similarly, next("/image.png", request) or next("/image.png", { method: request.method, headers: request.headers }). The headers in particular can offer some optimizations/features such as if-none-match/etag matching.

GregBrimble avatar Oct 25 '22 13:10 GregBrimble

To make the code clearer about what's happening I tend to use the .invalid top level domain to make it clear that the name is never intended to resolve. So I have something like:

        const html = await env.ASSETS.fetch("http://hostname.invalid/index.html")

buckett avatar Apr 11 '23 07:04 buckett