Exclude files from the manifest

[ericselin]

💡 Feature request

Currently (and as stated in the docs, Workers Sites do not work for sites with a large amount of pages. The limiting factor is the number of pages (/files) that you have, since those get included in the manifest and count toward the script size limitation of 1MB. It's not even particularly hard for a moderately sized ecommerce store to hit this limit (speaking from personal experience).

If we could exclude files from the manifest (based on glob or similar), this issue would be mitigated considerably. These files would be directly retrieved from the KV store based on path alone, but this is fine for assets that already have a hash in their filename.

Describe the feature

Some uploaded files (such as images) might already be hashed, and thus don't need hashing again by Wrangler. These files could be excluded from the manifest, making the resulting Worker script size smaller.

Concrete example: we're building our ecommerce stores with hugo, which already adds a hash to resized images filenames. Since we create many variants of each image, the number of files quickly grows (think 3 variants of 5 product pictures for 100 products = 1,500 images). Currently the manifest becomes too large with all those images, but if we could exclude all images from the manifest, we could use Worker Sites.

Describe the alternatives

The alternative of course is to use a separate service like Netlify or Cloudflare Pages for hosting the site, and use Workers only for HTML rewrites / authentication etc. This, however, is not as fast as using Worker Sites according to my tests. It also adds architectural complexity with yet another service to run.

The other solution would be to create a custom "Wrangler" of sorts with the KV and Worker APIs. Not impossible, but seems pretty inefficient.

[adamchal]

Thanks for posting this FR @ericselin. This would be a great feature. I’m often in a situation where my hashed assets are just re-hashed for no reason and adding to the bulk of the script __STATIC_CONTENT_MANIFEST. I’ve considered writing a custom solution, but that feels like a mistake. Wrangler does a really nice job of uploading/sync’ing and reinventing all of that for this workaround doesn’t seem prudent. If my Rust game was hot, I would be working on a PR right now.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity in the last 180 days. It will be closed if no further activity occurs in the next week. Please feel free to comment if you'd like it to remain open, and thank you for your contributions.

[penalosa]

Thanks for suggesting this feature! Wrangler v1 is now deprecated and support is only being provided for critical updates or security concerns. As such, we are closing this issue.

New versions of Wrangler are maintained in the workers-sdk repo. If this is a feature you’d like to see in wrangler v2+, please feel free to report it in the workers-sdk repo. For more info about wrangler v1 deprecation, please check out our blog post.

We recommend using Cloudflare Pages for static sites going forward!