feat(optimizer): holdUntilCrawlEnd option

[patak-dev]

Description

After Vite 3.0, our cold start strategy changed as described in https://github.com/vitejs/vite/pull/8869

An issue of this strategy is that for the default non-https server, the browser can only request 6 concurrent requests. Each optimized dep found takes one of these requests, so requests could stall until the crawling of static deps is finished in the server. For large projects, this means that the browser is potentially idle when it could be downloading and parsing requests. We did this to avoid full-page reloads when the scanner misses a dependency and forces a full reload. But we are paying a cost in all projects where all deps are discovered by the scanner or manually included.

This PR adds a new option optimizeDeps.holdUntilCrawlEnd. If true (the default), it uses the current strategy. If false, it releases to the browser the optimized deps computed after the scanner as soon as they are available. We avoid the possibility of the browser being idle while the server crawls static imports, with the trade-off that if dependencies are missed by the scanner, there could be a full-page reload.

There are some differences with a similar strategy we had in Vite 2:

• We have an optimization that avoids the need for the reload if the new dependencies don't generate new common chunks (from v3)
• The scanner and optimized deps are run in parallel to the processing (from v3)
• We have new warnings to guide the user to add the missed deps to optimizeDeps.include and avoid full-page reloads.

We discussed these strategies with @bluwy and @ArnaudBarre, and this PR could be a first step to change the default strategy for cold start to holdUntilCrawlEnd: false in a future minor, depending on the feedback we get from users.

---

What is the purpose of this pull request?

• [ ] Bug fix
• [x] New Feature
• [ ] Documentation update
• [ ] Other

[stackblitz[bot]]

Run & review this pull request in StackBlitz Codeflow.

[patak-dev]

I tried to understand the optimizer flow, but I don't quite grasp it 😅 Other than that, I tested locally and it seems to work well though

In a future minor (so we get some time for testing it in a beta), we should start from scratch and move all this code to a state machine. It ended up being quite hard to follow after all the accumulated changes.

[ArnaudBarre]

(I have the review of this one on my todo, I will review it this weekend I think)