Support resuming a complete HTML prerender that has dynamic flight data

[gnoff]

followup to: https://github.com/vercel/next.js/pull/60645

Background

When prerendering the determination of whether a prerender is fully static or partially static should not be directly related to whether there is a postponed state or not. When rendering RSC it is possible to postpone because a dynamic API was used but then on the client (SSR) the postpone is never encountered. This can happen when a server component is passed to a client component and the client component conditionally renders the server component.

Today if this happens the entire output would be considered static when in fact the flight data encoded into the page and used for bootstrapping the client router contains dynamic holes. Today this is blocked by an error that incorrectly assumes that this case means the user caught the postpone in the client layer but as shown above this may not be the case.

Implementation

A more capable model is to think of the outcome of a prerender as having 3 possible states

1. Dynamic HTML: The HTML produces by the prerender has dynamic holes. we save the static prelude but expect to resume the render later to complete the HTML. This means we will resume the RSC part of the render as well
2. Dynamic Data: The HTML is completely static but the RSC data encoded into the page is dynamic. We don't want to resume the render but we do need to produce new inlined RSC data during a Request.
3. Static: The HTML is completely static and so is the RSC data encoded into the page. We save the entire HTML output and there will be no dynamic continuation when this route is visited.

Really 1 & 3 are the same as today (Partially static & Fully Static respectively) but case 2 which today errors in a confusing way is now supported.

In addition implementing the Dynamic Data case the old warning about catching postpones is removed. The reason we don't want this is that catching postpones is potentially a valid way to do optimistic UI. We probably want a first-party API for it at some point (and maybe we'll add the warning back in once we do) but imagine you do something dynamic like look up a user but during prerender you want to render as if the user is logged out. you could call getUser() in a try catch and render fallback UI if it throws. In this case we'd detect a dynamic API was used but we wouldn't have a corresponding postpone state which would put us in the Dynamic Data case (2).

Technical Note

Another note about the implementation is that you'll see that regardless of which case we are in, if there is a postponed state but we consider the page to be Dynamic Data meaning we want to serialize all the HTML and NOT do a resume in the dynamic continuation then we immediately resume the render with and already aborted AbortSignal. The purpose here is to mark any boundaries which have dynamic holes as being client-rendered.

As a general rule if the render produces a postponed state we must do one of the following

1. save the postponed state and ensure there is a dynamic continuation that calls resume
2. immediately resume the render and save the concatenated output and ensure the dynamic continuation does NOT call resume.

or said another way, every postponed state must be resumed (even if it didn't come from Next's dynamic APIs)

Perf considerations

This PR modifies a few key areas to improve perf.

Reduces quantity of *Stream instances where possible as these add significant overhead Reduces extra closures to lower allocations and keep functions in monomorphic form where possible

Closes NEXT-2164

[ijjk]

Tests Passed

[ijjk]

Stats from current PR

Default Build

General Overall increase ⚠️

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
buildDuration	11.7s	11.8s	N/A
buildDurationCached	6.4s	5s	N/A
nodeModulesSize	196 MB	196 MB	⚠️ +282 kB
nextStartRea..uration (ms)	425ms	425ms	✓

Client Bundles (main, webpack)

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
3f784ff6-HASH.js gzip	53.5 kB	53.5 kB	N/A
423.HASH.js gzip	185 B	181 B	N/A
68-HASH.js gzip	29.6 kB	29.7 kB	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	238 B	240 B	N/A
main-HASH.js gzip	31.9 kB	31.9 kB	N/A
webpack-HASH.js gzip	1.7 kB	1.7 kB	✓
Overall change	46.9 kB	46.9 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
_app-HASH.js gzip	194 B	195 B	N/A
_error-HASH.js gzip	182 B	181 B	N/A
amp-HASH.js gzip	502 B	501 B	N/A
css-HASH.js gzip	320 B	322 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	N/A
edge-ssr-HASH.js gzip	255 B	256 B	N/A
head-HASH.js gzip	350 B	349 B	N/A
hooks-HASH.js gzip	368 B	369 B	N/A
image-HASH.js gzip	4.2 kB	4.2 kB	N/A
index-HASH.js gzip	257 B	256 B	N/A
link-HASH.js gzip	2.67 kB	2.67 kB	N/A
routerDirect..HASH.js gzip	310 B	311 B	N/A
script-HASH.js gzip	384 B	383 B	N/A
withRouter-HASH.js gzip	306 B	308 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	106 B	106 B	✓

Client Build Manifests

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
_buildManifest.js gzip	483 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
index.html gzip	527 B	527 B	✓
link.html gzip	541 B	539 B	N/A
withRouter.html gzip	523 B	522 B	N/A
Overall change	527 B	527 B	✓

Edge SSR bundle Size

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
edge-ssr.js gzip	94.4 kB	94.5 kB	N/A
page.js gzip	150 kB	150 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
middleware-b..fest.js gzip	619 B	624 B	N/A
middleware-r..fest.js gzip	151 B	149 B	N/A
middleware.js gzip	47.4 kB	47.4 kB	N/A
edge-runtime..pack.js gzip	1.94 kB	1.94 kB	✓
Overall change	1.94 kB	1.94 kB	✓

Next Runtimes

	vercel/next.js canary	gnoff/next.js render-with-dynamic	Change
app-page-exp...dev.js gzip	166 kB	166 kB	N/A
app-page-exp..prod.js gzip	95.4 kB	95.4 kB	N/A
app-page-tur..prod.js gzip	97.2 kB	97.2 kB	N/A
app-page-tur..prod.js gzip	91.6 kB	91.6 kB	N/A
app-page.run...dev.js gzip	136 kB	136 kB	N/A
app-page.run..prod.js gzip	90.2 kB	90.2 kB	N/A
app-route-ex...dev.js gzip	22 kB	22 kB	N/A
app-route-ex..prod.js gzip	14.9 kB	14.9 kB	N/A
app-route-tu..prod.js gzip	14.9 kB	14.9 kB	N/A
app-route-tu..prod.js gzip	14.7 kB	14.6 kB	N/A
app-route.ru...dev.js gzip	21.7 kB	21.7 kB	N/A
app-route.ru..prod.js gzip	14.7 kB	14.6 kB	N/A
pages-api-tu..prod.js gzip	9.43 kB	9.43 kB	✓
pages-api.ru...dev.js gzip	9.7 kB	9.7 kB	✓
pages-api.ru..prod.js gzip	9.43 kB	9.43 kB	✓
pages-turbo...prod.js gzip	22 kB	22.1 kB	N/A
pages.runtim...dev.js gzip	22.7 kB	22.7 kB	N/A
pages.runtim..prod.js gzip	22 kB	22.1 kB	N/A
server.runti..prod.js gzip	49.9 kB	49.9 kB	N/A
Overall change	28.6 kB	28.6 kB	✓

Diff details

Diff for page.js

Diff too large to display

Diff for edge-ssr.js

Diff too large to display

Diff for 68-HASH.js

Diff too large to display

Diff for app-page-exp..ntime.dev.js

Diff too large to display

Diff for app-page-exp..time.prod.js

Diff too large to display

Diff for app-page-tur..time.prod.js

Diff too large to display

Diff for app-page-tur..time.prod.js

Diff too large to display

Diff for app-page.runtime.dev.js

Diff too large to display

Diff for app-page.runtime.prod.js

Diff too large to display

Diff for app-route-ex..ntime.dev.js

Diff too large to display

Diff for app-route-ex..time.prod.js

Diff too large to display

Diff for app-route-tu..time.prod.js

Diff too large to display

Diff for app-route-tu..time.prod.js

Diff too large to display

Diff for app-route.runtime.dev.js

Diff too large to display

Diff for app-route.ru..time.prod.js

Diff too large to display

Diff for pages-turbo...time.prod.js

Diff too large to display

Diff for pages.runtime.dev.js

Diff too large to display

Diff for pages.runtime.prod.js

Diff too large to display

Diff for server.runtime.prod.js

Diff too large to display

Commit: b4555369b069ca7a8680b48742174251c1564b42