Miserlou
Oddities rendering `ReadableStream`
What version of astro are you using?
1.2.6
Are you using an SSR adapter? If so, which one?
Netlify edge
What package manager are you using?
npm
What operating system are you using?
mac
Describe the Bug
We (@matthewp and I) already had a twitter thread going on about this issue here, as well as some discussion on the Discord #netlify channel, but stuff tends to easily get lost in those channels, and I wanted to make a more complete/detailed summary of the issue.
Usecase
When using Astro in SSR mode on Netlify Edge, I'd like to fetch HTML, and stream the result of the HTML to the browser. My code looks as follows:
---
---
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<meta name="viewport" content="width=device-width" />
<meta name="generator" content={Astro.generator} />
<title>Astro</title>
</head>
<body>
<h1>Astro</h1>
{fetch(import.meta.env.SITE + '/foo.html').then(r => r.body)}
</body>
</html>
During local development using Node v16, if I refresh the page, I can clearly see that the HTML is correctly being streamed to the browser.
Once deployed on Netlify Edge, however, it seems to output/stringify a Uint8Array instead:
Investigation
Since Netlify Edge runs on Deno, and locally I'm running Astro with Node, I initially thought this might be a difference between the environments. So I tried to create a minimal reproduction of the rendering logic found here.
My reproduction looks like this:
async function* renderChild(child) {
child = await child;
if (child instanceof HTMLString) {
yield child;
} else if (typeof child === 'string') {
yield markHTMLString(child);
} else if (typeof child === 'object' && Symbol.asyncIterator in child) {
yield* child;
} else {
yield child;
}
}
for await (const chunk of renderChild(fetch('https://astro-blog-2.netlify.app/404.html').then(r => r.body))) {
console.log(chunk);
}
However, this outputs (in both Deno and Node):
Uint8Array(27) [
60, 112, 62, 79, 111, 112, 115,
33, 32, 80, 97, 103, 101, 32,
110, 111, 116, 32, 102, 111, 117,
110, 100, 60, 47, 112, 62
]
If I add the following code (taken from @natemoo-re 's Microstream here):
+const decoder = new TextDecoder();
async function* renderChild(child) {
child = await child;
if (child instanceof HTMLString) {
yield child;
+ } else if (child instanceof ReadableStream) {
+ const reader = child.getReader();
+ let res = await reader.read();
+ while (!res.done) {
+ yield decoder.decode(res.value);
+ res = await reader.read()
+ }
} else if (typeof child === 'string') {
yield markHTMLString(child);
} else if (typeof child === 'object' && Symbol.asyncIterator in child) {
yield* child;
} else {
yield child;
}
}
for await (const chunk of renderChild(fetch('https://astro-blog-2.netlify.app/404.html').then(r => r.body))) {
console.log(chunk);
}
The output I get is (as expected):
<p>Oops! Page not found.</p>
It's super confusing however that in the first example, the output is a Uint8Array, since during local development I definitely see the HTML being streamed:
https://user-images.githubusercontent.com/17054057/191459001-90a49971-91dc-4de0-8dc9-19b09ca37ece.mov
So the problem is:
Why does Astro output a Uint8Array on Netlify Edge?
Bonus question:
How is it possible that in my minimal reproduction I'm seeing the Uint8Array being output on Node, whereas in local development with Astro on Node, I'm seeing the HTML being streamed?
I'll continue to dig into this/debug this some more as well in the meantime, and will update this issue if I've made any progress.
Links
- Original twitter discussion thread
- Deployed demo on Netlify Edge
- Repo of demo
-
Astro
renderChildlogic -
Nate's Microstream solution for rendering
ReadableStreams
Link to Minimal Reproducible Example
see post
Participation
- [ ] I am willing to submit a pull request for this issue.
