PapaParse
PapaParse copied to clipboard
Upgrade XHR to use the fetch API
The next versions of Chrome, Opera, and Firefox will support fetch. It would be nice to phase out XHR and push the newer technologies.
Fetch isn't completed yet but when it is more stable and widely adopted, this would be great. Here's more info: http://jakearchibald.com/2015/thats-so-fetch/
(This issue is more of a memo to self. But comments/questions are welcome too.)
Tangential--it would be kind of neat if we spun up the next chunk fetch while we worked on parsing the current chunk. But that would probably require pulling in a promises library, and I'm not sure the performance gain would be so significant anyway.
Now is the time to add support for it.
Or at least add support for parsing a ReadableStream (fetch is not the only way to get a stream)
fetch(url).then(res => {
res.body.pipeThrough(new Papa.parser())
})
// get a stream from a large file...
new Response(file).body.pipeThrough(new Papa.parser())
This would break pause and resume functionality as fetch does not support those.
And otherwise, it's a fairly larger overhaul as moving from callbacks to Promises requires a lot of structural changes.
here is the code snippet I tried
const resp = await fetch("stream-csv.php", { redirect: "follow" }),
d = new TextDecoder(),
papaCSV = new Papa.ReadableStreamStreamer({
step: function({ data }) { console.log(data) },
complete: function() { console.log("ok") }
}), reader = resp.body.getReader();
var chunk = await reader.read();
while (!chunk.done) {
let text = d.decode(chunk.value);
// console.warn(text);
papaCSV.parseChunk(text);
chunk = await reader.read();
papaCSV._finished = !0;
papaCSV.parseChunk("");
}