marked low level stream / iterator support

My approche at potentially tackling a streamer / async render

Description

Add sync parser that can iterated over all token and yield the response back to the user to allow for some sort of streaming / flushing out the result.

the idea is not to concatenate the hole response into one single string, and instead yield all data as it spits it out

This could later allow for a higher level component to await for something if it where to return a promise

To prevent slowing down the synchronous parsing and slamming in async everywhere (like PR: #458) that would otherwise cause the event loop to trigger unnecessary delays everywhere. i opted to take a different approach in the way that the developer will have to wait for a chunk to resolve if it returns a promise. (similar to how i suggested it at https://github.com/markedjs/marked/issues/458#issuecomment-1081954012)

Things to consider:

This approach dose not mean we have to maintain two parsing classes.
if some renderer returns a promise then that will be returned to the streamer as a unresolved promise.

Fixes async renderer support #458

Contributor

some few stuff is breaking...

Committer

In most cases, this should be a different person than the contributor. and it also need to be bench tested against prev version

[ ] CI is green (no forced merge required).
[ ] Squash and Merge PR following conventional commit guidelines.

Mar 29 '22 21:03 jimmywarting

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.