core icon indicating copy to clipboard operation
core copied to clipboard

Render pages asynchronously ✈️

Open sanjaiyan-dev opened this issue 3 years ago • 8 comments

Improving the performance by running the independent async functions in parallel ✈️.

Ref-: https://github.com/vuejs/vitepress/pull/1320

Sorry if I made any mistakes :(

sanjaiyan-dev avatar Sep 18 '22 07:09 sanjaiyan-dev

This is pretty bad, as it may blow up machine memory. Image my blog have 800+ pages, with each process taking 40MB, I will use 32GB memory.

This may heavily do negative preformance effect due to Node.js "Old Space" behavior, as memory taken greater than old space will be swapped to hardrive.

You may have a test at my blog https://github.com/Mister-Hope/Mister-Hope.github.io, and I am pretty sure you will get the process killed while rendering, and a site with around 200 pages may get stuck for hours.

You should test this with real project to provide a brechmark with pr like this. The SSG process may actually faster if you make a "Promise queue" and limit it to 5-20 process at same time.

Welcome to try that and show brenchamrk.

Hi, Actually I thought node js runs the process in 4 default threads according to doc and it is also used for I/O tasks. And node js runs tasks concurrently or parallel according to device's capabilities.So I thought it won't break or hang the CPU by overloading because it will run certain tasks concurrently and parallel.

Ref link (about Node js parallel and concurrency) -: https://bytearcher.com/articles/parallel-vs-concurrent

Just my thoughts might be completely wrong

sanjaiyan-dev avatar Sep 18 '22 15:09 sanjaiyan-dev

This is pretty bad, as it may blow up machine memory. Image my blog have 800+ pages, with each process taking 40MB, I will use 32GB memory.

This may heavily do negative preformance effect due to Node.js "Old Space" behavior, as memory taken greater than old space will be swapped to hardrive.

You may have a test at my blog https://github.com/Mister-Hope/Mister-Hope.github.io, and I am pretty sure you will get the process killed while rendering, and a site with around 200 pages may get stuck for hours.

You should test this with real project to provide a brechmark with pr like this. The SSG process may actually faster if you make a "Promise queue" and limit it to 5-20 process at same time.

Welcome to try that and show brenchamrk.

Hi, Actually I thought node js runs the process in 4 default threads according to doc and it is also used for I/O tasks. And node js runs tasks concurrently or parallel according to device's capabilities.So I thought it won't break or hang the CPU by overloading because it will run certain tasks concurrently and parallel.

Ref link (about Node js parallel and concurrency) -: https://bytearcher.com/articles/parallel-vs-concurrent

Just my thoughts might be completely wrong

So I thought this won't break the CPU since it only uses the allocated amount of cores. 🥺

sanjaiyan-dev avatar Sep 18 '22 15:09 sanjaiyan-dev

I don't think a promise get gc before promise.all is fired.

However, I will show you some comparisons with my blog with your changes later.

Mister-Hope avatar Sep 18 '22 16:09 Mister-Hope

I don't think a promise get gc before promise.all is fired.

However, I will show you some comparisons with my blog with your changes later.

Hope it worked as expected :)

sanjaiyan-dev avatar Oct 01 '22 09:10 sanjaiyan-dev

I already approved this. Meteorlxy is in charge to merge this and publish new version.

Mister-Hope avatar Oct 07 '22 14:10 Mister-Hope

I already approved this. Meteorlxy is in charge to merge this and publish new version.

Sorry, Tks 💪🤝

sanjaiyan-dev avatar Oct 07 '22 16:10 sanjaiyan-dev

The CI failed

meteorlxy avatar Oct 08 '22 15:10 meteorlxy

The CI failed

@meteorlxy Extremely sorry, I think it failed due to some lint issues.

I used the GitHub web based editor to edit the code can you please help me to solve the issue by running a command:) Extremely sorry for the inconvenience.

sanjaiyan-dev avatar Oct 08 '22 15:10 sanjaiyan-dev

Let's just remove the spinner text updates for now.

meteorlxy avatar Dec 08 '22 13:12 meteorlxy

Let's just remove the spinner text updates for now.

🙌💪

sanjaiyan-dev avatar Dec 08 '22 14:12 sanjaiyan-dev