faustjs
faustjs copied to clipboard
wpengine
HEADLESS WORDPRESS CONNECTION LOST 524 on NEXTJS FRONTEND
Applicable Versions
-
@faustjs/coreversion: -
@faustjs/reactversion: -
@faustjs/nextversion: - WordPress version:
Steps To Reproduce
Link to code example: https://newfaust.vercel.app
The current behavior
The expected behavior
@esilverman682 It looks like the host is down there. Were you able to pinpoint why the connection fails? Is the kapstaging site hosted in WP Engine Atlas? We might also need more information about the Applicable Versions
It is hosted on DigitalOcean with Cloudflare DNS +SSL pointed to the IP. The NEXTJs portion is on Vercel. Ive been trying to figure out the issue for some time now. It works well until one navigates on a few pages / urls. Then the backend begins to slo down and eventually gives an error code of 524 by cloudflare then it goes back up again on its own. memory limit is set high 512M along with post_max_size 512M. The server is Litespeed by the way.
On Tue, Jun 21, 2022 at 5:35 AM Theofanis Despoudis < @.***> wrote:
@esilverman682 https://github.com/esilverman682 It looks like the host is down there. Were you able to pinpoint why the connection fails? Is the kapstaging site hosted in WP Engine Atlas? We might also need more information about the Applicable Versions
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1161504955, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD6YULEYQMSGVYG7ZXVLVQGEHTANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
I am willing to grant access if needed.
On Tue, Jun 21, 2022 at 5:35 AM Theofanis Despoudis < @.***> wrote:
@esilverman682 https://github.com/esilverman682 It looks like the host is down there. Were you able to pinpoint why the connection fails? Is the kapstaging site hosted in WP Engine Atlas? We might also need more information about the Applicable Versions
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1161504955, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD6YULEYQMSGVYG7ZXVLVQGEHTANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
According to the docs 524 is timeout error. How long did you wait for this to show because it mentions by default it is 100 seconds? If the the backend begins to slow down maybe there is a GraphQL request that is very expensive to resolve.
Is it possible to replicate this flow using Chrome Recorder tool?
https://developer.chrome.com/docs/devtools/recorder/
You will have to enable recording as shown in the guide and perform the interactions. Once you hit the 524 take a screen shot and export the recording as JSON and share it here:
Will do.
On Wed, Jun 22, 2022 at 6:00 AM Theofanis Despoudis < @.***> wrote:
According to the docs 524 https://support.cloudflare.com/hc/en-us/articles/115003011431-Troubleshooting-Cloudflare-5XX-errors#524error is timeout error. How long did you wait for this to show because it mentions by default it is 100 seconds? If the the backend begins to slow down maybe there is a GraphQL request that is very expensive to resolve.
Is it possible to replicate this flow using Chrome Recorder tool?
https://developer.chrome.com/docs/devtools/recorder/
You will have to enable recording as shown in the guide and perform the interactions. Once you hit the 524 take a screen shot and export the recording as JSON and share it here:
[image: Screenshot 2022-06-22 at 10 59 22] https://user-images.githubusercontent.com/328805/175001868-9afe86fa-f3ae-49cb-b6fb-6d2b9c2ca384.png
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1162898821, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD66D5QWX5XURPSVGILDVQLP2LANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
@esilverman682 I'm trying to reproduce with no success. I will try the workflow again today.
I recreated the instance. it was working as expected and suddenly began to freeze the backend once again. Im using Custom Post Type + atlas content modeler mix with faust for the schema generation. I can add you to the github repo on the Nextjs frontend and to the Wordpress Backend if you need to get a better look at whats going on. Your help is much appreciated.
Front: https://model-eta.vercel.app/products-services/invoice-factoring Back: https://ada-kapitus.com/
On Wed, Jun 22, 2022 at 6:00 AM Theofanis Despoudis < @.***> wrote:
According to the docs 524 https://support.cloudflare.com/hc/en-us/articles/115003011431-Troubleshooting-Cloudflare-5XX-errors#524error is timeout error. How long did you wait for this to show because it mentions by default it is 100 seconds? If the the backend begins to slow down maybe there is a GraphQL request that is very expensive to resolve.
Is it possible to replicate this flow using Chrome Recorder tool?
https://developer.chrome.com/docs/devtools/recorder/
You will have to enable recording as shown in the guide and perform the interactions. Once you hit the 524 take a screen shot and export the recording as JSON and share it here:
[image: Screenshot 2022-06-22 at 10 59 22] https://user-images.githubusercontent.com/328805/175001868-9afe86fa-f3ae-49cb-b6fb-6d2b9c2ca384.png
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1162898821, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD66D5QWX5XURPSVGILDVQLP2LANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Hey @theodesp thank you. I was able to replicate the timeout issue using a some quick load testing.
❯ wrk -t12 -c400 -d30s https://model-eta.vercel.app/partner/sales-partners
Running 30s test @ https://model-eta.vercel.app/partner/sales-partners
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 343.40ms 236.10ms 1.97s 88.01%
Req/Sec 57.78 27.43 160.00 61.77%
20637 requests in 30.09s, 0.86GB read
Socket errors: connect 158, read 0, write 0, timeout 13
Requests/sec: 685.81
Transfer/sec: 29.15MB
However it is difficult to discern which component fails here.
It could be the Wordpress instance it could be the Vercel function or the network. We would need some sort of instrumentation here. You could check the logs to see which part times out first. Then you could try to use the Vercel Analytics https://vercel.com/analytics to see some insights.
If you identify that the issue is related to Wordpress then you will need to check the DigitalOcean host to see which part of the server fails. The DigitalOcean dashboard provides a monitoring tool for droplets -> https://docs.digitalocean.com/products/monitoring/
After that step you will need to check the Wordpress instance using an APM tool. For example you can use the free tier for NewRelic PHP Agent to capture traces within the server to pinpoint which part fails to deliver:
https://docs.newrelic.com/docs/apm/agents/php-agent/installation/install-php-agent-shared-hosting-service/
After that step you will know which part fails and we can provide insightful suggestions for resolving it.
Hi Theofanis, Here are the results. I attached a new recording and some screenshots at the time when Wordpress Admin froze today. U went thru product-services-business-loans and other product links and this is when it all went downhill.
*DIGITALOCEAN MONITORING- *set for ( if CPU is above 70) No Alert notification was sent * RELIC- https://onenr.io/07wkvpNxmQL username: @. password: Xena1234 *VERCEL- *No issues Score 100
On Wed, Jul 27, 2022 at 6:25 AM Theofanis Despoudis < @.***> wrote:
Hey @theodesp https://github.com/theodesp thank you. I was able to replicate the timeout issue using a some quick load testing.
❯ wrk -t12 -c400 -d30s https://model-eta.vercel.app/partner/sales-partners
Running 30s test @ https://model-eta.vercel.app/partner/sales-partners
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 343.40ms 236.10ms 1.97s 88.01% Req/Sec 57.78 27.43 160.00 61.77%20637 requests in 30.09s, 0.86GB read
Socket errors: connect 158, read 0, write 0, timeout 13
Requests/sec: 685.81
Transfer/sec: 29.15MB
[image: Screenshot 2022-07-27 at 11 13 27] https://user-images.githubusercontent.com/328805/181222936-c7cba347-e271-47b0-be52-e470beddc85a.png
However it is difficult to discern which component fails here.
It could be the Wordpress instance it could be the Vercel function or the network. We would need some sort of instrumentation here. You could check the logs to see which part times out first. Then you could try to use the Vercel Analytics https://vercel.com/analytics to see some insights.
If you identify that the issue is related to Wordpress then you will need to check the DigitalOcean host to see which part of the server fails. The DigitalOcean dashboard provides a monitoring tool for droplets -> https://docs.digitalocean.com/products/monitoring/
After that step you will need to check the Wordpress instance. For example you can use the free tier for NewRelic PHP Agent to capture traces within the server to pinpoint which part fails to deliver:
https://docs.newrelic.com/docs/apm/agents/php-agent/installation/install-php-agent-shared-hosting-service/
After that step you will know which part fails and we can provide suggestions for that.
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1196547531, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD62VVGPQOBJJNIVSW6DVWEFAXANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
New Recording to accompany the Relic logs.
On Wed, Jul 27, 2022 at 6:25 AM Theofanis Despoudis < @.***> wrote:
Hey @theodesp https://github.com/theodesp thank you. I was able to replicate the timeout issue using a some quick load testing.
❯ wrk -t12 -c400 -d30s https://model-eta.vercel.app/partner/sales-partners
Running 30s test @ https://model-eta.vercel.app/partner/sales-partners
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 343.40ms 236.10ms 1.97s 88.01% Req/Sec 57.78 27.43 160.00 61.77%20637 requests in 30.09s, 0.86GB read
Socket errors: connect 158, read 0, write 0, timeout 13
Requests/sec: 685.81
Transfer/sec: 29.15MB
[image: Screenshot 2022-07-27 at 11 13 27] https://user-images.githubusercontent.com/328805/181222936-c7cba347-e271-47b0-be52-e470beddc85a.png
However it is difficult to discern which component fails here.
It could be the Wordpress instance it could be the Vercel function or the network. We would need some sort of instrumentation here. You could check the logs to see which part times out first. Then you could try to use the Vercel Analytics https://vercel.com/analytics to see some insights.
If you identify that the issue is related to Wordpress then you will need to check the DigitalOcean host to see which part of the server fails. The DigitalOcean dashboard provides a monitoring tool for droplets -> https://docs.digitalocean.com/products/monitoring/
After that step you will need to check the Wordpress instance. For example you can use the free tier for NewRelic PHP Agent to capture traces within the server to pinpoint which part fails to deliver:
https://docs.newrelic.com/docs/apm/agents/php-agent/installation/install-php-agent-shared-hosting-service/
After that step you will know which part fails and we can provide suggestions for that.
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1196547531, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD62VVGPQOBJJNIVSW6DVWEFAXANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Hey @esilverman682 could you post relevant tracing info from NewRelic here? I think you can also export them as JSON.
Hi Theofanis,
I couldn't find any data on the traces section. Sorry Im a noob at this if you need to access directly I dont mind. Creds listed below.
RELIC https://onenr.io/07wkvpNxmQL username: @.*** password: Xena1234
Best
On Thu, Jul 28, 2022 at 7:41 AM Theofanis Despoudis < @.***> wrote:
Hey @esilverman682 https://github.com/esilverman682 could you post relevant tracing info from NewRelic here? I think you can also export them as JSON.
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1198028568, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD656OHDZURSOKUKRDX3VWJWVNANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Hey @esilverman682 you cannot share credentials here as Github masks emails and stuff. Maybe you can post some screenshots instead or share me on discord https://discord.com/channels/836253505944813629/984146897231032350
Hi Theopanis.
What is your tag/username on discord so I can send creds?
On Fri, Jul 29, 2022 at 4:55 AM Theofanis Despoudis < @.***> wrote:
Hey @esilverman682 https://github.com/esilverman682 you cannot share credentials here as Github masks emails and staff. Maybe you can post some screenshots instead or share me on discord https://discord.com/channels/836253505944813629/984146897231032350
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199037775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD63OC4ZSZB5WJHQLRRTVWOL7JANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
I sent over 3 images but need to split it up since email claims its too large of a file. Here is the first.
On Fri, Jul 29, 2022 at 4:55 AM Theofanis Despoudis < @.***> wrote:
Hey @esilverman682 https://github.com/esilverman682 you cannot share credentials here as Github masks emails and staff. Maybe you can post some screenshots instead or share me on discord https://discord.com/channels/836253505944813629/984146897231032350
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199037775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD63OC4ZSZB5WJHQLRRTVWOL7JANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Second.
On Fri, Jul 29, 2022 at 4:55 AM Theofanis Despoudis < @.***> wrote:
Hey @esilverman682 https://github.com/esilverman682 you cannot share credentials here as Github masks emails and staff. Maybe you can post some screenshots instead or share me on discord https://discord.com/channels/836253505944813629/984146897231032350
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199037775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD63OC4ZSZB5WJHQLRRTVWOL7JANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Hi Theofanis,
Just sent over a friend request to later send the creds.
Best, Wil
On Fri, Jul 29, 2022 at 8:29 AM Theofanis Despoudis < @.***> wrote:
Theo Despoudis#1014
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199220396, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD6ZSFPXBBXRM4VI5BD3VWPFB7ANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
StephenG#3896 on discord
On Fri, Jul 29, 2022 at 8:29 AM Theofanis Despoudis < @.***> wrote:
Theo Despoudis#1014
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199220396, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD6ZSFPXBBXRM4VI5BD3VWPFB7ANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>
Hey @esilverman682 I've sent you a message on Discord about the screenshots. Thank you.
Hey @esilverman682, I'm going to close this since this issue is more server related. Please feel free to open up another issue if you are still experiencing problems.
Third
On Fri, Jul 29, 2022 at 4:55 AM Theofanis Despoudis < @.***> wrote:
Hey @esilverman682 https://github.com/esilverman682 you cannot share credentials here as Github masks emails and staff. Maybe you can post some screenshots instead or share me on discord https://discord.com/channels/836253505944813629/984146897231032350
— Reply to this email directly, view it on GitHub https://github.com/wpengine/faustjs/issues/907#issuecomment-1199037775, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASKUD63OC4ZSZB5WJHQLRRTVWOL7JANCNFSM5ZKXLPKQ . You are receiving this because you were mentioned.Message ID: @.***>