gatsby-source-strapi
gatsby-source-strapi copied to clipboard
strapi
Allow fetching all items with multiple calls instead of a very large one
Fix #104
Especially usefull on servers limiting the duration of requests (for example Heroku, which limits to 30sec each request).
Passing the loop parameter will perform multiple calls until all items are downloaded.
Readme.md updated with an example.
Hi @tonoli, thanks for reviewing. Could you tell what's missing in this issue before it could be merged ?
When can this feature be released? I think I need it. @philippeauriach @remidej
@remidej Changes made according to suggestions, thanks for your review. The only issue i didn't address is the part y ou ask for an explicit comment about the recursion, if you have an idea that would be nice ? thanks
Hey @remidej, is there anything we can do to so this can be merged?
I would really like to have this. Currently I'm trying to load 90k models, but whatever I try, this this plugin wants to fetch them all at once. I've even tried to split request by adding multiple types in a loop with the same name with api: { qs: { _limit: limit, _start: limit * index } },, but that fetches all those queries at once, still making Strapi crash out of memory.
We're also needing this to be able to deploy on heroku. Is the only change blocking a missing comment ?
I am closing the PR as a new version has been released, feel free to reopen it.
Since v2 is only compatible for Strapi 4, this fix would still be highly appreciated for us using Strapi 3 (MongoDB say hello 👋)
Would it be possible for this to be merged in as 1.1 @soupette ? Really, this plugin cannot be run without this fix. Without this, how would one load any data over a couple of hundred entries?
Since you are planning to go v2 for Strapi v4. Please merge this in as 1.1 for us poor souls running Strapi v3.
I've been running this code for a while now using patch-package and it's been working great for me. On cold build.
I just realized that on incremental builds on Gatsby Cloud, the loop isn't running. My assumption here is that Gatsby Cloud incremental builds (which detect a Strapi webhook) run their installed version of the source plugin, not my patched one.
So still I would really really really love to see this feature released as a version 1.1 or something for us (still stuck at) running projects on Strapi 3. I don't see any downsides in doing so.
If this PR needs any work still, I would be super happy to work on it.
@laurenskling have tried to the new version with the Strapi V4? It does support Gatsby incremental build
@soupette yes I know the v4 version is updated, but i'm running 10+ codebases on Strapi 3, and for all of them I need to: switch from mongo to sql, change all database fields to snake_case, update all my code, update all my plugins code, update my frontend code to use the new GraphQL output. It's not something I can do overnight, it will take me months, many months, before I'm ready to fully switch to v4. I am in the process, but until it's ready (probably need a full year for it), i would really really like it if we can get these little code snippets with big impact into v3. Thanks.
Closing this as it's targeting Strapi v3, which reaches end of life in a month, so we won't maintain the v3 version of this source plugin either
