Clear the cache if a build fails

[aponty]

Thanks for suggesting a new feature!

Please fill in the sections below.

Which problem is this feature request solving?

We're getting bad api responses cached, resulting in chains of broken builds. The bad response results in a Gatsby error in gatsby-node's onPostBuild, but the cache saves the build anyways, and then the next build fails because of the cached bad response, and the next, and the next, etc.

I've tried throwing errors inside the build process and running gatsby clean after the build command if there's an error, but that only remove .cache & .public, not whatever's attached to utils.cache.

We're working on upgrading to gatsby v4 to use the newer version, but unless I'm missing something skimming the source it looks like the same behaviour there.

I think the cache should, by default, be cleared if a build fails. We could check for the status code in onPostBuild and utils.cache.remove() if non-zero. No point in saving a bad image, especially if it leads to an endless chain of broken builds requiring manual intervention

[ascorbic]

Hey. It shouldn't be saving the cache if there is an error: the build should fail before onPostBuild is run, which is where the cache is saved. I don't think it makes sense to automatically clear the existing cache in these cases, because the error could be unconnected to it (it clearly built successfully before, or the cache wouldn't have been saved). The best solution is to choose "Clear cache and deploy" if this ever happens. Or are you saying that it's saving the new cache even when the build fails?

[aponty]

I think it must still be saving the cache, because the build does error out and the erroneous data is in the following caches.

I turned off caching last night and we no longer had these chains of failing builds, so I'm quite confident at this point that it's caching a build that should have failed.

Leaving this to clicking the button is a problem because we have 24 hr teams working on this site, not all of whom have netlify access, and requiring engineering time to go and cachebust periodically is a nuisance.

For reference, our build command is (essentially) gatsby build || { gatsby clean && exit 1; } so it was explicitly exiting as a failure if gatsby build ran -- and the logs reflected that. We also tried just gatsby build & other variations

For some reason, the logs from netlify-plugin-debug-cache are only accessible if a build succeeds, so can't see what exactly was being cached there.

Would be happy to talk this through via support ticket or email if you think that's a more appropriate venue

[aponty]

So, this problem still persists. We're getting occasionally erroneous data in createPages through an API call, and even with it set to throw an error and stop building there the bad data is cached.

We're seeing chains of 10+ builds, all failing, until you go in and manually clear the cache, and then it builds successfully until an error gets cached again. As manually monitoring our CI 24-7 isn't a real option, this means that we can't use gatsby caching at Flexport, significantly slowing down build times.

We're throwing errors before onPostBuild happens, so not sure why this data ends up in the cache, but it's definitely a caching issue.

[sandip192]

I am getting error cache. But if i press clear and deploy button it doesn't have any error. But when we push the code into github it gives same error. Again i have to press clear and deploy from dropdown from netlify

[orinokai]

Hi @aponty, could there possibly be some committed build artifacts that might cause this? If there's nothing obvious, I'd recommend popping by https://answers.netlify.com/ where the support team can take a look into it.

[nickytonline]

Hi @aponty, just following up on this.

[chandlervdw]

I'm also trying to ensure a clean build

[MarcL]

Is this still an issue @chandlervdw and @aponty?

[sandip192]

Dear I am not facing any problem as of now

On Mon, Nov 7, 2022, 4:50 PM Marc Littlemore @.***> wrote:

Is this still an issue @chandlervdw https://github.com/chandlervdw and @aponty https://github.com/aponty?

— Reply to this email directly, view it on GitHub https://github.com/netlify/netlify-plugin-gatsby/issues/199#issuecomment-1305461582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQT46QUVO3VJ67C7CUL2R6LWHDQW3ANCNFSM5JSVJFLQ . You are receiving this because you commented.Message ID: @.***>

[MarcL]

Closing this as fixed but please reopen if you think this is incorrect. Thanks!