All of a sudden receiving heap limit errors in node

[kmcaloon]

Very strange issue here. I've been using groq-js to run queries during Gatsby's build process and so far everything has worked fine. Then all of a sudden today I getting the following error:

success building schema - 0.151s
⠋ createPages

<--- Last few GCs --->

[6052:0x1045ed000]    35701 ms: Scavenge 4057.2 (4064.9) -> 4057.0 (4065.4) MB, 3.6 / 0.0 ms  (average mu = 0.137, current mu = 0.134) allocation failure 
[6052:0x1045ed000]    35705 ms: Scavenge 4057.8 (4065.4) -> 4057.6 (4066.1) MB, 3.4 / 0.0 ms  (average mu = 0.137, current mu = 0.134) allocation failure 
[6052:0x1045ed000]    35709 ms: Scavenge 4058.4 (4066.1) -> 4058.2 (4066.4) MB, 3.4 / 0.0 ms  (average mu = 0.137, current mu = 0.134) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10123d2a5 node::Abort() (.cold.1) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 2: 0x10009dd29 node::Abort() [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 3: 0x10009de8f node::OnFatalError(char const*, char const*) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 4: 0x1001d9db7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 5: 0x1001d9d57 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 6: 0x10036c885 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 7: 0x10036e11a v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 8: 0x10036a075 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
 9: 0x100367afe v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
10: 0x100374e3a v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
11: 0x100374ec1 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
12: 0x1003412a7 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
13: 0x1006a8050 v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]
14: 0x1009f3899 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/kmcaloon/.nvm/versions/node/v14.4.0/bin/node]

After much debugging, I found that by removing the following code everything works fine:

const groq = require( 'groq-js' );
...
const parsedQuery = groq.parse( query );
const value = await groq.evaluate( parsedQuery, { dataset } );
const result = await value.get();

It gets stuck and then eventually throws the error during value.get(). What's strange is that it has been fine until now, and it breaks with any query or dataset. I tried running the example from the docs and it caused the same issue. Nothing that I can think of changed on my system since it last worked, and I have tried using different versions of the package. Has anyone run into a similar issue before?

[judofyr]

What version of groq-js are you using? We recently switched to TypeScript. Maybe something changed?

[judofyr]

Nothing that I can think of changed on my system since it last worked, and I have tried using different versions of the package. Has anyone run into a similar issue before?

Oh, I see you mentioned it here. Hmm. v0.1.4 and v0.1.7 would be the most important ones to try out.

[kmcaloon]

Yes, tried both. What is also weird is I have another local gatsby site using the same functionality and it is still working. It's on v0.1.5 and I also tried that with this site. The only differences I can think of is this problem site is using styled components with a babel macro for tailwind css, but I can't see why any of that would be related. I've tried troubleshooting by disabling plugins, reinstalling all packages a bunch of times, etc. etc. I took a look at the code and saw that the get() method on the StreamValue class has the following loop:

  async get() {
    let result = []
    for await (let value of this) {
      result.push(await value.get())
    }
    return result
  }

Could this be resulting in an infinite loop for whatever reason? Again, it seems to be occurring with any dataset so I don't think that's the case. Will continue digging.

[kmcaloon]

Ah actually turns out 0.1.5 and below actually do work with this site. So the problem actually is with 0.1.7. Is there any way I can help debug?

[judofyr]

We've change things a bit internally. Could you try v0.2.0 and see if that's better?