w3 using 100% CPU

[dannyob]

I have a cronjob that runs this script every couple of hours: /usr/local/bin/w3 up /blahblah/dannyob.eth/

dannyob.eth has about 6GiB in it. The node process running w3 takes up 20% of my memory, and 100% of my CPU:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND 
3881583 danny     20   0   31.4g 881416  40464 R 109.0  22.3   4:59.28 node    

Should this be CPU bound? Is there something I can do to (bar niceing the process) to stop it taking up so much CPU/memory?

[Shubham-Rasal]

Having a similar issue.

[Pfed-prog]

@dannyob @Shubham-Rasal Could you please share more details for reproduction of the issue?

[Shubham-Rasal]

node storacha.js

<--- Last few GCs --->

[43741:0x3d1fb000]    20390 ms: Mark-Compact 4062.3 (4141.2) -> 4061.5 (4141.2) MB, pooled: 0 MB, 920.85 / 0.00 ms  (average mu = 0.664, current mu = 0.216) allocation failure; scavenge might not succeed
[43741:0x3d1fb000]    22025 ms: Mark-Compact 4077.2 (4141.2) -> 4076.8 (4172.7) MB, pooled: 0 MB, 1603.83 / 0.00 ms  (average mu = 0.404, current mu = 0.019) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe19de0 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 2: 0x1240390 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x1240667 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x146e1a5  [node]
 5: 0x1487a19 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 6: 0x145c0e8 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 7: 0x145d015 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x143540e v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node]
 9: 0x1423a6c v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawArray(int, v8::internal::AllocationType) [node]
10: 0x1424462 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArray(int, v8::internal::AllocationType) [node]
11: 0x15a1687  [node]
12: 0x15a2c22 v8::internal::JsonParser<unsigned char>::ParseJson(v8::internal::Handle<v8::internal::Object>) [node]
13: 0x12bd5e9 v8::internal::Builtin_JsonParse(int, unsigned long*, v8::internal::Isolate*) [node]
14: 0x7832afeac3b6 
zsh: IOT instruction (core dumped)  node storacha.js

cat storacha.js

cat storacha.js
import { create } from "@web3-storage/w3up-client";
import { readFile } from "fs/promises";
const client = await create();

console.log(client);

console.log(account);



await client.setCurrentSpace("did:ke...");

// const directoryCid = await client.uploadDirectory([
//   new File([await readFile("./package.json")], "package.json", {
//     type: "application/json",
//   }),
// ]);

// console.log(directoryCid);

const uploads = await client.capability.upload.list();

console.log(uploads);

[Pfed-prog]

There seems to be a missing requirements part in readme and repository, in general

[Shubham-Rasal]

It did work for the first time and then when I ran it again I was facing this issue.