MonitoRSS-Clone icon indicating copy to clipboard operation
MonitoRSS-Clone copied to clipboard

Using schedules.json on Heroku

Open typesoshee2 opened this issue 2 years ago • 0 comments

Hello. First of all, thank you so much for MonitoRSS and the instructions in this version for Heroku usage!

When I try including a settings/schedules.json to use custome refresh rates, I seem to get an error from Heroku and the bot shuts down. To be precise, when my MonitoRSS already has RSS feeds saved in it and the Heroku dyno is off and then started up, the bot seems to check every RSS feed that it has, and in this process an error occurs and it shuts down. To be more precise, this error happens when I have more than 20 RSS feeds saved in the bot and more than 20 custom refreshRateMinutes in schedules.json. 20 or under seems to be ok. (And if I have no schedules.json, I get no errors even with close to 100 RSS feeds saved in the bot.)

Here is an example log from Heroku when I get errors and it shuts down:

[2022-06-03 21:53:02.172 +0000] INFO	: [M] [0mAll shards have initialized by the Sharding Manager.
[2022-06-03 21:53:02.220 +0000] INFO	: [M] [0mStarted fetch intervals
[2022-06-03 21:53:12.813 +0000] INFO	: [default] [0mFinished feed retrieval cycle. No feeds to retrieve
[2022-06-03 21:53:12.815 +0000] INFO	: [user/2811699412] [0mFinished feed retrieval cycle (user/2811699412). No feeds to retrieve

user/2811699412 and user/5617891490 are keywords in my settings/schedules.json There are a bunch of other 'Finished feed retrieval cycle' lines here that are all the same type of message, just with different keywords that I put in my schedules.json. I've deleted these lines since they're basically all the same and just take up space.

[2022-06-03 21:53:13.758 +0000] INFO	: [user/5617891490] [0mFinished feed retrieval cycle (user/5617891490). No feeds to retrieve

Scaled to bot@1:Free web@0:Free by user [[Redacted, my e-mail]]
Process running mem=658M(128.6%)
Error R14 (Memory quota exceeded)
/app/.heroku/node/bin/node[259]: ../src/node_platform.cc:61:std::unique_ptr<long unsigned int> node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion `(0) == (uv_thread_create(t.get(), start_thread, this))' failed.
 1: 0xb58130 node::Abort() [/app/.heroku/node/bin/node]
 2: 0xb581ae  [/app/.heroku/node/bin/node]
 3: 0xbc93fe  [/app/.heroku/node/bin/node]
 4: 0xbc94d0 node::NodePlatform::NodePlatform(int, v8::TracingController*, v8::PageAllocator*) [/app/.heroku/node/bin/node]
 5: 0xb1b601 node::InitializeOncePerProcess(int, char**, node::InitializationSettingsFlags, node::ProcessFlags::Flags) [/app/.heroku/node/bin/node]
 6: 0xb1beb9 node::Start(int, char**) [/app/.heroku/node/bin/node]
 7: 0x7f145d6b4083 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
 8: 0xa93efe _start [/app/.heroku/node/bin/node]
[2022-06-03 21:53:27.451 +0000] INFO	: [M] [0mDetected process uncaughtException, sending kill signal to shards
node[240]: pthread_create: Resource temporarily unavailable
[2022-06-03 21:53:28.422 +0000] INFO	: [M] [0mDetected process exit, sending kill signal to shards
node:events:505
      throw er; // Unhandled 'error' event
      ^

Error: write EBADF
    at target._send (node:internal/child_process:864:20)
    at target.send (node:internal/child_process:737:19)
    at Processor.send (/app/node_modules/monitorss/src/structs/Processor.js:14:25)
    at ScheduleRun.processBatch (/app/node_modules/monitorss/src/structs/ScheduleRun.js:514:15)
    at node:internal/util:360:7
    at new Promise (<anonymous>)
    at ScheduleRun.processBatch (node:internal/util:346:12)
    at ScheduleRun.run (/app/node_modules/monitorss/src/structs/ScheduleRun.js:433:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ScheduleManager.run (/app/node_modules/monitorss/src/structs/ScheduleManager.js:283:7)
Emitted 'error' event on ChildProcess instance at:
    at node:internal/child_process:868:39
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
  errno: -9,
  code: 'EBADF',
  syscall: 'write'
}

Node.js v18.2.0

Side question: is there a way to prevent the bot from checking all the feeds when the Heroku dyno first starts up? (The Heroku dyno restarts automatically once a day due to a Heroku policy.) I only encounter the above problem when the bot starts up with (more than 20) saved RSS feeds and (more than 20 custom refresh rates in) schedules.json and in its start up process seems to check every feed it has saved (and uses up some sort of Heroku memory?). Thus, currently I remove all the RSS feeds from the bot, restart the dyno with a custom schedules.json (with > 20 custom refresh rates), which causes no errors, and then use the restore function to add back in (more than 20) RSS feeds via Discord, and then the bot functions with no problems until Heroku automatically restarts the dyno later. If there is a way I could have the bot not check all the feeds (the feed retrieval cycles?) when the dyno starts up, I wonder if I could circumvent this issue that Heroku is giving me.

Any help is appreciated!

typesoshee2 avatar Jun 12 '22 05:06 typesoshee2