sc3-plugins memory allocation fail: SuperDirt.start causes server crash if run after s.boot

[elgiano]

Environment

• SuperCollider version: 3.14-rc1
• Operating system: Linux
• Other details (Qt version, audio driver, etc.): jack

Steps to reproduce

Server.supernova
s.waitForBoot { SuperDirt.start }

Expected vs. actual behavior

Starting SuperDirt crashes supernova

---- core synth defs loaded ----
[...]
217 existing sample banks:
[...]
... file reading complete. Required 243 MB of memory.
SuperDirt: listening to Tidal on port 57120
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
node id 4 already in use
Server 'localhost' exited with exit code 0.
server 'localhost' disconnected shared memory interface

~This doesn't happen on 3.13, and does still happen before 247ef2e291a66f52ae0941ba18b164f3cc5b87b5 (udp buffer increase)~ It can happen also on 3.13

Can other people reproduce it?

[elgiano]

Actually this also happens on 3.13, and also on scsynth. It's a problem with sc3-plugins not using ClearUnitIfMemFail. I don't think this should block 3.14 at all, but we should fix sc3-plugins sometimes soon (however, it's around 800 matches for RTAlloc, we should split this work among more people)

What happens in this case:

• SuperDirt.start requires a bigger s.options.memSize. However, here I called SuperDirt.start after booting the server, thus ignoring this setting, so Supernova started with too little memory for SuperDirt.
• SuperDirt then creates some synths, some of which require too much memory for the default setting
• SwitchDelay fails memory allocation, dsp thread enters a Ctor loop, like sc used to do before ClearUnitIfMemFail was introduced. Memory is not cleaned up, the Ctor loop eats all available memory.
• server crashes with no helpful error messages

[telephon]

As a quick fix for SuperDirt, it could check if the memSize is too small and then enforce a reboot or throw an error instead of failing?

[elgiano]

The problem is that s.options.memSize is overwritten after the server has booted, so it doesn't reflect the previously booted server's state... and scsynth doesn't have an interface for checking the memory size of a running server... but clients could use /rtMemoryStatus to check for how much memory is currently free! Writing this in case anybody developing SuperDirt is looking :)

s.rtMemoryStatus { |freeBytes|
    // note that this function runs async after a server reply
    if (freeBytes < minSuperDirtFreeMemory) { "post some error".postln; s.reboot }
}

[telephon]

Nice, thank you! Do you have a good magic number in mind for the needed memory size?

[elgiano]

I don't know what's the recommended practice for starting SuperDirt, but as far as I can see all example startup files don't use SuperDirt.start (they use the instance method SuperDirt().start). So it looks like starting a server and then setting options is not recommended, yet SuperDirt.start seems to expect that server is not running:

	*start { |numChannels = 2, server, numOrbits = 12, port = 57120, senderAddr, path|
		~dirt.free;
		server = server ? Server.default;
		server.options.numBuffers = 1024 * 256;
		server.options.memSize = 8192 * 16;
		server.options.maxNodes = 1024 * 32;
		// boot the server and start SuperDirt
		server.waitForBoot {
			~dirt = SuperDirt(numChannels, server);
			~dirt.loadSoundFiles(path);   // load samples (path can be passed in)
			server.sync;
			~dirt.start(port, 0 ! numOrbits, senderAddr);
		};

		server.latency = 0.3;
	}

So, maybe this method could just check that the server is not running, or force a reboot. I would be a bit wary of checking for a certain amount of free memory, when the actual intention of this method seems to be to start a server from scratch. I think maybe the best would be to be explicit with the requirement that SuperDirt.start needs the server to reboot? It would also align it with all the example startup files (they all use .reboot)

However, if it can be in any way helpful, starting SuperDirt on my system uses about 34MB of server realtime memory (but probably this changes with sampleRate since most of this memory seems to be used for delay lines?)

[telephon]

That is a good idea I think, because these settings don't mean anything if the server is not booted.

[telephon]

OK, but if s.options.memSize was large enough, and then rtMemoryStatus returns too little, we have to basically warn and stop, right? So the idea would be:

1. check the memory size
2. if not enough set larger mem size
3. reboot
4. check the memory size, if nit enough warn and quit

what do you think?

[elgiano]

I'm afraid it would be overdoing it? I think in this case I would leave it up to the user... if s.options.memSize is high enough, but there is not enough free memory, it means that other processes on the same server are using it... so the user is already using a custom setup, has started a server and some processes, and in this case we shouldn't even reboot it, or maybe the user shouldn't use SuperDirt.start at all in this case (if that method is documented to reboot the server), but create a ~dirt = SuperDirt() and call start on that?

I also think we should move this discussion to SuperDirt's repo <3 I'm opening an issue there. This issue also should be perhaps moved to sc3-plugins...

[dyfer]

This issue also should be perhaps moved to sc3-plugins...

Yes! Could you do that @elgiano? I'm not sure if there's an automated way to do this; I suggest copy/paste, summarizing the discussion so far and linking to this issue, which should then be closed.

[capital-G]

You can transfer issues to other repos of the organization

[dyfer]

Thanks @capital-G !