supercollider icon indicating copy to clipboard operation
supercollider copied to clipboard
verified publisher icon supercollider

Speed up unit tests

Open capital-G opened this issue 1 year ago • 10 comments

Unit tests are best when they are fast, so they can be run often. But a current run of the unit tests take over 10 minutes, maybe it is possible to improve the runtime of the test frameworks with some clever tricks? If you have some idea how to reduce their runtime, please feel free to create a PR - it is greatly appreciated!

How to analyze the runtimes There is a python script which automates the reports, it is located @ https://gist.github.com/capital-G/253bdb368345a63f5e1180c13a00234e. Select a run and obtain its "View raw logs URL" (by actually clicking on it, as the browser will then generate a URL which is only valid for a limited amount of time). Pass it as a parameter or set the URL as a variable - the script will generate two html files which can be pasted directly into GH.

This post will be updated occasionally to get an overview - @ me me if you want a "re-run" of the calculations.

Logs from 2024-09-15 @ https://github.com/supercollider/supercollider/commit/381513e6dc504e6ddef60d05598b7c6fa84d52c1, based on https://github.com/supercollider/supercollider/actions/runs/10865580837/job/30152892774

Top 50 tests

file_name test_name time
21759 TestBus test_controlFree 13.76
22780 TestCoreUGens test_muladd 10.93
19074 TestSerialPort test_rxErrors_bufferOverflow 8.87
17948 TestPV_ChainUGen test_whiteframe_noop 7.93
22660 TestCoreUGens test_ugen_generator_equivalences 6.61
23066 TestCoreUGens test_oscillators_startAtCorrectPhase 6.37
24587 TestServer_boot test_rebootOffServer 5.00
17590 TestNodeProxy_Server test_schedAfterFade_afterQuant 4.75
24044 TestUGen_RTAlloc test_allUGens_allocPass 4.20
21547 TestFilterUGens test_time_invariance 3.91
21634 TestFilterUGens test_Integrator_add_presample_once_only 3.91
23006 TestCoreUGens test_impulse 3.89
24589 TestServer_boot test_rebootRunningServer 3.81
17997 TestPV_ChainUGen test_whiteframe_zeroing 3.69
23026 TestCoreUGens test_pitchtrackers 3.36
24084 TestUGen_RTAlloc test_allUGens_allocFail 3.30
22998 TestCoreUGens test_bufugens 3.11
24593 TestServer_boot test_notifyAndServerTreeActions 2.89
24591 TestServer_boot test_notifyAndServerBootActions 2.84
24583 TestServer_boot test_waitForBoot 2.79
17598 TestNodeProxy_Server test_reshaping_freeOldBus_after_fadeTime_quant 2.77
23033 TestCoreUGens test_delay1_2_predelayValues 2.74
24021 TestServer test_syncAll 2.71
24585 TestServer_boot test_bootSync 2.69
17776 TestIOUGens test_InFeedback_equals_In_for_write_after_read 2.51
17604 TestNodeProxy_Server test_control_fade 2.50
17774 TestIOUGens test_AudioControl_equals_In_for_read_after_write 2.49
17778 TestIOUGens test_AudioControl_equals_In_for_write_after_read 2.48
22573 TestIndexUGenRates test_indexRates 2.44
23044 TestCoreUGens test_binaryValue_isUniform 2.42
21941 TestNode_Server test_releaseGate 2.40
24019 TestServer test_sync50 2.31
21798 TestServer_clientID_booted test_repeatedRemoteLogin 2.30
21706 TestServer_ServerGUI test_quit 2.26
21673 TestServer_ServerGUI test_boot 2.26
21922 TestMixedBundle_Server test_addFunction 2.26
17904 TestMixedBundleTester test_findPreparationMessage 2.26
21684 TestServer_ServerGUI test_mute 2.26
17772 TestIOUGens test_InFeedback_equals_In 2.24
21926 TestMixedBundle_Server test_send_synth_bundle 2.24
17923 TestSynthAudioMapping test_map_with_control_rate 2.20
17596 TestNodeProxy_Server test_reshaping_freeOldBus_after_fadeTime 2.19
23889 TestNodeProxy test_control_fade 2.17
17921 TestSynthAudioMapping test_map_with_too_few_channels 2.16
17915 TestSynthAudioMapping test_map_normal_order 2.16
17919 TestSynthAudioMapping test_map_with_too_many_channels 2.15
17917 TestSynthAudioMapping test_map_reverse_order 2.15
21695 TestServer_ServerGUI test_unmute 2.15
23023 TestCoreUGens test_demand 2.15
24025 TestDoneActions test_freeSelfAndResumeNext 2.14

Top 30 files

time
file_name
TestNodeProxy_Server 50.84
TestCoreUGens 47.20
TestServer_boot 27.11
TestBus 17.27
TestBuffer_Server 16.93
TestServer_clientID_booted 14.35
TestServer 12.33
TestPV_ChainUGen 11.62
TestSynthAudioMapping 10.82
TestSerialPort 10.77
TestServer_ServerGUI 10.48
TestFilterUGens 9.82
TestIOUGens 9.72
TestQuery 9.60
TestUGen_RTAlloc 9.46
TestUGen_Duty 9.40
TestMixedBundleTester 8.09
TestServer_GUI 7.76
TestNetAddr 6.63
TestPanAz 6.63
TestEnvGen_server 6.05
TestNode_Server 5.90
TestMixedBundle_Server 5.85
TestSoundFile_Server 5.40
TestNodeProxy 3.98
TestProxySpace 3.76
TestVolume 3.55
TestNodeProxyBusMapping 3.30
TestIndexUGenRates 2.44
TestDoneActions 2.14

capital-G avatar Sep 03 '24 15:09 capital-G

Initial timings from 2024-09-03 for reference
Test # Test name Runtime (in seconds) Comment
78 TestPV_ChainUGen:test_whiteframe_noop 79.699997 See #6440
172 TestCoreUGens:test_ugen_generator_equivalences 30.309999 See #6374
174 TestCoreUGens:test_muladd 25.100000 See #6441 and #6374
79 TestPV_ChainUGen:test_whiteframe_zeroing 24.639999 See #6440
512 TestBus:test_controlFree 12.390000
175 TestCoreUGens:test_bufugens 11.290000
402 TestSerialPort:test_rxErrors_bufferOverflow 9.180000
183 TestCoreUGens:test_oscillators_startAtCorrectPhase 8.540000
176 TestCoreUGens:test_impulse 7.000000
153 TestServer_boot:test_rebootOffServer 5.010000
530 TestNodeProxy_Server:test_schedAfterFade_afterQuant 4.890000
177 TestCoreUGens:test_demand 4.830000
182 TestCoreUGens:test_line_startAndEndSampleAreOuputOverDur 4.160000
387 TestFilterUGens:test_time_invariance 4.080000
388 TestFilterUGens:test_Integrator_add_presample_once_only 3.900000
616 TestUGen_RTAlloc:test_PVUGens 3.640000
154 TestServer_boot:test_rebootRunningServer 3.620000
178 TestCoreUGens:test_pitchtrackers did not produce any results 3.160000
173 TestCoreUGens:test_exact_convergence 2.880000
155 TestServer_boot:test_notifyAndServerBootActions 2.870000
156 TestServer_boot:test_notifyAndServerTreeActions 2.840000
180 TestCoreUGens:test_delay1_2_predelayValues 2.710000
152 TestServer_boot:test_bootSync 2.670000
151 TestServer_boot:test_waitForBoot 2.640000
534 TestNodeProxy_Server:test_reshaping_freeOldBus_after_fadeTime_quant 2.640000
243 TestServer:test_syncAll 2.630000
503 TestNode_Server:test_releaseGate 2.410000
458 TestIOUGens:test_AudioControl_equals_In_for_write_after_read 2.360000
457 TestIOUGens:test_InFeedback_equals_In_for_write_after_read 2.360000
537 TestNodeProxy_Server:test_control_fade 2.310000
478 TestServer_clientID_booted:test_repeatedRemoteLogin 2.300000
623 TestDoneActions:test_freeSelfAndResumeNext 2.290000
456 TestIOUGens:test_AudioControl_equals_In_for_read_after_write 2.250000
455 TestIOUGens:test_InFeedback_equals_In 2.250000
242 TestServer:test_sync50 2.230000
520 TestNodeProxy_Server:test_synthDef_isReleased_afterFree 2.190000
291 TestNodeProxy:test_control_fade 2.180000
256 TestServer_ServerGUI:test_mute 2.170000
258 TestServer_ServerGUI:test_quit 2.160000
255 TestServer_ServerGUI:test_boot 2.150000
257 TestServer_ServerGUI:test_unmute 2.150000
501 TestNode_Server:test_get 2.150000
513 TestBus:test_get 2.130000
160 TestServer_boot:test_waitForBoot_fourWaysToPlaySound 2.110000
245 TestNodeProxyBusMapping:test_audiorate_mapping 2.100000
246 TestNodeProxyBusMapping:test_audiorate_mapping_elasticity 2.100000
525 TestNodeProxy_Server:test_reshaping_nil_numChannels_contraction 2.100000
353 TestBuffer_Server:test_cheby 2.100000
527 TestNodeProxy_Server:test_reshaping_elastic_numChannels_contraction 2.090000
524 TestNodeProxy_Server:test_reshaping_nil_numChannels_expansion 2.090000
515 TestNodeProxy_Server:test_send_loaded 2.090000
519 TestNodeProxy_Server:test_send_synthDef_rebuild 2.090000
528 TestNodeProxy_Server:test_reshaping_expanding_numChannels_expansion 2.090000
526 TestNodeProxy_Server:test_reshaping_elastic_numChannels_expansion 2.090000
84 TestMixedBundleTester:test_findMessage 2.080000
230 TestSynthAudioMapping:test_map_with_too_few_channels 2.080000
522 TestNodeProxy_Server:test_send_after_quit 2.080000
523 TestNodeProxy_Server:test_rebuild_after_quit 2.080000
85 TestMixedBundleTester:test_findPreparationMessage 2.080000
188 TestIndexUGenRates:test_indexRates 2.070000
227 TestSynthAudioMapping:test_map_normal_order 2.070000
516 TestNodeProxy_Server:test_send_isPlaying 2.060000
228 TestSynthAudioMapping:test_map_reverse_order 2.060000
685 TestSoundFile_Server:test_cue_playNow 2.050000
51 TestEnvGen_server:test_forced_release_respects_env_curve 2.050000
517 TestNodeProxy_Server:test_rebuild 2.050000
231 TestSynthAudioMapping:test_map_with_control_rate 2.050000
518 TestNodeProxy_Server:test_rebuild_loaded 2.050000
229 TestSynthAudioMapping:test_map_with_too_many_channels 2.050000
351 TestBuffer_Server:test_serverlang_dataexchange 2.010000
533 TestNodeProxy_Server:test_reshaping_freeOldBus_after_fadeTime 2.000000
52 TestEnvGen_server:test_shapeHold_setEndValue 1.990000
615 TestUGen_RTAlloc:test_LocalBuf 1.990000
42 TestQuery:test_multiNodeQuery 1.980000
41 TestQuery:test_groupQuery 1.980000
39 TestQuery:test_multiBufferQuery 1.970000
358 TestBuffer_Server:test_getToFloatArray_index_count 1.940000
40 TestQuery:test_synthQuery 1.930000
476 TestServer_clientID_booted:test_repeatedLogin 1.920000
629 TestPanAz:test_PanAz_equalPowerBetweenTwoAdjacentChannels 1.910000
543 TestUGen_Duty:test_ar_1_frame_demandrate_reset 1.900000
389 TestFilterUGens:test_ugen_generator_equivalences 1.900000
86 TestMixedBundleTester:test_send 1.900000
540 TestUGen_Duty:test_ar_16_frames 1.900000
266 TestMixedBundle_Server:test_send_synth_bundle 1.900000
264 TestMixedBundle_Server:test_addFunction 1.880000
626 TestPanAz:test_PanAz_ak_initializesAmps 1.880000
542 TestUGen_Duty:test_ar_1_frame_audiorate_reset 1.880000
350 TestBuffer_Server:test_freeAll 1.860000
87 TestMixedBundleTester:test_defNames 1.860000
239 TestServer:test_sync5 1.850000
241 TestServer:test_sync30 1.840000
240 TestServer:test_sync10 1.840000
625 TestServer_SynthDefVersion1:test_serverLoadsSonicPiBeepSynthDef 1.820000
38 TestQuery:test_bufferQuery 1.810000
265 TestMixedBundle_Server:test_onSend 1.810000
386 TestTWindex:test_TWindex_ar_skips_zero_probability 1.810000
477 TestServer_clientID_booted:test_remoteLogin 1.800000
356 TestBuffer_Server:test_regression_getn_float 1.800000
357 TestBuffer_Server:test_getToFloatArray 1.800000
532 TestNodeProxy_Server:test_schedAfterFade_cmdPeriod_removed 1.800000
541 TestUGen_Duty:test_ar_256_frames 1.800000
355 TestBuffer_Server:test_getn 1.800000
354 TestBuffer_Server:test_get 1.800000
277 TestNodeProxy:test_asCode_playState 1.800000
611 TestUGen_RTAlloc:test_allocFail_clearUnit 1.790000
620 TestServer_GUI:test_serverQuit 1.790000
475 TestServer_clientID_booted:test_lockedWhileRunning 1.780000
684 TestSoundFile_Server:test_cue_passInEvent 1.780000
614 TestUGen_RTAlloc:test_allocFail_setDoneFlag 1.780000
417 TestNetAddr:test_sendBundle30 1.780000
50 TestEnvGen_server:test_neg_gate_ignores_multinode_release 1.780000
612 TestUGen_RTAlloc:test_allocFail_continueProcessing_sameNode 1.770000
535 TestNodeProxy_Server:test_clear_fadeTime 1.770000
473 TestServer_clientID_booted:test_setWithConstructor 1.770000
244 TestServer:test_recorder_usesRecHeaderFormatFromServer 1.760000
539 TestUGen_Duty:test_ar_1_frame 1.760000
415 TestNetAddr:test_sendBundle5 1.760000
613 TestUGen_RTAlloc:test_allocFail_continueProcessing_otherNodes 1.760000
107 TestSanitize:test_sanitize 1.760000
418 TestNetAddr:test_sendBundle50 1.760000
416 TestNetAddr:test_sendBundle10 1.760000
628 TestPanAz:test_PanAz_crosstalk 1.750000
253 TestProxySpace:test_copy_as_currentEnvironment 1.750000
181 TestCoreUGens:test_binaryValue_isUniform 1.750000
627 TestPanAz:test_PanAz_ar_initializesAmps 1.750000
430 TestFunction:test_plot 1.740000
149 TestVolume:test_setVolume 1.730000
150 TestVolume:test_numOutputs 1.730000
352 TestBuffer_Server:test_allocAndQuery 1.730000
536 TestNodeProxy_Server:test_synthDefControl_build 1.720000
252 TestProxySpace:test_storeOn_recoversState 1.710000
165 TestOSCBundle:test_doPrepare 1.700000
157 TestServer_boot:test_ServerBootActionTiming 1.690000
619 TestServer_GUI:test_serverTreeDuringCmdPeriod 1.690000
474 TestServer_clientID_booted:test_nodeIDAlloc_ServerTree 1.690000
538 TestNodeProxy_Server:test_stop_object_should_cancel_old_bundle 1.690000
617 TestServer_GUI:test_serverBoot 1.670000
342 TestUnitTest:test_bootServer 1.650000
158 TestServer_boot:test_ServerQuitActionTiming 1.650000
471 TestServer_clientID_booted:test_default 1.650000
686 TestSoundFile_Server:test_cue_eventConfiguration 1.650000
179 TestCoreUGens:test_out_ugens 1.650000
618 TestServer_GUI:test_serverTreeDuringBoot 1.650000
472 TestServer_clientID_booted:test_setWithSetter 1.640000
159 TestServer_boot:test_allocWhileBooting 1.570000
502 TestNode_Server:test_getn 1.200000
514 TestBus:test_getn 1.190000
531 TestNodeProxy_Server:test_schedAfterFade_cmdPeriod 1.140000
529 TestNodeProxy_Server:test_reshaping_expanding_numChannels_contraction 1.140000
521 TestNodeProxy_Server:test_loaded_after_quit 1.130000
670 TestSCDocHTMLRenderer:test_htmlForLink_externalLink 0.710000
254 TestServer_ServerGUI:test_noBoot 0.610000
639 TestReadableNodeIDAllocator:test_permIDs 0.540000
34 TestEvent:test_server_messages_type_note 0.530000
259 TestServer_ServerGUI:test_default 0.500000
260 TestServer_ServerGUI:test_undefault 0.500000
414 TestTaskProxy:test_TaskProxy_playOnInstanceClock 0.490000
634 TestPatternProxy:test_time_precision 0.430000
557 TestArray:test_array_nonmutating 0.300000
689 TestNoOps:test_arithmetics 0.140000
468 TestMethod:test_method_argumentString 0.120000
283 TestNodeProxy:test_schedAfterFade 0.110000
0 TestThrow:test_throw_nested_threads 0.110000

Grouped by files - filtered out files that take less than 0.01s

file name accumulated runtime (in sec)
TestPV_ChainUGen 104.339996
TestCoreUGens 103.379997
TestNodeProxy_Server 49.379997
TestServer_boot 26.670000
TestBuffer_Server 16.840000
TestBus 15.710001
TestServer_clientID_booted 14.550000
TestUGen_RTAlloc 12.730000
TestServer 12.150001
TestSynthAudioMapping 10.309999
TestServer_ServerGUI 10.240001
TestFilterUGens 9.880000
TestSerialPort 9.690001
TestQuery 9.670000
TestUGen_Duty 9.240000
TestIOUGens 9.219999
TestMixedBundleTester 7.920000
TestPanAz 7.290000
TestNetAddr 7.060000
TestServer_GUI 6.800000
TestEnvGen_server 5.820000
TestNode_Server 5.760000
TestMixedBundle_Server 5.590000
TestSoundFile_Server 5.480000
TestNodeProxyBusMapping 4.200000
TestNodeProxy 4.090000
TestVolume 3.460000
TestProxySpace 3.460000
TestDoneActions 2.290000
TestIndexUGenRates 2.070000
TestServer_SynthDefVersion1 1.820000
TestTWindex 1.810000
TestSanitize 1.760000
TestUnitTest 1.750000
TestFunction 1.740000
TestOSCBundle 1.700000
TestSCDocHTMLRenderer 1.060000
TestEvent 0.700000
TestReadableNodeIDAllocator 0.560000
TestPatternProxy 0.510000
TestTaskProxy 0.490000
TestArray 0.370000
TestNoOps 0.160000
TestClockScheduling 0.160000
TestMethod 0.130000
TestThrow 0.110000
TestWebView 0.100000
TestPattern 0.060000
TestEventStreamCleanup 0.060000
TestProcessCreation 0.020000
TestPprotect 0.020000
TestPmono 0.020000
TestParser 0.010000

capital-G avatar Sep 03 '24 16:09 capital-G

Maybe we should discuss and coordinate efforts and ideas of speeding tests up, such as parallelization, here as a centralized space instead of spread out across multiple PRs?

Parallelization

As I expressed in https://github.com/supercollider/supercollider/pull/6462#issuecomment-2336697161, I would prefer a non-parallelized approach. I think we currently have testing problems b/c it is hard enough to maintain proper state synchronization between sclang/scsynth due to the async nature of their communication. Introducing parallelization would most likely introduce nondeterministic test runs that are hard to inspect.

TCP/UDP

It looks like some tests fail when run too fast because it looks like UDP messages are being dropped (would this "overflow" be logged somewhere?). We could/should check the UDP buffer size of the GHA runners, but there are also efforts via #5993 to increase the limits at the sclang/scsynth level. On the other hand - we could switch to TCP for testing, which guarantees that messages are not dropped and arrive in order, leading to a more deterministic environment. At the same time, this is a departure from the default UDP way of communicating within SuperCollider, so this could introduce other bugs that go unnoticed?

Booting

IMO most of the runtime of the tests can be "cut away" by reducing the number of times a server is started and stopped. This needs to be evaluated on a test-by-test basis, but I think not every test should spawn its own server multiple times. Maybe we should follow something like a dependency injection pattern that injects a running server into a ServerUnitTest - this would also allow us to run tests and benchmarks based on server configuration, as we can inject a single server that we have configured to our liking/what we want to test.

This could of course introduce another source of contamination (which is not trivial, as seen in https://github.com/supercollider/supercollider/issues/6410#issuecomment-2339309292), but the test suite would then fail in a deterministic way.

Perhaps we can use a possible `ServerUnitTest' class to specify whether we want to run the tests on an isolated per-class or per-test basis for debugging purposes?

I just grepd the number of boots for each test to get a feeling of our 152 boots in a test run (taken from https://github.com/supercollider/supercollider/actions/runs/10789650345/job/29975092134?pr=6453)

Booting occurences
ServerName (should match file)
'TestNodeProxy_Server' 24
'TestCoreUGens' 12
'TestServer_boot' 11
'TestBuffer_Server' 9
'TestUGen_RTAlloc' 9
'TestServer' 6
'TestServer_clientID_booted' 5
'TestSynthAudioMapping' 5
'TestUGen_Duty' 5
'TestQuery' 5
'TestServer_GUI' 4
'TestPanAz' 4
'TestIOUGens' 4
'TestNetAddr' 4
'TestServer_ServerGUI' 4
'TestMixedBundleTester' 4
'TestMixedBundle_Server' 3
'TestEnvGen_server' 3
'TestNode_Server' 3
'TestFilterUGens' 3
'TestSoundFile_Server' 3
'TestNodeProxyBusMapping' 2
'TestProxySpace' 2
'TestPV_ChainUGen' 2
'TestNodeProxy' 2
'test_setVolume' 1
'TestSanitize' 1
'TestDoneActions' 1
'TestTWindex' 1
'test_numOutputs' 1
'test_get' 1
'test_getn' 1
'TestOSCBundle' 1
'TestIndexUGenRates' 1
'localhost' 1
'test_plot' 1
'TestUnitTest' 1
'test_repeatedLogin' 1
'test_serverLoadsSonicPiBeepSynthDef' 1
Raw logs 
❯ cat job-logs.txt | grep "Booting server"
2024-09-10T09:30:42.3010283Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:44.2261829Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:46.2317383Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:48.1944235Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:50.2211339Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:52.2263756Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:54.3809539Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:56.4074211Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:30:58.4128089Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:00.4395991Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:02.4662869Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:04.4288843Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:06.4555810Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:08.4180622Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:10.4448344Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:12.4714923Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:17.2926274Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:19.3193336Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:21.4524029Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:23.7778480Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:26.6154253Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:28.7272147Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:30.7536128Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:33.3991654Z Booting server 'TestNodeProxy_Server' on address 127.0.0.1:57110.
2024-09-10T09:31:41.6447392Z Booting server 'TestIOUGens' on address 127.0.0.1:57110.
2024-09-10T09:31:44.1542120Z Booting server 'TestIOUGens' on address 127.0.0.1:57110.
2024-09-10T09:31:46.7570770Z Booting server 'TestIOUGens' on address 127.0.0.1:57110.
2024-09-10T09:31:49.3591650Z Booting server 'TestIOUGens' on address 127.0.0.1:57110.
2024-09-10T09:31:53.9161616Z Booting server 'TestProxySpace' on address 127.0.0.1:57110.
2024-09-10T09:31:55.8631044Z Booting server 'TestProxySpace' on address 127.0.0.1:57110.
2024-09-10T09:31:57.9495049Z Booting server 'TestEnvGen_server' on address 127.0.0.1:57110.
2024-09-10T09:32:00.1084639Z Booting server 'TestEnvGen_server' on address 127.0.0.1:57110.
2024-09-10T09:32:02.2844118Z Booting server 'TestEnvGen_server' on address 127.0.0.1:57110.
2024-09-10T09:32:07.1311367Z Booting server 'TestSanitize' on address 127.0.0.1:57110.
2024-09-10T09:32:09.5984376Z Booting server 'TestMixedBundleTester' on address 127.0.0.1:57110.
2024-09-10T09:32:11.7964623Z Booting server 'TestMixedBundleTester' on address 127.0.0.1:57110.
2024-09-10T09:32:14.2068574Z Booting server 'TestMixedBundleTester' on address 127.0.0.1:57110.
2024-09-10T09:32:16.4584482Z Booting server 'TestMixedBundleTester' on address 127.0.0.1:57110.
2024-09-10T09:32:18.4764186Z Booting server 'TestSynthAudioMapping' on address 127.0.0.1:57110.
2024-09-10T09:32:20.6096027Z Booting server 'TestSynthAudioMapping' on address 127.0.0.1:57110.
2024-09-10T09:32:22.9136221Z Booting server 'TestSynthAudioMapping' on address 127.0.0.1:57110.
2024-09-10T09:32:25.2400257Z Booting server 'TestSynthAudioMapping' on address 127.0.0.1:57110.
2024-09-10T09:32:27.5428579Z Booting server 'TestSynthAudioMapping' on address 127.0.0.1:57110.
2024-09-10T09:32:31.2930635Z Booting server 'TestPV_ChainUGen' on address 127.0.0.1:57110.
2024-09-10T09:32:39.3824509Z Booting server 'TestPV_ChainUGen' on address 127.0.0.1:57110.
2024-09-10T09:32:44.3858286Z Booting server 'TestUGen_Duty' on address 127.0.0.1:57110.
2024-09-10T09:32:46.5081636Z Booting server 'TestUGen_Duty' on address 127.0.0.1:57110.
2024-09-10T09:32:48.4706007Z Booting server 'TestUGen_Duty' on address 127.0.0.1:57110.
2024-09-10T09:32:50.4560217Z Booting server 'TestUGen_Duty' on address 127.0.0.1:57110.
2024-09-10T09:32:52.4173249Z Booting server 'TestUGen_Duty' on address 127.0.0.1:57110.
2024-09-10T09:32:54.5505611Z Booting server 'test_serverLoadsSonicPiBeepSynthDef' on address 127.0.0.1:57110.
2024-09-10T09:33:14.1166127Z Booting server 'TestFilterUGens' on address 127.0.0.1:57110.
2024-09-10T09:33:18.4496790Z Booting server 'TestFilterUGens' on address 127.0.0.1:57110.
2024-09-10T09:33:20.4486036Z Booting server 'localhost' on address 127.0.0.1:57110.
2024-09-10T09:33:22.7993886Z Booting server 'TestFilterUGens' on address 127.0.0.1:57110.
2024-09-10T09:33:26.9905527Z Booting server 'TestUnitTest' on address 127.0.0.1:57110.
2024-09-10T09:33:33.3546114Z Booting server 'TestServer_ServerGUI' on address 127.0.0.1:57110.
2024-09-10T09:33:35.9546670Z Booting server 'TestServer_ServerGUI' on address 127.0.0.1:57110.
2024-09-10T09:33:38.5608049Z Booting server 'TestServer_ServerGUI' on address 127.0.0.1:57110.
2024-09-10T09:33:41.1515602Z Booting server 'TestServer_ServerGUI' on address 127.0.0.1:57110.
2024-09-10T09:34:04.8430672Z Booting server 'test_get' on address 127.0.0.1:57110.
2024-09-10T09:34:07.1058943Z Booting server 'test_getn' on address 127.0.0.1:57110.
2024-09-10T09:34:10.8257178Z Booting server 'TestServer_clientID_booted' on address 127.0.0.1:57110.
2024-09-10T09:34:12.9504341Z Booting server 'TestServer_clientID_booted' on address 127.0.0.1:57110.
2024-09-10T09:34:15.0634972Z Booting server 'TestServer_clientID_booted' on address 127.0.0.1:57110.
2024-09-10T09:34:17.2081824Z Booting server 'TestServer_clientID_booted' on address 127.0.0.1:57110.
2024-09-10T09:34:19.3687818Z Booting server 'TestServer_clientID_booted' on address 127.0.0.1:57110.
2024-09-10T09:34:21.6434634Z Booting server 'test_repeatedLogin' on address 127.0.0.1:57110.
2024-09-10T09:34:50.8521201Z Booting server 'TestMixedBundle_Server' on address 127.0.0.1:57110.
2024-09-10T09:34:53.6313672Z Booting server 'TestMixedBundle_Server' on address 127.0.0.1:57110.
2024-09-10T09:34:55.4406413Z Booting server 'TestMixedBundle_Server' on address 127.0.0.1:57110.
2024-09-10T09:34:58.1935543Z Booting server 'TestNodeProxyBusMapping' on address 127.0.0.1:57110.
2024-09-10T09:35:00.4765046Z Booting server 'TestNodeProxyBusMapping' on address 127.0.0.1:57110.
2024-09-10T09:35:03.4940544Z Booting server 'TestNode_Server' on address 127.0.0.1:57110.
2024-09-10T09:35:05.3677890Z Booting server 'TestNode_Server' on address 127.0.0.1:57110.
2024-09-10T09:35:07.2474996Z Booting server 'TestNode_Server' on address 127.0.0.1:57110.
2024-09-10T09:35:14.7242385Z Booting server 'test_plot' on address 127.0.0.1:57110.
2024-09-10T09:35:30.5375541Z Booting server 'TestPanAz' on address 127.0.0.1:57110.
2024-09-10T09:35:33.2347755Z Booting server 'TestPanAz' on address 127.0.0.1:57110.
2024-09-10T09:35:34.9789265Z Booting server 'TestPanAz' on address 127.0.0.1:57110.
2024-09-10T09:35:37.6583288Z Booting server 'TestPanAz' on address 127.0.0.1:57110.
2024-09-10T09:36:24.0483973Z Booting server 'TestQuery' on address 127.0.0.1:57110.
2024-09-10T09:36:26.7994739Z Booting server 'TestQuery' on address 127.0.0.1:57110.
2024-09-10T09:36:28.8312936Z Booting server 'TestQuery' on address 127.0.0.1:57110.
2024-09-10T09:36:31.6177343Z Booting server 'TestQuery' on address 127.0.0.1:57110.
2024-09-10T09:36:33.6323924Z Booting server 'TestQuery' on address 127.0.0.1:57110.
2024-09-10T09:36:38.3623177Z Booting server 'TestIndexUGenRates' on address 127.0.0.1:57110.
2024-09-10T09:36:46.3460677Z Booting server 'TestOSCBundle' on address 127.0.0.1:57110.
2024-09-10T09:36:51.7288567Z Booting server 'TestServer_GUI' on address 127.0.0.1:57110.
2024-09-10T09:36:54.4200671Z Booting server 'TestServer_GUI' on address 127.0.0.1:57110.
2024-09-10T09:36:57.1357241Z Booting server 'TestServer_GUI' on address 127.0.0.1:57110.
2024-09-10T09:36:58.9258316Z Booting server 'TestServer_GUI' on address 127.0.0.1:57110.
2024-09-10T09:37:01.2222055Z Booting server 'TestSoundFile_Server' on address 127.0.0.1:57110.
2024-09-10T09:37:03.6634945Z Booting server 'TestSoundFile_Server' on address 127.0.0.1:57110.
2024-09-10T09:37:06.2480202Z Booting server 'TestSoundFile_Server' on address 127.0.0.1:57110.
2024-09-10T09:37:13.2091351Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:20.5274971Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:22.9649104Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:34.6263598Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:38.3430484Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:42.9351225Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:45.6688436Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:49.6963103Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:52.0505769Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:55.4797702Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:37:58.4704448Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:38:01.2486203Z Booting server 'TestCoreUGens' on address 127.0.0.1:57110.
2024-09-10T09:38:30.2227103Z Booting server 'TestNodeProxy' on address 127.0.0.1:57110.
2024-09-10T09:38:42.7363096Z Booting server 'TestNodeProxy' on address 127.0.0.1:57110.
2024-09-10T09:39:11.6116045Z Booting server 'test_setVolume' on address 127.0.0.1:57110.
2024-09-10T09:39:14.1481930Z Booting server 'test_numOutputs' on address 127.0.0.1:57110.
2024-09-10T09:39:23.5786904Z Booting server 'TestTWindex' on address 127.0.0.1:57110.
2024-09-10T09:39:26.1889640Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:28.9725956Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:31.7252183Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:34.3424040Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:37.3413674Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:40.8393442Z Booting server 'TestServer' on address 127.0.0.1:57110.
2024-09-10T09:39:43.4068509Z Booting server 'TestDoneActions' on address 127.0.0.1:57110.
2024-09-10T09:39:47.6266670Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:39:50.1935861Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:39:52.7344842Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:39:55.2905976Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:39:57.8120400Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:40:00.4473969Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:40:03.0169958Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:40:05.6636196Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:40:08.3096795Z Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
2024-09-10T09:40:41.1915476Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:43.8793446Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:46.6942416Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:49.2507869Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:52.0522498Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:54.6838039Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:57.2937380Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:40:59.9075760Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:41:02.5605513Z Booting server 'TestBuffer_Server' on address 127.0.0.1:57110.
2024-09-10T09:41:17.2059833Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:20.8955909Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:23.8993074Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:29.7777393Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:31.9002028Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:34.5733418Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:38.3265371Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:42.0352444Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:44.5961752Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:47.2379937Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:49.6693895Z Booting server 'TestServer_boot' on address 127.0.0.1:57110.
2024-09-10T09:41:56.1815557Z Booting server 'TestNetAddr' on address 127.0.0.1:57110.
2024-09-10T09:41:58.8366033Z Booting server 'TestNetAddr' on address 127.0.0.1:57110.
2024-09-10T09:42:01.3804995Z Booting server 'TestNetAddr' on address 127.0.0.1:57110.
2024-09-10T09:42:03.9301238Z Booting server 'TestNetAddr' on address 127.0.0.1:57110.

❯ cat job-logs.txt | grep "Booting server" | wc -l
152

capital-G avatar Sep 11 '24 08:09 capital-G

On the other hand - we could switch to TCP for testing, which guarantees that messages are not dropped and arrive in order, leading to a more deterministic environment. At the same time, this is a departure from the default UDP way of communicating within SuperCollider, so this could introduce other bugs that go unnoticed?

TCP isn't tested at all. https://github.com/search?q=repo%3Asupercollider%2Fsupercollider+%28%22%27tcp%27%22+OR+%22%5Ctcp%22%29&type=code I think we should test it as it is supported. Running all tests both on UDP and TCP is probably overkill in the CI, but it might be nice to provide a default server options used by all tests, so switch can be done easily.

JordanHendersonMusic avatar Sep 11 '24 08:09 JordanHendersonMusic

If TCP becomes a viable candidate (currently under test in #6463), I don't offhand see a downside as long as the tests aren't explicitly about messaging.

It looks like some tests fail when run too fast because it looks like UDP messages are being dropped (would this "overflow" be logged somewhere?)

This would indeed be good to know, or better yet, if the current queue use/capacity could be polled, allowing for selective use of .sync.

mtmccrea avatar Sep 11 '24 11:09 mtmccrea

allowing for selective use of .sync.

I did some benchmarks and some testing. Rather than repost them, here is the link.

https://scsynth.org/t/switching-unit-tests-to-use-tcp-and-increase-parallelization/10045/4?u=jordan

JordanHendersonMusic avatar Sep 11 '24 12:09 JordanHendersonMusic

Thanks for those tests!

The big takeaways seem to be:

  • UDP must be sync'd for each test, otherwise it's sporadically unstable, even at small batch sizes.
  • Batched TCP is ideal, saturating around batch sizes of 15
  • need to watch out that your tests, if running largely concurrently, aren't consuming all the buffers at once.

Would you agree?

(BTW, I tried on macos 14.6, M1 Pro, but wasn't convinced the protocol was actually switching, stats looked identical)

mtmccrea avatar Sep 11 '24 13:09 mtmccrea

I do agree :)

I think it's also worth saying that in one test with 10000 calls to loadToFloatArray batching in groups of 2 reduces the duration from 100 to 6 seconds. So this is definitely worth it for large inputs.

BTW, I tried on macos 14.6, M1 Pro, but wasn't convinced the protocol was actually switching, stats looked identical

... That's worrying.

JordanHendersonMusic avatar Sep 11 '24 13:09 JordanHendersonMusic

BTW, I tried on macos 14.6, M1 Pro, but wasn't convinced the protocol was actually switching, stats looked identical

... That's worrying.

Looking closer at the console during boot, I see:

NetAddr-Connect failed with exception: connect: Connection refused

This happens on both the 3.13 and develop build. I can't troubleshoot more atm...

mtmccrea avatar Sep 12 '24 09:09 mtmccrea

Looking closer at the console during boot, I see:....

So that error might actually be fine, when you make a server, SC has 10 attempts to do so. If it's just one of these errors and boots as normal it's probably just the TCP connection taking a moment - I'm not super familiar, but i believe TCP does an async call to confirm the connection was established?

JordanHendersonMusic avatar Sep 12 '24 09:09 JordanHendersonMusic

Maybe collect some "broken runs"?

https://github.com/supercollider/supercollider/actions/runs/10854848953/job/30128072583#step:7:7077

 	[0/1]	TestServer_boot:test_waitForBoot (5.5s)
		 ! waitForBoot should allow starting a synth.

capital-G avatar Sep 13 '24 20:09 capital-G

I have a promising approach by changing how the jobs are ordered and splitting the tests over several jobs. I'll send a PR once it's ready. https://github.com/frenchy64/supercollider/actions/runs/11746865238

frenchy64 avatar Nov 08 '24 17:11 frenchy64

Re: TCP

TL;DR TCP is more reliable once connected, but the connection process is more fragile in our case.

Just for reference, there are existing issues related to TCP: #5337, #932. I'm not sure if they do impact the problems I listed below, but they might...

Also, I'd contest that we wouldn't ever need to server.sync (if I'm reading correctly that this is implied). We still need it to wait for the server to allocate buffers etc, so most of it will need to still be there, IIUC.

I've done some tests recently trying to switch the whole testsuite to TCP by adding ServerOptions.defaultValues[\protocol] = \tcp; to the GHA test runner and observed the following:

  • The first attempt just ran out of connection attempts. Sometimes the server takes a while to book in CI and the max tcp connection attempts runs out (it's currently 20 x 0.2 sec, so 4 seconds total); I tried to increase maxAttempts to improve this
  • Then I got this weird error:
(...) // previous tests
UNIT TESTS FOR 'TestUGen_RTAlloc' COMPLETED
There were no failures

	Test: test_allocFail_setDoneFlag
TestUGen_RTAlloc : setting clientID to 0.
ERROR: Error in netAddrSend: Broken pipe
ERROR: Primitive '_NetAddr_SendMsg' failed.
Failed.
RECEIVER:
Instance of NetAddr {    (0x5654d72257f8, gc=70, fmt=00, flg=00, set=02)
  instance variables [4]
    addr : Integer 2130706433
    port : Integer 57110
    hostname : "127.0.0.1"
    socket : RawPointer 0x7fd7ec015c00
}
PATH: /home/runner/work/supercollider/supercollider/testsuite/scripts/sclang_test_runner.scd
CALL STACK:
	MethodError:reportError
		arg this = <instance of PrimitiveFailedError>
	Nil:handleError
		arg this = nil
		arg error = <instance of PrimitiveFailedError>
	Thread:handleError
		arg this = <instance of Thread>
		arg error = <instance of PrimitiveFailedError>
	Object:throw
		arg this = <instance of PrimitiveFailedError>
	Object:primitiveFailed
		arg this = <instance of NetAddr>
	Server:sendMsg
		arg this = <instance of Server>
		arg msg = [*2]
	< FunctionDef in Method AbstractResponderFunc:oneShot >
		arg args = [*4]
	OSCArgsMatcher:value
		arg this = <instance of OSCArgsMatcher>
		arg testMsg = [*6]
		arg time = 237.231764839
		arg addr = <instance of NetAddr>
		arg recvPort = 53876
		var msgArgs = [*5]
	OSCFuncAddrMessageMatcher:value
		arg this = <instance of OSCFuncAddrMessageMatcher>
		arg msg = [*6]
		arg time = 237.231764839
		arg testAddr = <instance of NetAddr>
		arg recvPort = 53876
	< FunctionDef in Method Collection:collectInPlace >
		arg item = <instance of OSCFuncAddrMessageMatcher>
		arg i = 4
	ArrayedCollection:do
		arg this = [*5]
		arg function = <instance of Function>
		var i = 4
	Collection:collectInPlace
		arg this = [*5]
		arg function = <instance of Function>
	FunctionList:value
		arg this = <instance of FunctionList>
		arg args = [*4]
		var res = nil
	OSCMessageDispatcher:value
		arg this = <instance of OSCMessageDispatcher>
		arg msg = [*6]
		arg time = 237.231764839
		arg addr = <instance of NetAddr>
		arg recvPort = 53876
	Main:recvOSCmessage
		arg this = <instance of Main>
		arg time = 237.231764839
		arg replyAddr = <instance of NetAddr>
		arg recvPort = 53876
		arg msg = [*6]
^^ The preceding error dump is for ERROR: Primitive '_NetAddr_SendMsg' failed.
Failed.
RECEIVER: a NetAddr(127.0.0.1, 5[7110](https://github.com/dyfer/supercollider/actions/runs/14577304744/job/40886355845#step:6:7111))


Server 'TestUGen_RTAlloc' exited with exit code 0.
Booting server 'TestUGen_RTAlloc' on address 127.0.0.1:57110.
terminate called after throwing an instance of 'boost::wrapexcept<boost::system::system_error>'
  what():  remote_endpoint: Transport endpoint is not connected [system:107 at /home/runner/work/supercollider/supercollider/external_libraries/boost/boost/asio/detail/reactive_socket_service.hpp:218 in function 'remote_endpoint']
timeout: the monitored command dumped core
/home/runner/work/_temp/4938a9b6-46f9-4974-838c-db6ccc78b8ba.sh: line 4:  4281 Aborted                 timeout 1200 $SCLANG $SCRIPT_RUN_TESTS $TEST_LIST_PROTO $TEST_LIST_RESULT
  • I also think that we may need to wait for the server to quit to make sure the TCP connection is cleaned up, but I don't have a hard evidence this actually causes issues. Waiting for the server to quit is implemented in #6738
  • When running locally, someTestServer_clientID_booted tests fail when run with TCP. As this is reproducible locally, it could be filed as an issue and addressed separately.
  • After addressing these issues, another run resulted in *** ERROR: failed to open TCP socket: address in use., which may mean that the previous server instance did not quit or clean up properly.
	Test: test_cue_eventConfiguration
TestSoundFile_Server : setting clientID to 0.
WARNING: Server name already exists: 'TestSoundFile_Server'. Please use a unique name
Booting server 'TestSoundFile_Server' on address 127.0.0.1:57110.
JackDriver: client name is 'SuperCollider-01'
SC_AudioDriver: sample rate = 48000.000000, driver's block size = 1024
JackDriver: connected  system:capture_1 to SuperCollider-01:in_1
JackDriver: connected  system:capture_2 to SuperCollider-01:in_2
JackDriver: connected  SuperCollider-01:out_1 to system:playback_1
JackDriver: connected  SuperCollider-01:out_2 to system:playback_2

*** ERROR: failed to open TCP socket: address in use.
This could be because another instance of scsynth is already using it.
You can use SuperCollider (sclang) to kill all running servers by running `Server.killAll`.
You can also kill scsynth using a terminal or your operating system's task manager.
Server 'TestSoundFile_Server' exited with exit code 1.

conclusion...

It seems that we need to check if the TCP implementation is solid and handles repeated connects/disconnects well before we can use it in the test runner.

dyfer avatar Apr 21 '25 17:04 dyfer