Test class platform event message limit of 500

[arbokrad]

Package Edition of Nebula Logger

Unlocked Package

Package Version of Nebula Logger

4.6.11

New Bug Summary

There appears to be a limit of 500 platform event messages that can be published as part of a test class:

@IsTest(IsParallel=true)
private class TEST_PlatformEventLimit {
	@IsTest
	static void nebula_500_logs() {
		for( Integer i = 0; i < 500; i++ ) {
			Logger.debug(String.valueOf(i) );
		}

		Logger.saveLog();
	}
}

<<snip>>

17:36:58.250 (8597575794)|USER_DEBUG|[639]|DEBUG|TEST_PlatformEventLimit.nebula_500_logs -- 495
17:36:58.250 (8619804359)|USER_DEBUG|[639]|DEBUG|TEST_PlatformEventLimit.nebula_500_logs -- 496
17:36:58.250 (8642440479)|USER_DEBUG|[639]|DEBUG|TEST_PlatformEventLimit.nebula_500_logs -- 497
17:36:58.250 (8669687712)|USER_DEBUG|[639]|DEBUG|TEST_PlatformEventLimit.nebula_500_logs -- 498
17:36:58.250 (8692536382)|USER_DEBUG|[639]|DEBUG|TEST_PlatformEventLimit.nebula_500_logs -- 499
17:36:59.428 (9428795845)|EXCEPTION_THROWN|[EXTERNAL]|System.LimitException: The number of platform event messages published from an Apex test context exceeded the limit of 500.
17:36:59.428 (9481295437)|USER_DEBUG|[2527]|DEBUG|Logger - LogEntryEvent__e save results: (Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=52480136-fab0-4658-b5c8-649d811d8c99;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000001AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=d3dcd0a1-d723-4a9d-a7c8-331bae9680c1;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000002AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=55fee963-a7c9-4853-b6b0-096aa9197e77;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000003AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=ec119814-7b9c-4bfa-a7f4-c7a082abed54;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000004AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=d47a082e-89e8-4e27-9a0a-a187c6a0d1b0;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000005AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=f1040e09-3e33-45ae-9b56-b59a088e4b73;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000006AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=9a979011-98c4-4af0-8c78-48faf102e2e6;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000007AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=329a90ea-4d8e-46cf-bada-4a784f63879c;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000008AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=6d73e9b0-d576-4167-84c0-3111ed6267a7;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx0000000009AAA;isSuccess=true;], Database.SaveResult[getErrors=(Database.Error[getFields=();getMessage=9ee73f7a-71d7-4592-8252-b3aae8fcd461;getStatusCode=OPERATION_ENQUEUED;]);getId=e00xx000000000AAAQ;isSuccess=true;], ...)

Strangely, this limit being exceeded did not actually cause a failure of the calling code, but it did seem to prevent additional logging messages from being actually published. As per a discord conversation on 8/9/2022, this limit comes as a bit of a surprise!

[jongpie]

This is really interesting, and I think it's a great sign that I need to add some tests like you provided in #350 where the test methods try to generate hundreds/thousands of log entries.

I think I'm going to also move forward with the idea we discussed on discord of providing a new LoggerParameter__mdt custom metadata record that would control if, by default, Nebula Logger actually publishes platform events during unit tests - some orgs would want to have this new record set to true to maintain the current behavior, but for orgs/specific tests that don't care about the platform events being published, the custom metadata record's value could be changed to false to avoid publishing the platform events.

I'm wrapping up the new release for v4.8.0, and then I'll spend some time prioritizing the list of open issues, I'll let you know once I have an ETA on getting this implemented.

[jamessimone]

@arbokrad just curious if you're able to repro this issue these days? We've been doing some benchmarking work, some of which has already been released, and I was able to publish at least 1200 log entry events (with some caveats - see the linked test class for more info, but essentially you can get significantly better bulk performance in tests by disabling the System.Limits logging fields in particular).

[arbokrad]

I confirmed today that this (and #350 ) are present in the the most recent release of nebula (4.13.4)

[arbokrad]

Note that this problem appears to be triggered when saveLog is called - the actually logger methods are unaffected.