Slow generation on .NET 6

[paoloambrosio]

When switching from .NET 5 to .NET 6, the Generate method takes over an order of magnitude longer, even for very simple records (e.g. Foo(int Bar)).

Job	Runtime	Mean	Error	StdDev	Ratio	RatioSD
.NET 5.0	.NET 5.0	7,574.85 ns	45.050 ns	42.139 ns	1.00	0.00
.NET 6.0	.NET 6.0	367,589.88 ns	2,163.086 ns	2,023.352 ns	48.53	0.27

Bogus does not show the same performance difference as AutoBogus:

Job	Runtime	Mean	Error	StdDev	Ratio	RatioSD
.NET 5.0	.NET 5.0	82.71 ns	0.111 ns	0.104 ns	1.00	0.00
.NET 6.0	.NET 6.0	71.28 ns	0.378 ns	0.353 ns	0.86	0.00

I have created a Gist with the code used to generate this benchmark.

It sounds odd that no one has noticed it before, so perhaps I'm using it wrong?

[pdevito3]

@paoloambrosio this is interesting. Haven't noticed this myself, but also haven't looked at it to be in a place where I'd notice per say either. Was there a particular fix you added to make it work better?

Also fwiw, I ended up publishing a forked version of Autobogus given @nickdodd79's inactivity. It was focused on updating to .NET 6 and has pretty minimal changes, but theoretically helps this by now using .NET 6 if you want to give it a try?

Not sure how active I'll be maintaining it and really hoping nick takes things back over, but it's seeming more and more abandoned.

In the meantime, if there's an easy add for this feel free to submit a PR or let me know what it is and I'll try and get it in at some point.

[paoloambrosio]

Was there a particular fix you added to make it work better?

Unfortunately, I didn't find a way to fix it 😞 I simply noticed that tests weren't failing anymore when sharing a single generated instance, so I'm just generating it once per test class, instead of once per test as I had to do before.

Also fwiw, I ended up publishing a forked version of Autobogus given @nickdodd79's inactivity. It was focused on updating to .NET 6 and has pretty minimal changes, but theoretically helps this by now using .NET 6 if you want to give it a try?

I can't see the public source code on GitHub (dead links in the NuGet page). If you want to try it yourself, the gist above contains two files: the C# project and the BenchmarkDotNet test.

[pdevito3]

Yeah, realized i typoed the repo pointer. Will be updated next time I do a release. Here's the updates: https://github.com/pdevito3/AutoBogus

[paoloambrosio]

Tried the benchmark and I get similar results with both AutoBogus 2.13.1 and AutoBogusLifeSupport 2.14.0. To be honest I didn't see anything in the code to suggest that it would behave differently in this case, but it was worth a try :)

[pdevito3]

Yeah, didn't have any updates targeting something like this so it was definitely a long shot, but good to know!

[sergey-fuflygin]

Hey gents, I can confirm the issue. We are using AutoBogus 2.13.1 in our project and I raised a PR to upgrade it from .NET 5 to .NET 6. Pipeline failed because of timeout. When I was investigating the core issue, I realized that AutoFaker generates fake data much slower.

That's why I had to:

1. Move AutoFaker<T>().Generate() out from SetUp method. It is called once per Fixture now, not once per test.

2. Limit depth by using .WithRecursiveDepth and .WithTreeDepth

[CaptaiNiveau]

I fixed this issue locally and found a bug with lists as well.

Will try to make sure nothing breaks with these changes (can't execute .NET Framework 4.5 tests on Linux though) and post a pull request when I'm done.

Basically, the issue lies in the ReflectionHelper, which recursively checks all interfaces for collections every time. I let it short circuit if it finds one of the base types with predefined generators (currently only used as last option) and also optimized the collection reflection part. The second part is a bit confusing and also has a bug with custom List types. For that I'll add additional test cases and guard against them.

[pdevito3]

@CaptaiNiveau -- I think this repo is abandoned -- if you want to submit it to my fork I'll get it in.

Cc @Soenneker

[soenneker]

@pdevito3 thanks for the tag.

After profiling it's pretty clear that reflection is one of the major time sinks in the library.

Thus I've been building a reflection caching library, that once complete, will feed into the updated version of AutoBogus: soenneker.utils.autobogus