Mock out the filesystem entirely?

[donquixote]

Hi, I have been experimenting with class loader benchmarks some time myself, and came to the conclusion that the biggest source of statistic variation is probably the filesystem.

You already removed every require/include.. so what about also removing or mocking out file_exists() ?

The idea would be to measure the time for the basic algorithm (foreach loops and string operations) and then count the number of file_exists() and require. Where require is probably pointless, because this is going to be the same with each loader.

This can be done either by giving each loader a MockFilesystem, or by having a class with static methods, e.g. MockFilesystem::file_exists(). I'd say the mock object is more elegant.

There could also be a version of the MockFilesystem that actually does the file_exists(), and another one which doesn't.. this would add one level of indirection, but maybe that's ok? (because it is evenly distributed, every loader has to go through it)

[Seldaek]

I sort of agree, but how do you quantify file_exists calls then? That's pretty hard. I think it has to remain fairly realistic to have any value. But yeah maybe benchmraking with/without would be the best to see the impact.