FrameworkBenchmarks icon indicating copy to clipboard operation
FrameworkBenchmarks copied to clipboard

Frameworks with versus=None get compared to "aah"

Open bdw429s opened this issue 4 years ago • 1 comments

This ticket is from the following discussion, which has received no replies in 3 weeks:

https://github.com/TechEmpower/FrameworkBenchmarks/discussions/6405


I've noticed that frameworks with this in their benchmark_config.json

        "versus": "None"

seem to get compared to the ahh framework in the "Framework overhead" tabs. Is this a misconfiguration or a bug in the benchmark? I would expect a framework with no "versus" wouldn't have overhead comparison at all.

An example would be the new cfml and cfml-adobe frameworks, though this seems to affect over 40 frameworks.

image

bdw429s avatar Mar 18 '21 19:03 bdw429s

@bdw429s Even if you think something is potentially a bug, I'd rather have it in the issue tracker than in discussions. It will get noticed quicker by us. The discussion alerts don't seem to be consistent yet.

Nonetheless, we are extremely busy with clients atm and will get back to you as soon as we can on all your reports.

NateBrady23 avatar Mar 18 '21 20:03 NateBrady23