json-path-comparison icon indicating copy to clipboard operation
json-path-comparison copied to clipboard

Should the implementations be scored?

Open ohler55 opened this issue 4 years ago • 6 comments

Should the implementations be scored for a quick glance summary?

It could be as simple as making the score the sum of the results of each test. Something like:

✓ The result of this implementation matches the consensus of results. 1.0 points ✗ The result does not match the consensus. 0.0 points ➚ and ➘: no clear consensus amongst the implementations, but ➚ indicates a majority (and possible future consensus). majority gets 0.7 points, minority get 0.3 points e The implementation failed executing the query. 0.0 points

Comments?

ohler55 avatar Jan 05 '21 16:01 ohler55

I guess there is no interest in this. You are welcome to close.

ohler55 avatar Mar 06 '21 01:03 ohler55

Hey, thanks for checking back. Happy to keep this for a while, as I remember this question coming up before!

cburgmer avatar Mar 06 '21 12:03 cburgmer

I would love to see this happen.

bwl21 avatar Apr 05 '21 07:04 bwl21

@cburgmer if you think it is worth doing I can put together a PR for the scoring. What do you think?

ohler55 avatar Apr 05 '21 14:04 ohler55

I like the idea of scoring against the consensus. In the fulness of time, supposing a standard is defined, perhaps we could also score against the reference implementation.

glyn avatar Apr 05 '21 15:04 glyn

If I could add one further suggestion, of the 41 implementations, two early ones have had disproportionate impact on the JSONPath ecosystem, and to some extent account for the divergent evolution of later ones - Jayway Java and Goessner Javascript. It would be interesting to see scoring against these two important ones as well.

danielaparker avatar Apr 05 '21 16:04 danielaparker