neo4j-jdbc icon indicating copy to clipboard operation
neo4j-jdbc copied to clipboard

Implement some test / report that checks which jOOQ API / Cypher DSL API is already covered by tests

Open michael-simons opened this issue 1 year ago • 2 comments

From sql2cypher created by lukaseder: neo4j-contrib/sql2cypher#22

The jOOQ integration tests have some agents that run e.g. during the tests of the parser to make sure all (most) jOOQ API is covered in some way by the parser. That way, when I add new jOOQ API, e.g. for some SQL function, I will not forget to add support for it also in the parser. This doesn't guarantee I'm parsing all the possible syntaxes correctly, but at least it guarantees I don't forget thinking about it.

Such an agent would be useful as well for this module. It could run in 2 ways:

  • As a possibly failing test that fails for as long as we don't cover everything that is possible
  • As a report showing some table with all available API vs all covered API

The benefit is obvious. As both jOOQ and Cypher DSL evolve, we will at least have feedback on what needs to be done here, but we will probably forget in 1 year.

Testing the jOOQ side

As far as the jOOQ side is concerned, it's probably easiest to just enhance the jOOQ integration tests (eventually) with this check. I'll then make some quarterly (?) reports here on what new SQL features could be covered.

Testing the Cypher DSL side

See description above.

michael-simons avatar Jan 17 '24 14:01 michael-simons

Would that use just a code-coverage tool (for a filtered subset of the API/DSL)?

Or a manual check/list of the things that we might be able to cover?

michael-simons avatar Jan 17 '24 14:01 michael-simons

In jOOQ, there's a single DSL API entry point (org.jooq.impl.DSL, see also https://github.com/neo4j-contrib/cypher-dsl/issues/573), and all of the DSL methods throughout the API are annotated with @Support (to list dialect support). That @Support annotation is used for documentation by users, by a compiler plugin that prevents accidental access to unsupported syntax, and by the test suite for various things, including discovering the implicit grammar encoded in the BNF.

As such, we can attach the agent to all methods that are annotated @Support and check if they're all called though some code path. That's probably more precise than some blunt code-coverage tool, but I haven't evaluated all of those, so perhaps they will suffice?

In any case, there also needs to be a list of exceptions, i.e. Cypher syntax that can't be represented in SQL, which won't be covered by these tests.

michael-simons avatar Jan 17 '24 14:01 michael-simons