codeql-cli-binaries icon indicating copy to clipboard operation
codeql-cli-binaries copied to clipboard

codeql run test doesn't match absolute fs paths in graph queries

Open smowton opened this issue 5 years ago • 3 comments

Per @max-schaefer codeql run test should adapt queries that emit absolute FS paths such that they pass when run in a different environment. For example my test emits # 0| [File] /home/chris/codeql-home/codeql-go/ql/test/library-tests/semmle/go/PrintAst/input.go but the test doesn't work when the .expected file uses a truncated relative version of that path, or when it uses a different absolute path and the .actual uses the absolute path appropriate to a different environment.

smowton avatar Jun 17 '20 14:06 smowton

To clarify: codeql run test does this already, but it doesn't seem to work reliably (or at all?) for graph queries.

max-schaefer avatar Jun 17 '20 14:06 max-schaefer

The results comparison for graph queries is not entirely straightforward, and I'm not sure I fully understand the subtleties it has to deal with. @dbartol, do you have a clear enough understanding of the problem here to figure out what would be involved in solving it?

hmakholm avatar Jun 23 '20 12:06 hmakholm

I think the difference is that we only replace the location in graph queries, but we replace all strings with the file location in normal queries.

alexet avatar Jun 23 '20 14:06 alexet