Offering help: Improving Unit Tests

[erlapso]

Hi everyone! Big fan of this repo.

I'd like to help improve the test coverage using CodeBeaver, an AI agent that I built (it's free for open source!). To demonstrate what's possible, I've created a PR: https://github.com/CodeBeaverAI/compress/pull/1 that adds comprehensive tests to the last Pull Request that was opened here: huff0: Improve fuzz base set.

What the PR demonstrates:

• Added 2 test files: huff0/decompress_test.go and gzhttp/writer/gzkp/gzkp_test.go.
• Increased coverage for gzhttp/writer/gzkp/gzkp_test.go to 100%, improving it by 30%.
• Found 1 bug related to the SetHeader method in gzhttp/writer/gzstd/stdlib.go, which could cause compile-time errors. Detailed fix suggestions are included in the PR.

On top of that: If you check the PR you will see that CodeBeaver does not only go for coverage - it also adds all the edge cases it can think of (we make it iterate 3 times on that alone).

How it works:

CodeBeaver analyzes your code changes in PRs

• Automatically generates and updates tests
• Opens PRs with new/updated tests
• Helps catch bugs before they reach production

Next steps

If this interests the maintainer team, I'm happy to:

• Walk through how the tests were generated
• Help set up automated test generation for future PRs (takes ~5 minutes)
• Focus on specific areas you'd like to improve coverage for

Let me know what you think! Happy to adapt this approach based on the project's needs.

[klauspost]

I am always open to things that can reduce the amount of regressions and new bugs.

Right now it is a bit hard to commit to a meeting.

My main concern is tests that are relying on implementation, though the generated tests look fine, except the formatting.

[erlapso]

Hey Klaus, thanks for the reply! Quick follow up question for me to understand:

My main concern is tests that are relying on implementation

If I get your point right, you are worried that the tests will try to test the implementation and not the requirements behind it, correct? We thought a lot about this. What CodeBeaver is instructed to do is consider the comments at the start of the function as the requirements, rather than the implementation. So if the functions have a good enough explanation, the tests are written with that in mind. That's how it ends up being able to spot bugs when writing tests.

Btw, there is no need for a meeting actually, you can use the open-source version here or just sign up with GitHub on the hosted version that takes ~5 minutes and it's free for open source.

[klauspost]

@erlapso Sounds great. I will give it a whirl!