autoretrieve
autoretrieve copied to clipboard
application-research
Increase test coverage
Add more unit or integration tests, and hook it up to the ci/cd build pipeline.
Sub tickets:
- [ ] #84
- [x] #39
- [ ] #41
- [ ] #42
- [ ] #60
- [ ] #88
FWIW, I'm pretty hesitant to add too many unit tests (i'm a bigger fan of integration tests) - but, as always, whatever we feel like is the right bit of work.
Do we have an integration test framework that ties this all together?
As a separate point, the biggest concern i have looking at this is that you could write tests forever. I don't want us to pick up a code coverage number, but is there a way to describe a "1-2 week" chunk of defined work here?
I agree with @aronchick, I think integration tests will be more useful for less work. although i think "unit test" is also being used kinda flexibly here.
i'm pretty sure we have mock options for all/almost all of autoretrieve's sub-services. not sure about indexer endpoint, but we can pretty easily write a dummy implementation for it since it's already behind an interface. working on tests is the next item on my list, so i'll have some more details soon.
i don't think this should take up too much time.
I certainly support integration tests-- they are extremely useful. That said, they can be difficult to setup and maintain, flaky in CI, and they're slow. Unit testing is just my personal way of programming, but I know that it doesn't work for everyone. If I can't write a test quickly for my code, I tend to think I need to revisit it's design to make it less complicated or use less dependencies. My typical cycle involves running "go test ./..." on a regular basis so if that takes a long time I get frustrated (though as long as I can "go test ." I'm pretty good).
Truthfully though, I don't care about approach. I just prefer not to ship untested, unreviewed code no matter what it looks like. What I hope we can agree on, based on experience so far, is that lack of testing, lack of CI, and lack of a strong PR process has caused us to ship several rounds of updates to production that broke the site. My hope is going forward, we can have a strong review and testing process, whatever that looks like.
I've updated this to include an integration test ticket but I'm letting @rvagg who's generally going to own this essentially do these in the order he wants to.