Metrics

[dialex]

• https://inspguilfoyle.wordpress.com/2013/10/12/right-measures-measured-right/
• http://blog.codepipes.com/testing/software-testing-antipatterns.html#anti-pattern-6---paying-excessive-attention-to-test-coverage
• coverage bad https://failfastmoveon.blogspot.com/2022/05/why-test-coverage-targets-backfire.html
• https://medium.com/avitotech/e2e-ios-ui-tests-lots-green-on-pr-24f30e08e81d
• https://augustl.com/blog/2019/best_bug_predictor_is_organizational_complexity/
• https://blog.theactframework.com/testing-metrics-why-measure-what-to-measure-and-how-to-measure?utm_campaign=Software%2BTesting%2BWeekly&utm_medium=email&utm_source=Software_Testing_Weekly_28
• https://club.ministryoftesting.com/t/what-are-software-quality-metrics/46781
• https://speakerdeck.com/jennydoesthings/the-only-good-quality-metric-is-morale
• https://itnext.io/its-a-trap-avoid-focusing-on-vanity-metrics-in-software-testing-7a627c7848

[dialex]

Coverage

• http://labs.ig.com/code-coverage-100-percent-tragedy
• https://danashby.co.uk/2019/02/14/code-coverage-vs-test-coverage/
• https://knowledgetester.org/2018/01/08/code-coverage-dos-and-donts/
• https://dev.to/teamcoder/the-problem-with-high-test-coverage-4dh
• http://www.developsense.com/blog/2018/06/test-cases-and-coverage/
• https://www.eviltester.com/blog/eviltester/2020/04/09-how-to-document-coverage-of-automation/
• http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf
• http://www.developsense.com/articles/2008-10-GotYouCovered.pdf
• http://www.developsense.com/articles/2008-11-CoverOrDiscover.pdf
• http://www.developsense.com/articles/2008-11-AMapByAnyOtherName.pdf
• https://testing.googleblog.com/2020/08/code-coverage-best-practices.html
• https://itnext.io/its-a-trap-avoid-focusing-on-vanity-metrics-in-software-testing-7a627c7848
• https://increment.com/reliability/testing-beyond-coverage/

example: single unit test when i=2

array[i] = 'text value'

100% coverage -- our code does not have bugs! WRONG if i=-1 then code will explode

Focus on covering critical piece of code; Focus on testing previous fail/bug scenarios (regression) 0% coverage is for sure bad; 100% coverage is... we can't be sure, depends the use cases

[dialex]

GQA-SourceCodeIndicators-290419-1324.pdf

[dialex]

Here are some examples:

1. Product quality
   • A number of bugs reported by your customers — can tell you how customers see the quality of your service.
   • Product reviews score (e.g. on Apple/Google stores) — can tell you how your end-users see your service.
   • NPS (Net Promoter Score) rate — similarly, it can tell you how your end-users or customers are satisfied with your service.
2. Team quality
   • Average time of open bugs — can tell you about the state of your engineering culture (e.g. Are bugs triaged and prioritised? Is tech debt tackled regularly?).
   • A number of ignored tests — similarly, can tell you about the culture and respect to tests.
3. Code quality
   • A number of release rollbacks — can tell you about the state of your software development life cycle process.
   • A number of hotfixes deployed — similarly, it can tell you about the state of your software development life cycle process.
4. Test quality
   • Test flakiness rate — can tell you how stable and reliable your automated tests are.
5. System performance
   • Uptime % of your services — can tell you about your system’s reliability.

https://morethantesting.com/blog/ask-me-anything/