anax
anax copied to clipboard
open-horizon
Increase Unit test coverage in anax
The intention of this issue is to increase unit test coverage of all anax components. A good target for coverage is about 80%. There are only a few components in anax that have this much coverage. The introduction of exchange handlers into the implementation, enables code to be unit tested that was previously untestable due to the external HTTP dependencies. This should enable components like governance and agreement making functions to be unit testable. Also consider restructuring code in order to make it unit-testable.
Checkout https://blog.golang.org/cover for instructions on how to figure out which lines of go code are covered by unit tests, and which aren't. The tools presented in this article are VERY helpful to learn.
@dabooz , @linggao, @joewxboy could you please assign this issue to me? I am interested in it. Thank you
@dabooz Yes I am still working on it, and yes I have some questions.
First of all, I read the article in https://blog.golang.org/cover introducing some instructions and toolings, which are very helpful to unit test coverage check and improvement.
Secondly, "exchange handlers" you mentioned means the exchange-api component? What you proposed was to set up an "exchange-api" component including dummy data during the unit test period, so the "exchange-api" can provide HTTP service for HTTP dependency component in Anax such as governance?
Thirdly, I tried to restructure code in order to make some components of Anax unit-testable. For example, adding function pointers for independent functions that have HTTP dependencies to unlock dependencies, and insert test points at the same time. The problem is that new changes in source code will result in a lot of regression tests. My question is if it is really necessary to make some components of Anax unit-testable with this approach, can I create new issues for the specific components as sub-issues of this issue? in order to reduce review work and avoid too many changes in one commit.
Fourthly, I got a solution suggestion from China-SIG colleagues to solve the HTTP dependencies issue of the Anax unit test. The proposal is to set up a simple HTTP server once users start unit test, providing responses with JSON format data to simulate exchange-api returns. I am still working on PoC and once I finish it, I will present more details. @joewxboy suggest to confirm with you first and get your feedback before we dive deeply, so could you please let me know your opinion about this solution? Thank you.
Hello. Is there an issue which is being worked on against this PR? If not I would love to take on this. @gripson please let me know! Thank you!
Welcome @shubhank-saxena. You can refer my previous reply here. Since I still work on it and the progress is not productive, it will be nice if you would like to join, and please feel free to propose any new solutions.
@gripson Let me try to answer a few of your questions:
- Exchange handlers are here: https://github.com/open-horizon/anax/blob/master/exchange/handlers.go. They allow a piece of unit test code to replace the exchange with a local function call. An example of using an exchange handler for unit tests is here: https://github.com/open-horizon/anax/blob/master/agreementbot/nodehealth_manager_test.go#L34. Look especially at the use of this function: https://github.com/open-horizon/anax/blob/master/agreementbot/nodehealth_manager_test.go#L330 by the unit test. You will see that it enables the unit test to make a call to the exchange, which is actually implemented locally. If you dig deeper, you will also see that at runtime, when the real agent is running, the code uses an HTTP handler that actually calls the exchange. We can implement a lot more unit tests with this strategy.
- restructuring code: what is needed is to change code that DOES NOT use the exchange handler to start using it, so that we can remove direct dependencies on making HTTP calls.
- I dont like the idea of using a real HTTP server to support unit tests. I know it's a common thing to do, but that doesn't mean it's the right thing. These things are fragile and often cause more problems than they are worth, especially in this project where the unit test will require the "fake" HTTP exchange to be bootstrapped with a number of artifacts that are specific to test. This means lots of setup and tear down for every test. Further, testing against a real HTTP service is not really a unit test, it's more of a function or system test. We have those kinds of tests in the built in test environment (e2edev) so we dont need the unit tests to run against a real HTTP exchange.
@dabooz Thank you very much for your explanation and the examples you provide. The task is clear for me now.
Hello there, I'm looking to take up some beginner-friendly tasks on this project. I can help out with writing unit test cases @gripson for certain portions. Please suggest the parts I can take up. 😄
Hello @zerefwayne ! For starters and how to pick up the parts, you can do the following -
- Run
make test. This will run all the unit tests component wise. - There are some components which have zero coverage (ie- they have no tests written)
- There are some which have coverage around 15-20% (probably 1-2 tests written)
- You can pick any of the components and start to write tests for it. We can use this PR as to make sure that none of us is working on the same set of files, thus we can be more productive and complete all the tests in an orderly manner!
There are some specific test questions that are asked by @gripson above, you can rrefer to them if you have decided to pick up test for exchange handlers. Hope this covers up everything. Did I miss something @gripson ? Please feel free to add as I am a beginner too!
Hello, I would like to join this issue. I would really appreciate any tips for beginning to contribute to it!
