brownie-strategy-mix icon indicating copy to clipboard operation
brownie-strategy-mix copied to clipboard

Test enhancements

Open jmonteer opened this issue 3 years ago • 5 comments

WIP

Working on improving the tools that a strategist have to test a new strategy. The objective is (i) reduce time that we spend setting up tests, (ii) test strategies in a more comprehensive way (iii) ...

This will include:

  • [x] more detailed check list of tests to be done
  • [x] more detailed templates for those tests than can be thought of in advance
  • [x] tests run isolated in a fresh fork every time
  • [x] utils like unit converters, standard debugging prints or standard checks to be done after certain actions
  • [ ] improvement of the time to run tests to avoid lengthy waits that make strategists waste time while writing tests
  • [x] parametrized tests for testing different tokens (or other scenarios) using the same tests
  • [ ] integration with live vaults to run tests in production
  • [ ] integration with Tenderly to improve debugging

Please add any other suggestions or requests

Linear ticket: https://linear.app/yearn/issue/BRN-75/test-suite-enhancements

jmonteer avatar Jun 22 '21 10:06 jmonteer

I will most probably open a new branch for a deeper than initially thought brownie <> tenderly integration

jmonteer avatar Jun 28 '21 11:06 jmonteer

So far:

  • added healthcheck support using -1 dependency
  • added strat_ms fixture to act from strategists ms
  • parametrized token fixture and added a list of supported tokens (only including those that are most important atm)
  • modified fixture "amount" to be a deposit of around $1m, independently of the token
  • created a fixture with whales for each token used
  • added a clean fixture to snapshot and revert to a clean state before each specific test
  • added a couple of fixtures regarding registry, to support the automation of getting latest production vault for each token
  • added a fixture to get a cloned instance of the strategy being tested
  • added a fixture to always withdraw after each test, to be sure that withdrawing is possible in any situation, unless explicitely avoided
  • added a test_airdrop as requested in #29
  • added a test_clone to test that clones are correctly created and are harvestable
  • modified test_harvest to add a case with both profits and losses, to ensure accounting is working
  • created utils suite with a) actions: repetitive actions like deposit in a vault from a user b) checks: standard checks with asserts c) utils: unit converters or sleeps, to avoid repetitive code and improve readability
  • added a test file for manual functions (for those strategies that implement them)
  • made a bit more detailed test for test_migration
  • added a test for restricted functions so strategist can be sure they thought through the correct set up
  • added a test for test_revoke_with_profit to be sure that even when profit is added to the strategy, it is revokable (we've had problems in the past with this)
  • added a test for a strategy unwind

Tenderly network management and debugging might be included in another PR, depending on timing

It would be nice to have your comments @salazarguille @storming0x @Grandthrax @poolpitako @fameal @orbxball

jmonteer avatar Jun 28 '21 11:06 jmonteer

Great job @jmonteer! qq, are you planning to add unit tests to cover different scenarios with the health check?

salazarguille avatar Jun 28 '21 22:06 salazarguille

yeah would love to add those. still have to do some catch up with healthcheck tho

jmonteer avatar Jun 29 '21 15:06 jmonteer

@salazarguille added test_healtcheck

please add any other ideas you feel would be useful!

jmonteer avatar Jul 01 '21 17:07 jmonteer