gocryptotrader icon indicating copy to clipboard operation
gocryptotrader copied to clipboard

logger: reduce go routine generation

Open shazbert opened this issue 2 years ago • 1 comments

PR Description

  • ~~Create worker pool balanced to gomaxprocs~~ Have it locked to one worker to keep logs in order.
  • Segregate caller with a buffered jobs channel
  • Increase event pool slice capacity for a wcs (This does not seem to influence memory usage)
  • Reduction in go routine generation all exchanges enabled from master 6511 routines in 30 seconds to branch 1649 routines in 30 seconds
  • Segregate worker pool writers from caller, if the writing process is slow this now does not interfere with the caller it need not know or be impeded if the write fails.

~~NOTE: Future performance improvement and segregation can occur with the newLogEvent method and construction.~~ Should be done now to a better extent.

Fixes # (issue)

Type of change

Please delete options that are not relevant and add an x in [] as item is complete.

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] This change requires a documentation update

How has this been tested

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration and also consider improving test coverage whilst working on a certain feature or package.

  • [ ] go test ./... -race
  • [ ] golangci-lint run
  • [ ] Test X

Checklist

  • [ ] My code follows the style guidelines of this project
  • [ ] I have performed a self-review of my own code
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation and regenerated documentation via the documentation tool
  • [ ] My changes generate no new warnings
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] New and existing unit tests pass locally and on Github Actions/AppVeyor with my changes
  • [ ] Any dependent changes have been merged and published in downstream modules

shazbert avatar Jul 07 '22 06:07 shazbert

Unfortunately, looks like there's still some racey races and test issues: https://github.com/thrasher-corp/gocryptotrader/runs/7463222648?check_suite_focus=true#step:7:4533 https://github.com/thrasher-corp/gocryptotrader/runs/7463222648?check_suite_focus=true#step:7:4262 https://github.com/thrasher-corp/gocryptotrader/runs/7463222648?check_suite_focus=true#step:7:3663 https://github.com/thrasher-corp/gocryptotrader/runs/7463222648?check_suite_focus=true#step:7:5026 https://ci.appveyor.com/project/thrasher-/gocryptotrader-vxsws/builds/44247500#L6486 https://ci.appveyor.com/project/thrasher-/gocryptotrader-vxsws/builds/44247500#L7097 https://ci.appveyor.com/project/thrasher-/gocryptotrader-vxsws/builds/44247500#L7365 https://ci.appveyor.com/project/thrasher-/gocryptotrader-vxsws/builds/44247500#L7900

2022-07-22T06:24:25.9203656Z --- FAIL: TestStartStopDoesNotCausePanic (0.06s)
2022-07-22T06:24:25.9204381Z     engine_test.go:86: failed to setup global logger. invalid worker count must be >= 1, received '0'

The randomness in how they all show up is the fun part of globals 🦀

gloriousCode avatar Jul 24 '22 23:07 gloriousCode