feat(rust): Connect consumer to clickhouse
This is an attempt to get the consumers to actually write to clickhouse on submit. A challenge here was our clickhouse client is async, so I had to modify the ProcessingStrategy to use an async submit function. This change had ripple effects all across the board.
The quick solution in many places where the submit function was called in sync functions was to use block_on but this would sometimes hang, so I avoided block_on and used async await + tokio::spawn.
In some parts of the code, I had to replace mutexes with async-aware mutexes to make sure that we are not holding the locks synchronously in the submit async function.
Codecov Report
Patch coverage: 100.00% and no project coverage change.
Comparison is base (
46b90d2) 90.55% compared to head (ee7676d) 90.55%.
:exclamation: Current head ee7676d differs from pull request most recent head 8736a89. Consider uploading reports for the commit 8736a89 to get more accurate results
Additional details and impacted files
@@ Coverage Diff @@
## master #4437 +/- ##
=======================================
Coverage 90.55% 90.55%
=======================================
Files 806 806
Lines 39739 39742 +3
Branches 255 255
=======================================
+ Hits 35985 35990 +5
+ Misses 3720 3718 -2
Partials 34 34
| Impacted Files | Coverage Δ | |
|---|---|---|
| snuba/clusters/cluster.py | 96.34% <100.00%> (+0.04%) |
:arrow_up: |
| snuba/consumers/consumer_config.py | 98.85% <100.00%> (+0.01%) |
:arrow_up: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
looks like this was completed in https://github.com/getsentry/snuba/pull/4490