1169 sync engine with block cache trait

[Oscar-Pepper]

trafficstars

• Builds on #1184, implementing an async wallet synchronization function
• Replaces FsBlockDb with BlockCache for generic implementation of block caches, solving cyclic dependencies.
• Implements BlockCache for FsBlockDb and TestBlockCache (formerly BlockCache)

[str4d]

This needs to be rebased now that #1184 has been merged.

[Oscar-Pepper]

This needs to be rebased now that #1184 has been merged.

Ok thanks. I think its best I separate this PR into more managable chunks. I will try to fit this in soon so it doesnt get too far behind

[str4d]

This PR has now been blocking parts of zcashd deprecation for almost 4 months. I'm going to do the rebase myself and open a new PR.

[zancas]

This PR has now been blocking parts of zcashd deprecation for almost 4 months. I'm going to do the rebase myself and open a new PR.

Zingo Labs reconsidered its strategy and significantly reallocated resources with the April 30th release of Zashi.

I have done my best to make my position clear and consistent.

Prior to the entrance of Zashi into the competitive wallet market it was in our interest to allocate our limited resources towards librustzcash functionality that's useful across the ecosystem. We bore the significant personal cost when this bet went poorly for us with the introduction of the new Binance requirements.

In spite of the fact that our primary funding stream was (and is) payment on delivery for specified results, we allocated precious developer time to unpaid contributions to community-critical resource like librustzcash, and zebrad.

In the context of Zashi's release it's essential that we differentiate ourselves to survive. Pro-bono work on the common sync core is not just less efficient than more targeted efforts on our own sync engine, it is actively counter to this necessary differentiation.

This inefficient allocation of librustzcash development effort is certainly not the alignment I would have preferred.

Any implication that our efforts are not efficient, and consistent, should be considered both in light of the Zashi shift in our operating environment, and in light of our substantial track record of FOREGOING UNEARNED PAYMENT until such time as we complete our contracts.

Thanks for your thoughtful attention, --Za

[str4d]

Zingo Labs reconsidered its strategy and significantly reallocated resources with the April 30th release of Zashi.

I have done my best to make my position clear and consistent.

Had you mentioned this here or elsewhere, I would have taken over the PR much earlier. As it was, I was trying to ensure you were included in the development process, something you had at the time expressed significant interest in, and was taking pains to ensure I was not doing work in parallel that would obsolete work you were presumably (given no updates here to the contrary) working on at your own pace.

I will not make this mistake in future.

[zancas]

https://forum.zcashcommunity.com/t/zcashd-deprecation-needs-a-champion/48580/22

[zancas]

I can check with Zingoistas maybe someone would be happy to pick this up. Certainly we can add it to our work if there's compensation. Maybe the ECC would like to contract with us.

[str4d]

As I said above, I've already picked this up myself, due to its necessity for zcashd deprecation. I'm currently on PTO though and not working on it until at least end of next week, so if anyone else is likely to make progress in the next week they are more than welcome to do so.

[Oscar-Pepper]

As I said above, I've already picked this up myself, due to its necessity for zcashd deprecation. I'm currently on PTO though and not working on it until at least end of next week, so if anyone else is likely to make progress in the next week they are more than welcome to do so.

Hi str4d, I apologise for my part in the way this has turned out, it was not my intention to cause blockers on zcashd deprecation or cause you frustration. I was not aware of this until now as I missed your comment from three weeks ago.

I think zancas has already mentioned we are limited on resources and funds. The zip317 release turned out to be a lot more work than expected as we replaced a large % of our legacy code along with a massive increase in test coverage. I had to pull off sync to prioritise this work. I agree this should have been communicated better.

This PR was initally developped alongside #1192 . As i mentioned in #1192 and also on signal and discord, this PR was intended to replace #1184 so you wouldnt have to repeat the work i had already done to get #1184 building and ready for merge. At the time, this PR was complete and only draft because #1192 was still under review and had multiple change requests which lead to me rebasing and re-working this PR multiple times. Simplifying this rebasing was actually the reason why i squashed my commits which was one of your conerns.

I understand you were very busy with the zashi release but I feel the lack of communication at this time is also a factor here.

In this context, I hope you see that I wasnt so much renaming 'db_data', I was intending these names to be the first to merge and to better reflect the fact that this sync engine was also going to be used by wallets like zingo where the wallet data is not purely a database persistance layer like sqlite. In heinsight I should have left it alone.

I am away for the next week but I will prioritise this PR when i return if you have not got onto it before hand.

[Oscar-Pepper]

superseded by #1535