Merge issue43 hotfix

[carterian8]

Closes #43. Watcher::add_appointment spawns an async tokio::task to handle cases when Watcher::store_triggered_appointment runs into bitcoind connectivity issues.

Introducing this async change caused a substantial ripple effect through the code with the most notable being an update to the Carrier to use tokio::sync::Notify to listen for changes to the status of bitcoind_reachable (please see #43 for justification). Other changes just further support the constraints introduced by incorporating more async operations.

All unit tests were updated to verify these changes and an additional test, Watcher::test_add_appointment_bitcoind_unreachable, was added to verify the correctness of the specific corner case presented in #43.

[carterian8]

@sr-gi

Yeah I agree this fix is a bit meaty for the problem it addresses.

Regarding lifetimes and Rust, I'm not sure you'll find a clean solution combining async and self. I stumbled across some workarounds but, based on my understanding of Rust, referring to self within an async call violates the static lifetime requirement and calls for some unconventional techniques.

In my opinion, if you'd like to use async to fix this issue, its going to require a redesign of the magnitude presented here.

Maybe we can chat when you're ready and converge on a solution you'd be happy with.

[sr-gi]

@carterian8 I chatted about this recently with @TheBlueMatt since it goes above my current understanding of Rust. He mentioned that Rust needs to make sure that the object is pointed by &self will exist in the same place in memory forever (mainly what static does by my understanding, hence the compiler's suggestion). He also mentioned that the simplest way to make this work may be to replace &self for something like us: Arc<Object>.

I may try to make this work in a simpler scenario, like for the watchtower-plugin retrier to convince myself how this may work (i.e. convert it into an object given it is currently spawning async tasks).