roles icon indicating copy to clipboard operation
roles copied to clipboard

Bisq Bitcoinj Fork Maintainer

Open cbeams opened this issue 7 years ago • 22 comments

This role is responsible for maintaining Bisq's fork of bitcoinj at https://github.com/bisq-network/bitcoinj. This includes but is not limited to doing big, valuable work like implementing Bisq's needs around SegWit and integrating that support into Bisq (see https://github.com/bisq-network/exchange/issues/1139).

This role is (or at least can be) more than narrowly bitcoinj-specific: we need one or more contributors to can stay on top of all Bisq's Bitcoin needs, to stay abreast of what's happening at the protocol level, to get proactive about things.

Note that Bisq also maintains a federation of Bitcoin Core nodes as a service to its users for the purpose of mitigating privacy risk caused by bitcoinj's approach to bloom filter-based SPV. The contributor playing this role could also become one of the @bisq-network/btcnode-maintainers (see #66) and help tune settings for all the nodes in our federation, maintain common bitcoin.conf files for the set of @bisq-network/btcnode-operators (#67) that run them, etc.


Docs: none, other than the above Team: @bisq-network/core-maintainers Primary owner: @oscarguindzberg

cbeams avatar Sep 04 '17 12:09 cbeams

2018.04 report

Following roughly the BitcoinJ mailinglist or updates about Segwit. Seems there is still no release in sight. Looking out for a dedicated developer for the Bitcoin part, but so far nobody has showed up.

ManfredKarrer avatar May 01 '18 14:05 ManfredKarrer

2018.05 report

Updated BitcoinJ to fix a wrong value for dust limit and filter out the confusing alert warning caused by old bitcoin nodes. Followed a bit discussion on the Bitcoin mailing list where they are discussing to fade out support for BIP 37 (bloom filter).

ManfredKarrer avatar May 30 '18 15:05 ManfredKarrer

2018.06 report

Beside a small commit for a log level change nothing to report.

ManfredKarrer avatar Jun 30 '18 15:06 ManfredKarrer

Update: Acting as @bisq-github-admin-1, I've set up access rights to our Bitcoinj fork, as well as our libdohj and btcd-cli4j forks such that @bisq-network/core-maintainers have write access to the repositories. This was to cut down on creating additional GitHub teams where they're not really necessary, and to reflect the fact that in practice, it is the core maintainers (particularly @ManfredKarrer as primary) who are caring for these forks for the time being. Assuming someone does come along to take care of our Bitcoinj fork, we can set up a dedicated team at that time. The description of this role issue has been updated to reflect.

cbeams avatar Jul 06 '18 12:07 cbeams

2018.07 report

Nothing to report.

/cc bisq-network/compensation#93

ManfredKarrer avatar Jul 26 '18 16:07 ManfredKarrer

2018.08 report

We reverted a code change done in January to improve the broadcast reliability. As it turned out that change had the opposite effect for unknown reasons. The default BitcoinJ policy was to broadcast to half of the connected nodes and wait to hear back from the half of those. As 12 nodes was default connection target, it was broadcast to 6 nodes and waited to hear back from 3. As we have less nodes (8-9) as default connection target we changed the polity to broadcast to all and wait to hear back from only 2. There was one bug that we waited in fact only for one node to hear back - but even that often did not happen. It seems BitcoinJ has some weird side effect that if we broadcast to more nodes the likelihood to hear back is lower, which is very counter intuitive. After longer testing with different settings we reverted to the original policy so we broadcast to 50% and then wait for 50% of those, so with 8 connections we broadcast to 4 nodes and wait for 2 nodes. This change was shipped with 0.8.0 and hopefully reduces the bugs with failed transactions broadcasts.

/cc bisq-network/compensation#112

ManfredKarrer avatar Aug 30 '18 21:08 ManfredKarrer

2018.09 report

Fixed an issue with duplicated peers in case of disconnections and re-connections.

/cc bisq-network/compensation#125

ManfredKarrer avatar Sep 27 '18 00:09 ManfredKarrer

2018.10 report

Nothing to report.

/cc bisq-network/compensation#155

ManfredKarrer avatar Oct 31 '18 04:10 ManfredKarrer

2018.11 report

Nothing to report.

/cc bisq-network/compensation#180

ManfredKarrer avatar Nov 30 '18 21:11 ManfredKarrer

2018.11 report

Coordinate with @oscarguindzberg update to 0.14.7 and review of Bisq changes.

/cc bisq-network/compensation#189

ManfredKarrer avatar Dec 29 '18 14:12 ManfredKarrer

2019.01 report

Nothing to report.

/cc bisq-network/compensation#205

ManfredKarrer avatar Jan 28 '19 17:01 ManfredKarrer

2019.02 report

Nothing to report.

/cc bisq-network/compensation#227

ManfredKarrer avatar Feb 28 '19 05:02 ManfredKarrer

2019.03 report

Audits:

  • [WIP] Connection handling changes audit
    • https://github.com/bisq-network/bitcoinj/issues/30
  • "addr" message support audit
    • Audit report: https://github.com/bisq-network/bitcoinj/issues/28
    • Bugfix: https://github.com/bisq-network/bisq/pull/2591
    • Add link to this report on the source code: https://github.com/bisq-network/bitcoinj/pull/29
  • fee calculation changes audit
    • Commit to audit: https://github.com/oscarguindzberg/bitcoinj/commit/7dd1bfd9c3028266a5f8d1954cbf259ee3907e9b
    • Fix: https://github.com/bisq-network/bitcoinj/pull/27
  • Tor support audit
    • Audit report: https://github.com/bisq-network/bitcoinj/issues/25
    • Improvements: https://github.com/bisq-network/bitcoinj/pull/24
    • Created a couple of PR on bitcoinj upstream
      • https://github.com/bitcoinj/bitcoinj/pull/1741
      • https://github.com/bitcoinj/bitcoinj/pull/1742

Other improvements:

  • Compile bitcoinj with openjdk10
    • https://github.com/bisq-network/bitcoinj/pull/23
    • https://github.com/bisq-network/bisq/pull/2564
  • Move clear blockstore feature to bitcoinj
    • https://github.com/bisq-network/bitcoinj/pull/22
    • https://github.com/bisq-network/bisq/pull/2563
  • Update testnet dns seeds https://github.com/bisq-network/bitcoinj/pull/26
  • Restore from seed
    • Retest bitcoinj upstream bug https://github.com/bitcoinj/bitcoinj/issues/1704 because I was told it was solved. Report what is working and what is not. I had to update to java 11 because the wallet template bitcoinj subproject now requires that. It's important for us that have a benchmark wallet to know how restore from seed should work on bitcoinj.
    • Complete “Restore from seed using blockstore.clear() fix” upstream PR https://github.com/bitcoinj/bitcoinj/pull/1702 (not merged yet)

Research/exploratory tasks:

  • Signet network research
    • It is a “stable testnet” with proof of authority.
    • See email thread on bitcoin core mailing list https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016734.html
    • Started bitcoinj signet implementation https://github.com/oscarguindzberg/bitcoinj/tree/signet
  • Rebroadcasting research
    • See bitcoinj mailing list https://groups.google.com/forum/#!topic/bitcoinj/lkK9N3NhoPY
    • See “reject” msg removal in bitcoin core mailing list https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016701.html

/cc https://github.com/bisq-network/compensation/issues/250

oscarguindzberg avatar Mar 29 '19 15:03 oscarguindzberg

2019.04 report

  • bisq’s bitcoinj 0.14.7 connection handling changes audit (not
    • Audit: https://github.com/bisq-network/bitcoinj/issues/30
    • Fixes: https://github.com/bisq-network/bitcoinj/pull/31
  • Migrate to bitcoinj 0.15
    • Use bisq's bitcoinj 0.15 in bisq
      • https://github.com/bisq-network/bisq/pull/2772
    • Rebase bisq’s bitcoinj from bitcoinj 0.15.1
      • https://github.com/bisq-network/bitcoinj/commits/bisq_0.15.1
      • All commits added by Oscar have a description linking to the commit on the 0.14.7 branch that originated it.
      • Many commits were combined.
      • Many commits were already included on upstream bitcoinj 0.15.1
  • Broadcasting DAO proposal txs / bitcoinj "deprecated" confidence problem
    • Report and investigate bug: https://github.com/bisq-network/bisq/issues/2639
    • Create Fix PR (need more analysis) https://github.com/bisq-network/bitcoinj/pull/32
    • Create upstream issue: https://github.com/bitcoinj/bitcoinj/issues/1769
    • Tests to prove the problem: https://github.com/oscarguindzberg/bitcoinj/commits/tx-confidence-cache
  • Upstream PR
    • Wallet.findKeyFromPubKeyHash(): Fix ScriptType, use P2PKH.
      • https://github.com/bitcoinj/bitcoinj/pull/1782

oscarguindzberg avatar May 08 '19 19:05 oscarguindzberg

I'd be happy to help out with this role. I'm interested in contributing to bisq and think adding support for segwit would be a good first task for me to tackle. I have started by creating a bisq specific bitcoinj based off of upstream release 0.15.3. https://github.com/bodymindarts/bitcoinj/tree/bisq_0.15.3

bodymindarts avatar Sep 04 '19 07:09 bodymindarts

@bodymindarts glad you are interested, don't forget about the existing bisq fork of bitcoinj (https://github.com/bisq-network/bitcoinj) and the segwit code which is either in a PR or merged.

mrosseel avatar Sep 04 '19 07:09 mrosseel

@bodymindarts there is a PR for bisq's updated bitcoinj at https://github.com/bisq-network/bisq/pull/2772

sqrrm avatar Sep 04 '19 07:09 sqrrm

@mrosseel I'm pretty sure the segwit code has already been merged in bitcoinj 0.15.x

@sqrrm thank you, I'm aware of that PR but its based on bitcoinj 0.15.1. I think we should rebase it on 0.15.3 before merging. Thats what I'm currently working on.

bodymindarts avatar Sep 04 '19 07:09 bodymindarts

After discussing with @ManfredKarrer it seems preferable as a long term strategy to reduce the dependency on bitcoinj as much as possible. To accelerate that I am investigating https://github.com/bisq-network/proposals/issues/32 which may enable us to remove large section of wallet functionality. This seems like a better activity for the longterm than putting effort into something that ideally should be removed anyway.

bodymindarts avatar Sep 05 '19 08:09 bodymindarts

I've removed @oscarguindzberg as the assignee for this role as there have been no cycle updates since 2019.04 and I'm leaving it unassigned to reflect the reality that no one is owning this role at the current time. I've also re-added the help wanted label (@m52go, I'm not sure why it was removed in 2019.08 https://github.com/bisq-network/roles/issues/28#issuecomment-529623284).

cbeams avatar Apr 13 '20 11:04 cbeams

@cbeams discussions with the chimp at the time indicated that updating bitcoinj was so arduous and time-consuming (at least for the purpose of supporting segwit) that it didn't make sense to make it a priority given all the other high priorities of the project.

The Bisq v2 proposal from that time was also a consideration (i.e., seemed better to spend resources to develop and build that instead of maintaining bitcoinj in Bisq v1).

I think 1 or 2 folks had come by expressing interest in the role (not sure how serious) and weren't met with a whole lot of interest.

Taking all of this into account, it didn't make sense to keep the role marked as help-wanted.

m52go avatar Apr 13 '20 13:04 m52go

This role has been repurposed for Bisq 2 Security Manager

HenrikJannsen avatar Mar 04 '24 05:03 HenrikJannsen