taproot-assets icon indicating copy to clipboard operation
taproot-assets copied to clipboard

[bug]: Group key send: Assets never show up at recipient's node

Open Liongrass opened this issue 2 months ago • 8 comments

Background

I minted a new asset in three tranches. I sent a portion of the newly minted assets to another tapd node. The send transaction succeeds, but the assets are not available on the recipient's node.

Your environment

Sender tapd: 0.7.0-alpha.rc1 commit=v0.7.0-rc1-149-ge6ae082c

Recipient tapd: 0.7.0-alpha.rc1 commit=v0.7.0-rc1-149-ge6ae082c

Universe tapd: 0.7.0-alpha.rc1 commit=v0.7.0-rc1

Steps to reproduce

Tell us how to reproduce this issue. Please provide stacktraces and links to code in question.

On the sender's node:

tapcli assets mint --type normal --name eightee --supply 1000000 --decimal_display 3 --meta_file_path eight.json --meta_type json --new_grouped_asset tapcli assets mint --type normal --name eightee --supply 1000000 --decimal_display 3 --grouped_asset --meta_file_path eight.json --meta_type json --group_key 036138b75291d094416a6c6905c3f78446785ae6f498373418d1d103a53d90cde3

On the recipient's node:

tapcli addrs new --group_key 036138b75291d094416a6c6905c3f78446785ae6f498373418d1d103a53d90cde3

Then again on the sender's node:

tapcli assets send --addr_with_amount taptb1qqqsyqspqqzjzqmp8zm49ywsj3qk5mrfqhpl0pzx0pdwdaycxu6p35w3qwjnmyxduvrzzqu6paynlc88ddypzvyyzhlxmvuc0zug6wsqwlxqv8nsv0jc74mnfuyzzq3xygecfjvqq992jryyce6gqfk2dfxhy5argtx4qsad7ds4ha2tz59qzqqv89sh2argd4skjmrzdauzkatwd9mx2unnv4e8qce69uhh2mnfwejhyum99eekjemwv46zumrpd9ek2efwdaexww3cxs6rxe287ht:2500000

2025-10-14_tapd-universe.log 2025-10-14_tapd-receiver.log 2025-10-14_tapd-sender.log

`

Expected behavior

I expect to find the asset on the recipient node with tapcli assets groups

Actual behavior

I cannot find the asset on the recipient's node.

Sender node

2025-10-14 23:51:45.748 [INF] PROF: Starting proof transfer backoff procedure (transfer_type=send, locator_hash=f06d74ebe12e36c3f90bb8804e9156844f6e7ad2415da3ccc42b8c5a1e94d533)
2025-10-14 23:51:45.757 [INF] PROF: Transferring send fragment to receiver with claimed outpoint aaffa98c066ef64852fa679e7c83393c054f31d7d4f304f3375323fbcded2241:0
2025-10-14 23:51:45.763 [INF] PROF: Successfully sent send fragment to server, ID=4
2025-10-14 23:51:45.774 [INF] PROF: Starting proof transfer backoff procedure (transfer_type=send, locator_hash=1af7d231f87abad156ef18e152a44a48e575d0dc7f2955c063931f64e727f957)
2025-10-14 23:51:45.777 [INF] PROF: Transferring send fragment to receiver with claimed outpoint aaffa98c066ef64852fa679e7c83393c054f31d7d4f304f3375323fbcded2241:0
2025-10-14 23:51:45.782 [INF] PROF: Successfully sent send fragment to server, ID=4
2025-10-14 23:51:45.870 [INF] FRTR: Transfer output proof delivery complete (anchor_txid=aaffa98c066ef64852fa679e7c83393c054f31d7d4f304f3375323fbcded2241, output_position=3)
2025-10-14 23:51:45.870 [INF] FRTR: ChainPorter executing state: SendStateComplete
2025-10-14 23:51:45.870 [INF] FRTR: Parcel transfer is fully complete (anchor_txid=aaffa98c066ef64852fa679e7c83393c054f31d7d4f304f3375323fbcded2241)
2025-10-14 23:51:45.870 [INF] FRTR: ChainPorter completed state machine for parcel (anchor_txid=aaffa98c066ef64852fa679e7c83393c054f31d7d4f304f3375323fbcded2241)

Liongrass avatar Oct 15 '25 00:10 Liongrass

The receiver node successfully created a V2 tap address, which confirms it connected to the authmailbox service. However, since the receiver tapd node isn’t running with debug-level logging, it’s difficult to determine why it later fails to establish a stream connection to the authmailbox for receiving events.

We should add more detailed logging to /taprpc.TaprootAssets/NewAddr to clearly indicate when address creation succeeds and to report which authmailbox was connection-checked.

@Liongrass could you please restart the receiver tapd node with debug-level logging and then share the logs once more?

ffranr avatar Oct 15 '25 13:10 ffranr

I changed the debugging level, restarted tapd and let it run for three hours. I've attached the logs below:

2025-10-15_tapd-receiver.log

Liongrass avatar Oct 15 '25 18:10 Liongrass

Thanks for the additional logs.

I’ve spun up a tapd node locally and successfully connected to your authmailbox (universe server) address, reproducing the connection your receiver node should be able to make. I used the proof courier address embedded in the tap address you reported. This suggests that your receiver tapd node may be unable to reach your authmailbox universe server due to a network issue external to tapd.

I’ll open a PR to add more detailed logging on the receiver node side.

ffranr avatar Oct 16 '25 11:10 ffranr

I am running pull request #1853 on the receiver node. I made another transfer from sender to receiver.

The transaction confirmed at block 274176.

2025-10-16 19:47:21.341 [INF] GRDN: New block at height 274176
2025-10-16 19:48:25.500 [DBG] UNIV: Federation envoy handling new tick event
2025-10-16 19:48:25.502 [INF] UNIV: Synchronizing with 2 federation members
2025-10-16 19:48:25.502 [INF] UNIV: Syncing Universe state with server=universe.signet.laisee.org:8443
2025-10-16 19:48:25.503 [INF] UNIV: Attempting to sync universe: host=universe.signet.laisee.org:8443, sync_type=full, ids=([]universe.Identifier) <nil>

2025-10-16 19:48:25.503 [INF] TSVR: Connecting to Universe server at universe.signet.laisee.org:8443
2025-10-16 19:48:25.503 [INF] UNIV: Fetching all roots for remote Universe server...
2025-10-16 19:48:25.503 [DBG] UNIV: Fetching roots in range: 0 to 512
2025-10-16 19:48:25.560 [INF] UNIV: Obtained 58 roots from remote Universe server
2025-10-16 19:48:25.561 [DBG] UNIV: Looking up root node for base Universe (universe.Identifier) issuance-745ce18798c8a7a05e2091c656bf61398044c725421f2546cdf00f78375afeb3

2025-10-16 19:48:25.566 [DBG] UNIV: Root for issuance-745ce18798c8a7a05e2091c656bf61398044c725421f2546cdf00f78375afeb3 matches, no sync needed
2025-10-16 19:48:25.566 [DBG] UNIV: Looking up root node for base Universe (universe.Identifier) transfer-ca390620398bdc5134f91b9e2453ccf8e70ef4566446de124d498abd5029635d

2025-10-16 19:48:25.569 [DBG] UNIV: Root for transfer-ca390620398bdc5134f91b9e2453ccf8e70ef4566446de124d498abd5029635d matches, no sync needed
2025-10-16 19:48:25.569 [DBG] UNIV: Looking up root node for base Universe (universe.Identifier) transfer-e7eeaf69307f19ea67795e482af7de40de854cb14a6277f7965d1b04fd6ad299

Full logs:

2025-10-16_tapd-receiver.log

2025-10-16_tapd-sender.log

Liongrass avatar Oct 16 '25 20:10 Liongrass

I made another transfer edebf14ce619e3b67273ecd96063585d406e9df9829f46d41d2f146f9fc1cfc9. It confirmed in block 274193.

2025-10-16_tapd-receiver2.log

2025-10-16_tapd-sender2.log

Liongrass avatar Oct 16 '25 23:10 Liongrass

Thanks for the latest logs.

The receiving node’s proof courier (authmailbox) address is incorrect; it’s using signet.universe.lightning.finance:443.

I was able to establish an authmailbox connection to the correct address, universe.signet.laisee.org:8443, using the same tapd commit in a unit test. Patch:

Index: authmailbox/client_test.go
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/authmailbox/client_test.go b/authmailbox/client_test.go
--- a/authmailbox/client_test.go	(revision 5e5780270130cdc0c6b46401142804285c23800b)
+++ b/authmailbox/client_test.go	(date 1760690318618)
@@ -9,6 +9,7 @@
 	"time"
 
 	"github.com/btcsuite/btclog/v2"
+	"github.com/lightninglabs/taproot-assets/address"
 	"github.com/lightninglabs/taproot-assets/fn"
 	"github.com/lightninglabs/taproot-assets/internal/test"
 	"github.com/lightninglabs/taproot-assets/proof"
@@ -157,9 +158,20 @@
 
 	// We also add a multi-subscription to the same two keys, so we can make
 	// sure we can receive messages from multiple clients at once.
+	clientCfg.Insecure = true
+	clientCfg.SkipTlsVerify = true
+
+	addr := "taptb1qqqsyqspqqzjzqmp8zm49ywsj3qk5mrfqhpl0pzx0pdwdaycxu6p35w3qwjnmyxduvrzzqu6paynlc88ddypzvyyzhlxmvuc0zug6wsqwlxqv8nsv0jc74mnfuyzzq3xygecfjvqq992jryyce6gqfk2dfxhy5argtx4qsad7ds4ha2tz59qzqqv89sh2argd4skjmrzdauzkatwd9mx2unnv4e8qce69uhh2mnfwejhyum99eekjemwv46zumrpd9ek2efwdaexww3cxs6rxe287ht"
+	recvAddr, err := address.DecodeAddress(addr, &address.SigNetTap)
+	require.NoError(t, err)
+
+	desc := keychain.KeyDescriptor{
+		PubKey: &recvAddr.ScriptKey,
+	}
+	clientCfg.ServerAddress = recvAddr.ProofCourierAddr.Host
 	multiSub := NewMultiSubscription(*clientCfg)
-	err := multiSub.Subscribe(
-		ctx, url.URL{Host: clientCfg.ServerAddress}, clientKey1, filter,
+	err = multiSub.Subscribe(
+		ctx, recvAddr.ProofCourierAddr, desc, filter,
 	)
 	require.NoError(t, err)
 	err = multiSub.Subscribe(

In that test, I extract the proof courier (authmailbox) service address from the tap address used by the sender. It shows the correct address: universe.signet.laisee.org:8443, which explains why sending works but receiving does not.

Here’s the log line from the test showing a successful ping:

2025-10-17 09:45:37.692 [INF] AMBX: Successfully connected to mailbox server receiver_key=039a0f493fe0 server=false server_addr=universe.signet.laisee.org:8443

ffranr avatar Oct 17 '25 08:10 ffranr

This is what I think happened:

  • The recv node created a tap address without specifying an authmailbox proof courier address.

    • Since no proof courier service address was specified, the default signet address was used: signet.universe.lightning.finance:443.
    • The tap address and default proof courier address were stored in the recv node’s database.
  • The sender attempted to send via signet.universe.lightning.finance:443, but that service is currently nonfunctional.

  • The recv node generated a new address using a working authmailbox service at signet.universe.lightning.finance:8443.

    • This new address may have reused the same script key as the earlier one, which could lead to an incomplete database update (see below).
    • The SQL query UpsertAddr does not update the proof courier address when the taproot output key matches an existing entry. For V2 addresses, this key corresponds to the script key.
    • As a result, a new address could have been created with the correct service address, but the database entry may still point to the old, broken one.
  • The sender successfully sends to the new address and uploads proofs to the authmailbox, completing the transfer from its perspective.

  • The recv node then tries to fetch proofs but still uses the outdated service address stored in its database, causing the fetch to fail.

ffranr avatar Oct 17 '25 09:10 ffranr

A few improvements to make:

  • When creating a tap address that uses the authmailbox proof courier, we should verify the server connection. This check isn’t currently performed: https://github.com/lightninglabs/taproot-assets/blob/33dd9a560b5e76ea0a0f8fe021a2ab7f233b1e74/rpcserver.go#L1720-L1723

  • The database record should be updated on tap address upsert to reflect any change in the proof courier service address.

ffranr avatar Oct 17 '25 09:10 ffranr

I think this one is arguably complete with the merge of #1854.

jtobin avatar Nov 19 '25 10:11 jtobin

I think this one is arguably complete with the merge of #1854.

I agree. Also, we don't need this issue to track https://github.com/lightninglabs/taproot-assets/pull/1858

ffranr avatar Nov 19 '25 12:11 ffranr