gosdk
gosdk copied to clipboard
retrying an interrupted upload results in `merkle: leaf exists` error
steps to reproduce:
first try to upload a file, during the upload operation, cancel the upload
./zbox upload --localpath /Users/manohar/Downloads/AdvancedDataManagement_18_2021.pdf --remotepath /new229/mgmt.pdf --allocation 50c6ab435ba15a420f6f6cf14eb77405f330dd83a20b7783be40696a85039473
0chain-core-sdk [INFO] 2022/10/01 13:17:59.740640 wallet_base.go:261: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (SetLogFile)
0box-sdk [INFO] 2022/10/01 13:17:59.741051 sdk.go:72: ******* Storage SDK Version: v1.8.8-8-g3c6d3d6c *******
0chain-core-sdk [INFO] 2022/10/01 13:17:59.962284 transaction.go:1332: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (InitZCNSDK)
0box-sdk [DEBUG] 2022/10/01 13:18:00.180728 networkworker.go:105: Get network result:{"miners":["https://manohar.devnet-0chain.net/miner02","https://manohar.devnet-0chain.net/miner01","https://manohar.devnet-0chain.net/miner03"],"sharders":["https://manohar.devnet-0chain.net/sharder01","https://manohar.devnet-0chain.net/sharder02"]}
1376256 / 2565049 [=================================================================>--------------------------------------------------------] 53.65% 00m04s
^C
next check the conform and check the list of all the files for the allocation
./zbox list-all --allocation 50c6ab435ba15a420f6f6cf14eb77405f330dd83a20b7783be40696a85039473
0chain-core-sdk [INFO] 2022/10/01 13:31:13.123380 wallet_base.go:261: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (SetLogFile)
0box-sdk [INFO] 2022/10/01 13:31:13.124414 sdk.go:72: ******* Storage SDK Version: v1.8.8-8-g3c6d3d6c *******
0chain-core-sdk [INFO] 2022/10/01 13:31:13.378003 transaction.go:1332: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (InitZCNSDK)
0box-sdk [DEBUG] 2022/10/01 13:31:13.597946 networkworker.go:105: Get network result:{"miners":["https://manohar.devnet-0chain.net/miner03","https://manohar.devnet-0chain.net/miner02","https://manohar.devnet-0chain.net/miner01"],"sharders":["https://manohar.devnet-0chain.net/sharder01","https://manohar.devnet-0chain.net/sharder02"]}
0box-sdk [DEBUG] 2022/10/01 13:31:14.005722 listworker.go:111: List result:{"allocation_root":"","meta_data":{"allocation_id":"50c6ab435ba15a420f6f6cf14eb77405f330dd83a20b7783be40696a85039473","chunk_size":0,"created_at":0,"hash":"","lookup_hash":"","name":"","num_of_blocks":0,"path":"/","path_hash":"","size":0,"type":"d","updated_at":0},"list":[]}
0box-sdk [DEBUG] 2022/10/01 13:31:14.005744 listworker.go:111: List result:{"allocation_root":"","meta_data":{"allocation_id":"50c6ab435ba15a420f6f6cf14eb77405f330dd83a20b7783be40696a85039473","chunk_size":0,"created_at":0,"hash":"","lookup_hash":"","name":"","num_of_blocks":0,"path":"/","path_hash":"","size":0,"type":"d","updated_at":0},"list":[]}
0box-sdk [DEBUG] 2022/10/01 13:31:14.006394 sync.go:80: Remote List: map[]
[]
you can see that the files list is empty.
when you attempt run the same command again (which you interrupted earlier), I am seeing an error:
./zbox upload --localpath /Users/manohar/Downloads/AdvancedDataManagement_18_2021.pdf --remotepath /new229/mgmt.pdf --allocation 50c6ab435ba15a420f6f6cf14eb77405f330dd83a20b7783be40696a85039473
0chain-core-sdk [INFO] 2022/10/01 13:31:25.565414 wallet_base.go:261: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (SetLogFile)
0box-sdk [INFO] 2022/10/01 13:31:25.566373 sdk.go:72: ******* Storage SDK Version: v1.8.8-8-g3c6d3d6c *******
0chain-core-sdk [INFO] 2022/10/01 13:31:25.827661 transaction.go:1332: ******* Wallet SDK Version:v1.8.8-8-g3c6d3d6c ******* (InitZCNSDK)
0box-sdk [DEBUG] 2022/10/01 13:31:26.053441 networkworker.go:105: Get network result:{"miners":["https://manohar.devnet-0chain.net/miner03","https://manohar.devnet-0chain.net/miner02","https://manohar.devnet-0chain.net/miner01"],"sharders":["https://manohar.devnet-0chain.net/sharder01","https://manohar.devnet-0chain.net/sharder02"]}
65536 / 2565049 [===>----------------------------------------------------------------------------------------------------------------------------] 2.55% 0s
Error in file operation: merkle: leaf exists, it can be skipped
Upload failed. merkle: leaf exists, it can be skipped
but if I change the remote path, it works fine.
expected:
since the list-files shows that the files doesn't exist in remote path, I should be able to upload file with the same remote path.
This is due to the cache in gosdk. @cnlangzi would be the right person to discuss about this issue.
@lpoli resume upload was working. it might be broken by recent related PRs
@boddumanohar it is gosdk issue, not blobber. let me transfer this issue to gosdk repo
Cannot replicate this issue. @boddumanohar Can you please recheck this? Also please verify if resume of upload is working.
tried using the same instructions in the ticket. Not able to able reproduce. Must be fixed in one of the associated PRs. Thanks.