[Bug][API] Get account resources API returns 500

[nolanxyg]

🐛 Bug

To reproduce

Code snippet to reproduce

# Your code goes here
# Please make sure it does not require any external dependencies
curl -i --request GET --url https://fullnode.mainnet.aptoslabs.com/v1/accounts/0x05a97986a9d031c4567e15b797be516910cfcb4156312482efc6a19c0a30c948/resources --header 'Content-Type: application/json'

Stack trace/error message

// Paste the output here
HTTP/2 500
date: Wed, 26 Oct 2022 05:38:58 GMT
content-type: application/json; charset=utf-8
content-length: 319
set-cookie: AWSALB=CaCYo2CrYCvqqycj2ZLWLph/hoIx0VWikBaXn/ZIrTu+VQwsorwYqojeU7Gp7xhRnL6ubPUNH8bJpXE6s/n1Dyv7yc3KOIujJkNxPIh1gwTt/0IP3yFVuwe0fZFY; Expires=Wed, 02 Nov 2022 05:38:58 GMT; Path=/
set-cookie: AWSALBCORS=CaCYo2CrYCvqqycj2ZLWLph/hoIx0VWikBaXn/ZIrTu+VQwsorwYqojeU7Gp7xhRnL6ubPUNH8bJpXE6s/n1Dyv7yc3KOIujJkNxPIh1gwTt/0IP3yFVuwe0fZFY; Expires=Wed, 02 Nov 2022 05:38:58 GMT; Path=/; SameSite=None; Secure
x-aptos-chain-id: 1
x-aptos-ledger-version: 13190847
x-aptos-ledger-oldest-version: 0
x-aptos-ledger-timestampusec: 1666762736871920
x-aptos-epoch: 163
x-aptos-block-height: 3586028
x-aptos-oldest-block-height: 0

{"message":"Too many values requested for key_prefix StateKeyPrefix { tag: AccessPath, bytes: [5, 169, 121, 134, 169, 208, 49, 196, 86, 126, 21, 183, 151, 190, 81, 105, 16, 207, 203, 65, 86, 49, 36, 130, 239, 198, 161, 156, 10, 48, 201, 72] } - maximum allowed 10000","error_code":"internal_error","vm_error_code":null}%  

Expected Behavior

response should be 200

System information

Please complete the following information:

• aptoslabs/validator:mainnet_cf280b9a9013dddaeda359174b90236b60990533

Additional context

Add any other context about the problem here.

[olivbau]

Same problem here, It seems that this is because there is more than 1000 resources. It surely means that a user with more than 1000 tokens or nft will encounter the same problem. Is there a work around?

[olivbau]

Problem happens also on the explorer. https://explorer.aptoslabs.com/account/0x5a97986a9d031c4567e15b797be516910cfcb4156312482efc6a19c0a30c948

[banool]

Hi all, we're currently working on a fix to this problem.

The issue

Currently in the API there is a hardcoded limit (10,000) on the amount of resources the storage layer will retrieve at any one time for an account. If an account has more than this, the /account/<addr>/resources endpoint will throw a 500.

The fix

We're working on adding cursor based pagination. This means uses will be able to specify ?limit=1000 in their query. We will then include the cursor in a header. In the next query, you can get the next 1000 items with something like ?start=<cursor>&limit=1000. Continue this until the final call returns < 1000 items or no cursor.

Progress

We expect to have the fix itself complete today, but rolling out may take another few days, depending on how fast node operators adopt the fix.

[banool]

Relevant changes:

• ~~https://github.com/aptos-labs/aptos-core/pull/5304~~
• https://github.com/aptos-labs/aptos-core/pull/5353
• https://github.com/aptos-labs/aptos-core/pull/5313

[banool]

We're almost there!

[banool]

API support has landed, SDK support coming here: https://github.com/aptos-labs/aptos-core/pull/5405.

[guijunchen]

Is this bug fixed now?

curl --request GET
--url 'https://fullnode.mainnet.aptoslabs.com/v1/accounts/0x05a97986a9d031c4567e15b797be516910cfcb4156312482efc6a19c0a30c948/resources?start=1&limit=500'
--header 'Content-Type: application/json' {"message":"Too many values requested for key_prefix StateKeyPrefix { tag: AccessPath, bytes: [5, 169, 121, 134, 169, 208, 49, 196, 86, 126, 21, 183, 151, 190, 81, 105, 16, 207, 203, 65, 86, 49, 36, 130, 239, 198, 161, 156, 10, 48, 201, 72] } - maximum allowed 10000","error_code":"internal_error","vm_error_code":null}%

[xandyxor]

Is this bug fixed now?

[guijunchen]

Is this bug fixed now?

it hased merge into release

[banool]

The fix has rolled out to devnet and testnet, but not yet mainnet. Hang tight!

[xandyxor]

Can I set the maximum value allowed first to solve the problem? maximum allowed 10000","error_code":"internal_error

Where can I set the maximum allowed on the local node? maximum allowed 999999999999

[guijunchen]

After we upgraded the node, we used this interface to get data, and found that the request speed was very slow, it took 13 seconds

sdk: poolInfosRaw = (await this.aptosClient.getAccountResources("0x5a97986a9d031c4567e15b797be516910cfcb4156312482efc6a19c0a30c948")

log show call time book@BookdeMBP test % node ./test1.js [2022-11-16T10:38:57.285] [DEBUG] aptos - get liquid swap pool start [2022-11-16T10:39:10.675] [INFO] aptos - resource len: 18442 [2022-11-16T10:39:10.721] [DEBUG] aptos - get liquid swap pool end @banool

[banool]

@guijunchen I've opened a different issue for this one: https://github.com/aptos-labs/aptos-core/issues/5589.

[banool]

We have updated our fullnodes for every network and made announcements for other operators to update theirs, this should now be fixed.