Display price and price history for tokens that are on L2 or wrapped versions based on the original token

[hboon]

Jump to https://github.com/AlphaWallet/alpha-wallet-ios/issues/3423#issuecomment-1111723198 instead

~Refer to #2629 starting from https://github.com/AlphaWallet/alpha-wallet-ios/issues/2629#issuecomment-915278303~

[hboon]

FWIW, I think the discussion in #2629 is over-engineering things. If we want to ship this, we should just create a manually list and run with it.

[hboon]

We fetch (fiat) prices for tokens with CoinGecko and display them in the Wallet tab and token screens. Not all the tokens have prices. The CoinGecko API output are 1:N, they may already map a token to multiple contracts on different chains. We have tokens.json which includes a similar mapping, so we should use tokens.json to supplement it, so we can display prices

One example is the first entry in tokens.json:

{
    "contracts": [
        {"address": "0x21bfbda47a0b4b5b1248c767ee49f7caa9b23697", "chainId": 1},
        {"address": "0x1631244689EC1fEcbDD22fb5916E920dFC9b8D30", "chainId": 137},
        {
            "address": "0x55704A0e9E2eb59E176C5b69655DbD3DCDCFc0F0",
            "chainId": 42161
        }
    ],
    "group": "Governance"
}

CoinGecko apparently has the price for:

{"address": "0x21bfbda47a0b4b5b1248c767ee49f7caa9b23697", "chainId": 1},

but not:

{"address": "0x1631244689EC1fEcbDD22fb5916E920dFC9b8D30", "chainId": 137},

So, let's show the same price for 0x1631244689EC1fEcbDD22fb5916E920dFC9b8D30 + Polygon (137)

[eviltofu]

So how do I add a token with contract address 0x1631244689EC1fEcbDD22fb5916E920dFC9b8D30 and chainID 137? @hboon

[hboon]

Make sure Polygon chain is enabled. Then Wallet tab > … button > Add/Hide Tokens > Paste address

[hboon]

@eviltofu are you on this? Otherwise would you assign it to @oa-s?

[oa-s]

I will double check it

[hboon]

Sure. Reassign to avoid stepping on each other :)

[oa-s]

@hboon Am i understand right, instead of fetching all ticker ids (as we do now we have to use api call curl -X 'GET' \ 'https://api.coingecko.com/api/v3/coins/polygon-pos/contract/0x8bb30e0e67b11b978a5040144c410e1ccddcba30' \ -H 'accept: application/json') and retrieve information from it, not clear why do we need tokens.json?

[hboon]

We will still use CoinGecko like we do now, but tokens.json is our own data source for token groups and mapping:

• that we know are the "same" on different chains or have the same value, eg. ETH (native token) on mainnet == WETH (ERC20) on mainnet. Or WETH on mainnet == WETH on polygon. Coingecko API results already provide this for some of the tokens (and we already use it to show pricing). We hope to build a richer one and use the mapping. We haven't implemented this yet. Mappings like that can be used:
  • for price (like how Coingecko's mapping in results already let us do, but the data might not overlap completely),
  • in the future, we can potentially group or have button to link these tokens together in the Wallet tab or subscreens, or show a total. Eg. You know you have a total of 10 WETH (doesn't matter on which chain), or we can make it easier to bridge across chains when we want to pool together WETH in Ehhereum mainnet.
• Group tokens as "DeFi", "Governance", eg. This is used to group tokens for the Wallet tab (already implemented)
• There is a special group called "Spam". It's in the tokens repo but not in our copy yet. We can hide those tokens by default in the Wallet tab
• etc

So in that other issue, if SBUSD (correct symbol?) isn't "mapped" to say BUSD (and it should) on CoinGecko, we can update tokens.json, we can add it to tokens.json.

[oa-s]

@Boon, so if i understood right we will have something like this:

...
{
    "contracts" : [
        {
            "address" : "<0x...address.1>",
            "chainId" : 1
        },
        {
            "address" : "0x...address.2",
            "chainId" : 137
        },
        ...
    ],
    "group" : "<CoinGecko ticker id>"
},
{
...

[hboon]

@boon, so if i understood right we will have something like this:

OK, you picked up the keyword "will have". Correct, we don't have this yet :)

{
    "contracts" : [
        {
            "address" : "<0x...address.1>",
            "chainId" : 1
        },
        {
            "address" : "0x...address.2",
            "chainId" : 137
        },
        ...
    ],
    "group" : "<CoinGecko ticker id>"
},
//...

No, the group value will not be CoinGecko's ticker ID, it will be general string that is displayed in the Wallet tab. i.e. "Governance", "DeFi", etc. But in the example above, we can do something like this:

Discover that 0x...address.1 + chainId = 1 and 0x...address.2 + chainId = 137 are the same tokens, so say CoinGecko provides a tickerId = 999 == (0x...address.1 + chainId = 1), then we know that both of these tokens have the same price:

• 0x...address.1 + chainId = 1
• 0x...address.2 + chainId = 137

In this sense, it is de-coupled from CoinGecko's API

[hboon]

So the issue title:

Display price and price history for tokens that are on L2 or wrapped versions based on the original token

Basically means to do this:

Discover that 0x...address.1 + chainId = 1 and 0x...address.2 + chainId = 137 are the same tokens, so say CoinGecko provides a tickerId = 999 == (0x...address.1 + chainId = 1), then we know that both of these tokens have the same price:

[oa-s]

so if i understood an idea right, we want to find group where token is located. The groupd of same token on different chains. and try to figure out ticker id for first matching token for Coingecko.

[hboon]

so if i understood an idea right, we want to find group where token is located. The groupd of same token on different chains. and try to figure out ticker id for first matching token for Coingecko.

Yes, correct. So I'm not sure if we have enough data in tokens.json at the moment that matches this criteria. Do we?

[oa-s]

haven't checked it yet, but do you think this process could take a lot of time, searching ticker id for one token takes a lot of time, and in this case we should look up fro all group of tokens, so the time increases with amount of tokens in group that could contains 10 + tokens. In this case we have to update logic of fetching, because searching for 1 ticker id, will block the rest.

[hboon]

because searching for 1 ticker id, will block the rest.

I thought fetching from CoinGecko is 3-step no matter how many tokens like this:

1. Get a list of supported tickerIds, A
2. Filter A to the tokens we own to get B
3. Get prices for B with a single call

If this is right, then we can modify it so that step 2 doesn't just use the list of tokens we own, but also include the "mapped" tokens from tokens.json for tokens we own. Do you think this would work? Without O(n) calls.

[oa-s]

To resolve ticker we perform few steps:

• resolve ticker id: this may take vary time to resolve as we do different operaion:
  • if we already know ticker id -> return it.
  • else fetching all ticker ids, from coinGecko, (about 13k elements), store it in local db.
  • searching for ticker id in all list, store it if found.
  • else look up for tokens groups:
    • load all token groups;
    • lookup for appropriate group,
  • search for ticker id for each element in group, take first one, store it in local db.
• resolve tikers for all found ticker ids;
• store them in local db.

[hboon]

fetching all ticker ids, from coinGecko, (about 13k elements), store it in local db.

Does this take exactly 1 CoinGecko call, and not N calls? If so, why is it slow like this:

so the time increases with amount of tokens in group that could contains 10 + tokens. In this case we have to update logic of fetching, because searching for 1 ticker id, will block the rest.

Do we have to make that 1 CoinGecko call again per token? Why?

[oa-s]

Does this take exactly 1 CoinGecko call, and not N calls? If so, why is it slow like this:

yes, in this case we make only one call

[oa-s]

Do we have to make that 1 CoinGecko call again per token? Why?

that is about searching in local list of already fetched ticker ids. So in this case no network call

[hboon]

Do you mean it's slow because it's trying to match the ticker IDs in memory? Otherwise, write some code, I'm misunderstanding you :)

[oa-s]

Do you mean it's slow because it's trying to match the ticker IDs in memory?

yes, as instead of searching for one token in 13k ticker ids, we search for n tokens in 13k of ticker ids.

[hboon]

Oh I see. Maybe we can build a lookup data structure to see speed it up. Do you want to try without it and see if where/how much is the bottleneck first?