subgraphs icon indicating copy to clipboard operation
subgraphs copied to clipboard

Saddle Finance QA (Ethereum) Schema Version 1.3.0 Subgraph Version 1.1.0 Methodology Version 1.0.0

Open bye43 opened this issue 3 years ago • 4 comments

Description Value
Subgraph Reviewed https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/name/messari/saddle-finance-ethereum&tab=protocol
Date Reviewed July 12 2022
Schema Version 1.3.0
Subgraph Version 1.1.0
Methodology Version 1.0.0
Evidence Spreadsheet https://docs.google.com/spreadsheets/d/1d-BxTuefAKYALD6YjyI8aeV1K5PPj8OCxIUpR__7XF8/edit?usp=sharing

Metrics To Review

Protocol Metrics

Section Metric Issue
financialsDailySnapshots totalValueLockedUSD Trend is similar to DefiLlama and token terminal, but they have the exact same trend. Subgraph values are slightly higher 5%-15% for recent data, which is ok, but older data shows more disparity (>40%)
financialsDailySnapshots dailyVolumeUSD There are quite large differences, with subgraph having generally higher values from 55% to 437% difference in volume. Erroneous peak in subgraph data on Apr 30 (>160 million in volume)
financialsDailySnapshots cumulativeVolumeUSD Off because daily is
financialsDailySnapshots dailySupplySideRevenueUSD Values are generally higher on token terminal, in the range of 20-45%. This might be a combination of how we we are calculating revenue and that daily volume is off
financialsDailySnapshots cumulativeSupplySideRevenueUSD Off because daily is
financialsDailySnapshots dailyProtocolSideRevenueUSD I think subgraph data is right here, but Saddle Finance claims this is 0 in their docs. I believe that the admin fee is being applied now, but need to find source that says that admin fees started on 1/19/2022 and double check. For now, I will go with the "official" documentation and make a note.
financialsDailySnapshots cumulativeProtocolSideRevenueUSD Off because daily is
financialsDailySnapshots dailyTotalRevenueUSD Off because protocol and supply side are
financialsDailySnapshots cumulativeTotalRevenueUSD Off because daily is

Pool Overview

Pool Metric Issue
ALL POOLS Overall Pool Count Missing 4 pools compared to what is showing on the Saddle UI.
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Should have SDL as reward token
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Base Yield % Lower Base APR than Saddle UI
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Should have SDL as reward token
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Base Yield % Lower Base APR than Saddle UI
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Should have SDL as reward token
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Base Yield % Lower Base APR than Saddle UI

Pool Metrics

Pool Section Metric Issue
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenAmount [0] - LUSD: LUSD Stablecoin Seems like assets are not being mapped correctly. For example, the values for LUSD are matching more correctly to the values of alUSD shown in the Dune query. However even accounting for this, volume numbers dont line up compared to Dune and there are large differences (see evidence sheet). Link - https://dune.com/queries/18342
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenAmount [1] - FRAX: Frax Seems like assets are not being mapped correctly. For example, the values for LUSD are matching more correctly to the values of alUSD shown in the Dune query. However even accounting for this, volume numbers don't line up compared to Dune and there are large differences (see evidence sheet). Link - https://dune.com/queries/18343
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenAmount [2] - FEI: Fei USD Seems like assets are not being mapped correctly. For example, the values for LUSD are matching more correctly to the values of alUSD shown in the Dune query. However even accounting for this, volume numbers don't line up compared to Dune and there are large differences (see evidence sheet). Link - https://dune.com/queries/18344
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenAmount [3] - alUSD: Alchemix USD Seems like assets are not being mapped correctly. For example, the values for LUSD are matching more correctly to the values of alUSD shown in the Dune query. However even accounting for this, volume numbers don't line up compared to Dune and there are large differences (see evidence sheet). Link - https://dune.com/queries/18345
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenUSD [0] - LUSD: LUSD Stablecoin Same comments as above in regards to mapping. Also, this is exactly the same as the TokenAmount values. There should probably be some small difference or is this assuming all stables hold peg?
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenUSD [1] - FRAX: Frax Same comments as above in regards to mapping. Also, this is exactly the same as the TokenAmount values. There should probably be some small difference or is this assuming all stables hold peg?
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenUSD [2] - FEI: Fei USD Same comments as above in regards to mapping. Also, this is exactly the same as the TokenAmount values. There should probably be some small difference or is this assuming all stables hold peg?
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots dailyVolumeByTokenUSD [3] - alUSD: Alchemix USD Same comments as above in regards to mapping. Also, this is exactly the same as the TokenAmount values. There should probably be some small difference or is this assuming all stables hold peg?
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots inputTokenBalances [0] - LUSD: LUSD Stablecoin Same mapping problem here. If the index is corrected, values match TVL for the individual tokens compared to Dune. Link - https://dune.com/queries/981081
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots inputTokenBalances [1] - FRAX: Frax Same mapping problem here. If the index is corrected, values match TVL for the individual tokens compared to Dune. Link - https://dune.com/queries/981082
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots inputTokenBalances [2] - FEI: Fei USD Same mapping problem here. If the index is corrected, values match TVL for the individual tokens compared to Dune. Link - https://dune.com/queries/981083
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots inputTokenBalances [3] - alUSD: Alchemix USD Same mapping problem here. If the index is corrected, values match TVL for the individual tokens compared to Dune. Link - https://dune.com/queries/981084
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-stakedOutputTokenAmount - saddleD4: Saddle alUSD/FEI/FRAX/LUSD LPs are staked in Saddle, so there should be an output here
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardTokenEmissionsAmount SLD and rewards from ALCX, FXS, LQTY, TRIBE should be accounted for here
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardTokenEmissionsUSD SLD and rewards from ALCX, FXS, LQTY, TRIBE should be accounted for here
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardAPR SLD and rewards from ALCX, FXS, LQTY, TRIBE should be accounted for here
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots rewardTokenEmissionsAmount [0] - SDL: Saddle DAO Outputting 0 for the last few days when Saddle UI is saying there is rewards
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots rewardTokenEmissionsUSD [0] - SDL: Saddle DAO Outputting 0 here, but there should be values since SLD is being emitted
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots baseYield Does not match values in Saddle UI
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots rewardAPR Does not match values in Saddle UI

Event Metrics

Pool Section Issue
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Swaps Not sure if this is actually wrong since the swaps might be routed differently, but etherscan is showing last swap was 13 days ago and the subgraph is showing swaps for present time. Link - https://etherscan.io/address/0xc69ddcd4dfef25d8a793241834d4cc4b3668ead6

bye43 avatar Jul 12 '22 05:07 bye43

TVL: It does seem that DefiLlama data is not including all pools. Also TokenTerminal uses DefiLlama TVL data for their own graph, so they should be identical.

Volume: The 160m spike happened on the day of the hack, TokenTerminal data seems to not include the hacked pool. Most likely missing other newer pools as well.

Protocol side revenue: Here's the tx where admin fee was first set for 3 pools: https://etherscan.io/tx/0x2782125d106949008c3caf62a31b5724704f84e4d5e68a2a9b45d109eef02d56#eventlog

The prices for pegged-value assets are indeed fixed to the underlying.

spitko avatar Jul 15 '22 22:07 spitko

TVL: It does seem that DefiLlama data is not including all pools. Also TokenTerminal uses DefiLlama TVL data for their own graph, so they should be identical.

Volume: The 160m spike happened on the day of the hack, TokenTerminal data seems to not include the hacked pool. Most likely missing other newer pools as well.

Protocol side revenue: Here's the tx where admin fee was first set for 3 pools: https://etherscan.io/tx/0x2782125d106949008c3caf62a31b5724704f84e4d5e68a2a9b45d109eef02d56#eventlog

The prices for pegged-value assets are indeed fixed to the underlying.

Sounds good then for those things. Thanks!

bye43 avatar Jul 15 '22 23:07 bye43

@bye43 this is ready for another round

this-username-is-taken avatar Jul 22 '22 11:07 this-username-is-taken

Description Value
Subgraph Reviewed https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/name/messari/saddle-finance-ethereum&tab=protocol
Date Reviewed July 22 2022
Schema Version 1.3.0
Subgraph Version 1.1.2
Methodology Version 1.0.0
Evidence Spreadsheet https://docs.google.com/spreadsheets/d/1zOvzx27qR9IxhovSHw59FXWILecUXy_GgZwM14g04b0/edit?usp=sharing

Metrics To Review

Protocol Metrics

Section Metric Issue
financialsDailySnapshots dailyVolumeUSD Scaling is off. There are dates with volume measuring 8E16.
financialsDailySnapshots cumulativeVolumeUSD Off because daily is
financialsDailySnapshots dailySupplySideRevenueUSD Values are too high here, but I am guessing that this is screwed up because of the high volume values
financialsDailySnapshots cumulativeSupplySideRevenueUSD Off because daily is
financialsDailySnapshots dailyProtocolSideRevenueUSD Same as Supply Side revenue
financialsDailySnapshots cumulativeProtocolSideRevenueUSD Off because daily is
financialsDailySnapshots dailyTotalRevenueUSD Same as Supply Side revenue. Math looks right though for all revenues, so assuming once volume is addressed then revenues will be fine
financialsDailySnapshots cumulativeTotalRevenueUSD Off because daily is

Pool Overview

Pool Metric Issue
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Says there is 0% rewards here for SDL, which is incorrect. Looks like Daily reward emissions are 0 here in the equation. I am not sure how saddle distributes emissions, but I assume it is on a per second inflation, so this daily value should not be 0. Link Saddle UI that says there are emissions - https://saddle.exchange/#/pools.
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Says there is 0% rewards here for SDL, which is incorrect. Same as above.
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) Reward Tokens Says there is 0% rewards here for SDL, which is incorrect. Same as above
Some Pools ALL Certain pools like FRAX-USDC-BP and FRAX-sUSD are not showing output on the pool overview tab, but are prevalent on the Saddle UI. These pools have little volume, so maybe this is not wrong, but figured I would make a note, as there are errors when you search the pool address in the subgraph UI. Link to Saddle UI - https://saddle.exchange/#/pools. Pool address for FRAX-USDC-BP - 0x13Cc34Aa8037f722405285AD2C82FE570bfa2bdc

Pool Metrics

Pool Section Metric Issue
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-stakedOutputTokenAmount - saddleD4: Saddle alUSD/FEI/FRAX/LUSD Saddle LPs can be staked for more yield, but I do not believe they are staked on the actual Saddle Protocol. I am guessing that this makes it super hard to track / there most likely will not be output for this?
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardTokenEmissionsAmount Should be value here. Link to Saddle UI showing SDL APY - https://saddle.exchange/#/pools
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardTokenEmissionsUSD Same as above
Saddle alUSD/FEI/FRAX/LUSD (saddleD4) liquidityPoolDailySnapshots liquidityPoolDailySnapshots-rewardAPR Same as above

bye43 avatar Jul 22 '22 16:07 bye43

@bye43 this is ready to QA here: https://okgraph.xyz/?q=QmcNs5HMR1Uk76tnoLZqHP5Tfui8HtH55kABmsbzsGDGVQ

Some of the pools that were deployed manually instead a deployer contract were missing. I'm adding them now by looking at a registry contract. Technically it would be possible for some swaps, deposits and withdrawals to be missing if it takes too long between deploying the pool and adding it to the registry, but other than monitoring the deployer address manually and hardcoding the new pools in the config, this might be the most accurate way.

So, missing pools should now be there. Even paused or deprecated ones. Issues with giant numbers should be fixed too.

The only thing I've left out for now is rewards. Saddle structured their rewards in 2 steps:

  • Via masterchef initially, for 6 months or so.
  • After initial rewards period ended, via gauges, like curve.

The rewards from first period should be correct. The ones from the second period I want to take a look at how we've done it with curve first to follow a similar approach. Tracking this here: https://github.com/messari/subgraphs/issues/1179

jaimehgb avatar Oct 10 '22 13:10 jaimehgb

@jaimehgb Couple comments here:

Description Value
Subgraph Reviewed https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/id/QmcNs5HMR1Uk76tnoLZqHP5Tfui8HtH55kABmsbzsGDGVQ&tab=protocol
Date Reviewed October 17 2022
Schema Version 1.3.0
Subgraph Version 1.1.4
Methodology Version 1.0.0

Pool Overview

Pool Metric Issue
Saddle USX/saddleFraxBP LP Token (0x2bff1b48cc01284416e681b099a0cddca0231d72) TVL (USD) Looks like USX is being blacklisted here (0x0a5e677a6a24b2f1a2bf4f3bffc443231d2fdec8). This is resulting in the TVL not matching the saddle UI (https://saddle.exchange/#/pools/USDC-USX/deposit). USX is a real token, so do not think it should be blacklisted, but would like to hear @jaimehgb thoughts.
Some Pools stakedOutputTokenAmount Based on the Saddle UI (https://saddle.exchange/#/farm), I believe there should staked token values for some the pools that currently say null (https://api.thegraph.com/subgraphs/id/QmcNs5HMR1Uk76tnoLZqHP5Tfui8HtH55kABmsbzsGDGVQ/graphql?query=%7B%0A++liquidityPools+%28orderBy%3A+totalValueLockedUSD%2C+orderDirection%3A+desc%29+%7B%0A++++name%0A++++id%0A++++totalValueLockedUSD%0A++++stakedOutputTokenAmount%0A++%7D%0A%7D). For example, 0x69baa0d7c2e864b74173922ca069ac79d3be1556 looks like there should be a value for staked LPs.

bye43 avatar Oct 17 '22 14:10 bye43

Deployed a new version with pricing support for tokens not supported by the price lib. USX now has a price :D

https://okgraph.xyz/?q=QmPKQCSMjDEo45W9U4x7RFB5aXhqXfWzvDJhfzJn77pTU1 (synced from scratch)

stakedOutputTokenAmount as discussed is part of #1179 .

jaimehgb avatar Oct 19 '22 21:10 jaimehgb

@jaimehgb USX looks good now! @this-username-is-taken Everything except for rewards is good here. I am going to close this comment and after it is moved to Messari hosted it should be good to be backfilled!

bye43 avatar Oct 20 '22 03:10 bye43