[backend] Use Deribit for the price feed (https://test.deribit.com/api_console/?method=%2Fpublic%2Fget_index_price) and update every slot.

[JustinDrake]

trafficstars

[JustinDrake]

Another thing that would be great is to keep the values displayed in the tab header (see screenshot) significantly fresher.

I noticed that this Deribit tab keeps a fresh price value much longer than ultrasound.money.

[alextes]

I'd break this work into two parts. First, start depending on Deribit for our ETH price feed instead of FTX. Second, change the frequency of the updates to every twelve seconds. Then there are extras that we can skip for now.

For part one:

• Make sure the Deribit API is precise down to 12s, otherwise let's discuss more first.
• Make sure the API is extremely reliable. If the analysis logic detects an ETH price is more than ~10min old, it will halt all analysis. So even 10min of downtime every few months is problematic.
• Make sure every 12s is under the rate limit.
• Reuse the existing eth_prices table which has a timestamp and an ethusd column. Where the timestamp is rounded down to the minute.
• The new service can run in parallel with the old, where they'd overwrite each others. Then if the new looks good the old can be shut off.

For part two:

• Change the frequency to be every 12 seconds.
• Rewrite the script which can heal missing prices. I was thinking of leaving this for part three but on second thought it's quite critical. Both when the TS side handled it and when the Rust side handled it I needed to deal with this scenario. Basically, when an outage happens, and this includes one we cause, there's a gap and analysis can't resume until the now historically missing prices are present. It's not enough to turn the service back on. So I wrote a script that looks at all minutes since London and whenever there is one missing knows how to fetch and store it. You should update that too. If we change frequency to 12 seconds you then also need to answer the question of whether you will historically backfill with this script or write it to know to backfill every minute until some point in time where we switched to every 12 seconds and then fill 12 seconds gaps from that point on. I'm leaning backfill if we have the script anyway.
• If you backfill don't forget it needs to be done for two environments.
• If you need an API key for Deribit check that the rate-limit is respected in a way where the price feed doesn't suffer when the script backfills.

For part three (the stuff we can postpone indefinitely, nice to have):

• Add a slot to each row. I'm unsure how this would work. It has wider implications. Now we look if there is a price in the DB which is close enough when we sync. Looking for a price of the correct slot would introduce a need to wait on the Deribit call. What if that fails? etc.

[alextes]

Right, I'd put keeping the price fresh in the tab title in part three, or separate it out into a different issue. It has nothing to do with the price feed I believe, only with whether the browser keeps our JS running or not and how one can update a page title in the background reliably.