rindexer icon indicating copy to clipboard operation
rindexer copied to clipboard

Alternative data sources to RPC

Open ozgrakkurt opened this issue 1 year ago • 8 comments

Hey,

Did you consider alternative data sources to RPC? What do you think about something like this: https://github.com/enviodev/hypersync-client-rust ?

I am working on hypersync so this is a bit of a shill but It is genuinely a massive upgrade compared to RPC on indexing cases like rindexer

ozgrakkurt avatar Jul 16 '24 00:07 ozgrakkurt

Hey JSONRPC was the first release but as explained here https://rindexer.xyz/docs/introduction/what-is-rindexer#future-vision-for-rindexer we want rindexer to not care how the data is synced so for sure want to add many different data sources so would be open to seeing how we could make rindexer hypersync compatible and then editable in the yaml..?

joshstevens19 avatar Jul 16 '24 11:07 joshstevens19

Hey JSONRPC was the first release but as explained here https://rindexer.xyz/docs/introduction/what-is-rindexer#future-vision-for-rindexer we want rindexer to not care how the data is synced so for sure want to add many different data sources so would be open to seeing how we could make rindexer hypersync compatible and then editable in the yaml..?

sounds good! I can implement hypersync as a source if you can ping me when code is ready on the indexer side. Think it can boost the performance a lot

ozgrakkurt avatar Jul 16 '24 12:07 ozgrakkurt

guess just need to make this a trait? https://github.com/joshstevens19/rindexer/blob/a43544eaea6735acd2f64e5f94c1b4910fc8e41b/core/src/provider.rs#L101

I can work on converting it into a trait as well if it makes sense. Don't want to distrupt your flow though if you plan to change other things

ozgrakkurt avatar Jul 16 '24 12:07 ozgrakkurt

I am happy to accept a PR for this for sure, way better for you to do then me as you are building the hypersync lol you know way more then me on it.

If its a hyperscan url in the yaml use the hyperscan provider - go for it 💪 shout if you need any help

joshstevens19 avatar Jul 16 '24 12:07 joshstevens19

Would be happy to add SQD as an alternative data source too!

dzhelezov avatar Jul 30 '24 09:07 dzhelezov

if its as simple as adding this trait and calling via jsonrpc as per the hyperscan URL then sounds good to me would accept a PR @dzhelezov

joshstevens19 avatar Aug 06 '24 00:08 joshstevens19

Someone has submitted a hypersync draft https://github.com/joshstevens19/rindexer/pull/103

joshstevens19 avatar Sep 16 '24 10:09 joshstevens19

Someone has submitted a hypersync draft https://github.com/joshstevens19/rindexer/pull/103

To clarify Misha is in hypersync team so the implementation is "official" by envio

ozgrakkurt avatar Sep 16 '24 11:09 ozgrakkurt