blobscan icon indicating copy to clipboard operation
blobscan copied to clipboard

Filtering by rollup (non rollup)

Open tunnckoCore opened this issue 2 years ago • 6 comments

Heya there!

I tried to pass null to the rollup query param on the API endpoints it fails with error (invalid input, ok logical).

But since the beginning of Blobs in march we started using the blobspace for creating Ethscriptions (and also storing it permanently), thus in you api it will say rollup: null. More on that at ESIP-8 we developed and implemented.

Now, I think we can either add ethscriptions as a "rollup" label (despite it's not), or support filtering by rollup=null in the API.


Another a bit side question: can somehow we access the stored data from these dataReferences? Or it's not public. I know we can fetch the blob data hex from the blob endpoint but still, can we access the raw data?


A third question: can we add the "transaction fee", the normal gas fee used?


Another one: what are the API rate limits?

tunnckoCore avatar Apr 19 '24 20:04 tunnckoCore

It's here https://github.com/Blobscan/blobscan/blob/23c26e146b16602f4f8ac8be5e9f69162da694cd/packages/api/src/routers/tx/getAll.ts#L72C22-L72C48

If it's empty string in the database, then we can detect if the param is null as a string, if it as a string, then pass string to the db.

I assume that if it was null in the database, that the rollup=null would have worked.

Btw, gg for Prisma, it's awesome. I recently switched to Drizzle and it's a frickin breeze.

tunnckoCore avatar Apr 19 '24 21:04 tunnckoCore

Hey folks, @PabloCastellano @0xGabi @sembrestels,

don't want to bother you too much, but can you check this please :wink: :raised_hands:

tunnckoCore avatar Apr 27 '24 21:04 tunnckoCore

Hey @tunnckoCore, sorry for the late reply on this.

I tried to pass null to the rollup query param on the API endpoints it fails with error (invalid input, ok logical).

This is probably a zod issue. I'll add a fix to the api to be able to filter by null values.

Also, we're planning to generalize therollup field to something more appropriate, such as type, to accommodate new codifications like Ethscriptions that are emerging and also create a new "decoder" component in charge of decoding these blobs and storing the decoded data on the db. This way we could show the decoded blob data on the web app and serve it through the api as well.

Another a bit side question: can somehow we access the stored data from these dataReferences? Or it's not public. I know we can fetch the blob data hex from the blob endpoint but still, can we access the raw data?

The blob data pointed by these references are public. We're actually redirecting to these resources in the web app when you click on any of the storage badges of any blob.

We just need to add this logic to the api so it returns the same built url for every available blob data reference (currently we support google cloud storage and ethereum swarm, but we're planning to propagate blobs to other storages as well such as filecoin or arweave)

A third question: can we add the "transaction fee", the normal gas fee used?

Sure, we can expose this as well.

I'll work on these PRs during these days.

PJColombo avatar Apr 30 '24 01:04 PJColombo

@PJColombo

such as type, to accommodate new codifications like Ethscriptions that are emerging and also create a new "decoder" component in charge of decoding these blobs and storing the decoded data on the db. This way we could show the decoded blob data on the web app and serve it through the api as well.

Sounds about right and awesome! Would love that.

The blob data pointed by these references are public. We're actually redirecting to these resources in the web app when you click on any of the storage badges of any blob.

Yeah, i figured that out and found the public bucket url.

I'll work on these PRs during these days.

Great, waiting for them :P

What are the rate limits on the API? I'm not planning to spam constantly, just maybe few times since a given past block. I'm yet to try out, and will see if i get limited at all. I'm going to fetch from the gcp storage url too which is cached anyway so, i might not even get rate limited in reality.

tunnckoCore avatar Apr 30 '24 01:04 tunnckoCore

@PJColombo btw, you can check ESIP-8 on how to decode Blobscriptions specifically (it's basically all blobs data combined, passed to cbor decode and having content and contentType inside). Cuz in reality it better be labeled Blobscriptions if at all. But yea.

I also have blobscriptions "stateless" indexer, chain listener, which parse and process things and send webhooks to you, or run it on your own and pass handler function that writes to your db

It's here https://github.com/tunnckoCore/blobscriptions

tunnckoCore avatar Apr 30 '24 02:04 tunnckoCore

Another note: include transaction's index :P

tunnckoCore avatar Apr 30 '24 03:04 tunnckoCore

@PJColombo bump

tunnckoCore avatar Jun 27 '24 06:06 tunnckoCore

Hey @tunnckoCore. So sorry for the long wait on this 😥

I merged two PRs: (#424 and #442) that tackle the issues we discussed.

Also, I'm working on a new https://github.com/Blobscan/blobscan/pull/446 that changes the api to now return blob data URLs instead of URIs so it's easier to access the blob data

In regards of the ethscriptions blobs, we're planning to change the rollup field to something more general such as type to being able to support blob types in addition to rollups.

PJColombo avatar Jul 17 '24 14:07 PJColombo

@PJColombo yeah, i noticed the first pr back then.

Thanks! And yeah, something like type makes more sense.

tunnckoCore avatar Aug 16 '24 19:08 tunnckoCore