support for extended eth_subscribe logs

[gholam-deblan]

Describe the Feature

I have not found a way to replicate the following subscription by using provider.on:

provider.send("eth_subscribe",['logs', { fromBlock: "pending", toBlock:"pending", address: "0xFf00000000000000000000000000000000008453", topics:[] }]

provider.on accepts a filter as parameter but it does not seem to replicate the exact 'logs' subscription above, in particular fromBlock and toBlock are being ignored.

Code Example

    const filter = {
        fromBlock: 'pending',
        toBlock: 'pending',
        address: '0xFf00000000000000000000000000000000008453',
        topics:[],
    }
    provider.on(filter, log => {
        console.log(log.transactionHash)
    });

Does not replicate provider.send("eth_subscribe",['logs', { fromBlock: "pending", toBlock:"pending", address: "0xFf00000000000000000000000000000000008453", topics:[] }]

[ricmoo]

When you have a fromBlock and to block, you should use the .getLogs instead.

Is the above filter syntax widely supported by existing nodes?

[gholam-deblan]

The above syntax has been checked and is supported by quite a few providers: I could check Quicknode, Blast, Fastnode, ValidationCloud and ZMok are supporting it. .getLogs works but what I need is a subscription. With web3js this is straightforward using web3.eth.subscribe('logs', filter). This is what prevents me from migrating from web3js to ethers.

[ricmoo]

Ethers normalizes the event responses (such as converting tx values to BigInt, validating addresses, etc), so it needs to know more about the supported events. I can add new events in minor versions if there is a meaningful, concise and useful api that makes sense.

I can also investigate adding some sort of VerboseFilter which allows a raw filter to be passed along, but would mean that the consumer would be responsible for normalizing emitted responses.

[gholam-deblan]

A VerboseFilter would be great as it would give flexibility to power users. Otherwise this is just eth_subscribe 'logs' supporting additional fields in filter, the message produced follow the same format. Then I probably miss the complexity, that may not be easy to adapt.

I am also getting the result I want using the below:

const WebSocket = require('ws');

const webSocket = new WebSocket('wss://yo-yo.quiknode.pro/12345/');

async function subscribeToNewLogs() {
  
  const request = {
    id: 1,
    jsonrpc: '2.0',
    method: 'eth_subscribe',
    params: ['logs', { fromBlock: "pending", toBlock:"pending", address:["0xFf00000000000000000000000000000000008453"], topics:[] }],
  };

  const onOpen = (event) => {
    webSocket.send(JSON.stringify(request));
  };

  const onMessage = (event) => {
    const response = JSON.parse(event.data);
    console.log(new Date(), response);
  };

  try {
    webSocket.addEventListener('open', onOpen);
    webSocket.addEventListener('message', onMessage);
  } catch (error) {
    console.error(error);
  }
}

subscribeToNewLogs();

[kewlfft]

@ricmoo have you decided if you are going to implement something to allow these 'pending' logs subscriptions?

[ricmoo]

The RawFilter (I’ve renamed it from Verbose ;)) will certainly allow it. I’d also like to find out which networks and which nodes support the various formats for pending things. If anyone wants to assist with the research.

Like Infura on mainnet, does it support full pending transactions and if not, what error does it give if you ask for them. and then I need to repeat that for Alchemy, Ankr, etc. for other common events.

[gholam-deblan]

I have tested the following providers for this service and may be able to assist: QuickNode,ValidationCloud,nodiesDLB,GetBlock,NodeReal,Blast,EXTRNODE,NodeRPC,BlockPI,Infura,Ankr,Alchemy,grove (pokt),Kriptonio,1RPC,onfinality,Omnia,ZMOK. I have not saved a lot of information and will need to start again.