boost icon indicating copy to clipboard operation
boost copied to clipboard

Request to expand miner filter hooks

Open ribasushi opened this issue 5 years ago • 2 comments

A request from Murmuration Labs is to expand the set of hook slots for external programs. Currently lotus provides Filter which runs before storage-deal acceptance, and determines whether to accept the deals bytes based on wallet/peerid/payload-cid/other criteria. Request is to expand these hooks as follows:

  • ~StorageNegotiationFilter : same as current Filter: receives the JSON of the deal on STDIN and indicates with exit0/1 whether the payment/data for the deal should be accepted~ implemented as Filter
  • StorageContentFilter: when specified, upon receipt of the entire DAG from the client, but before inserting it into a WaitDeals sector, an external program receives the DAG determinisitcally on STDIN as a .car stream, and indicates with exit0/1 whether to proceed
  • ~RetrievalNegotiationFilter: same as StorageNegotiationFilter but for retrieval deals ( allows blocking of known-unwelcome wallets/peerids, or commP/payloadCIDs that became classified as problamtic material post-sealing )~ implemened as RetrievalFilter
  • RetrievalContentFilter: same as StorageContentFilter, the dag is fed to on STDIN prior to graphsyncing the first byte to the client.

It is likely that the 4 hooks will invoke portions of the same suite, which will coordinate caching and known-block-list updates out of band.

ribasushi avatar Sep 25 '20 20:09 ribasushi

~Closing ticket for now, the ask was misunderstood, sorry for the noise!~ No, the task is apparently still needed.

ribasushi avatar Sep 25 '20 21:09 ribasushi

for RetrievalContentFilter - we should try to understand if there's a way to implement such a filter deterministically rather than in the critical retrieval path

willscott avatar Mar 30 '23 15:03 willscott