archethic-node icon indicating copy to clipboard operation
archethic-node copied to clipboard

Optimize DB stream chain

Open Neylix opened this issue 1 year ago • 0 comments

Describe the problem you discovered

Currently there is 2 functions in the ChainReader that read multiple transactions, stream_chain and get_transaction_chain. stream_chain return a stream from the beginning of the chain in asc order get_paginated_chain return a batch of 10 transactions in asc or desc order

Currently stream_chain use the same sub function process_get_chain than get_transaction_chain and it return a batch of 10 transactions while the stream could send transaction by transaction after reading it. Also this sub function does some extra work to calculate the field to fetch while this could be done only once before the stream.

In other side when we want to stream a chain in reverse order or from a paging_address we can only use get_transaction_chain which means open the file -> read 10 transactions -> close the file -> send these tx in the stream -> loop again to open file ... This is not really efficient

Describe the solution you'd like

I think we could have only one function stream_chain that can take an order and a paging address. This function could send transaction by transaction on asc order (need to do batch of 10 tx in desc order) and it would open and close the file only once.

Additional context

Epic: #261

Neylix avatar May 31 '23 11:05 Neylix