near-explorer
near-explorer copied to clipboard
near
Export transactions in CSV
Story
It is a feature request from our users. I just log it here to track and discuss it.
It might be quite a resource intensive task easily causing deny-of-service if abused.
Do you mean exporting all transactions? That could potentially be gigabytes of data
I know. Still, etherscan has this feature (see at the bottom of the page: https://etherscan.io/address/0xea674fdde714fd979de3edf0f56aa9716b898ec8)
Did anyone create this? The capability to export transactions is very important to pay taxes
Nope, we have not worked on it yet. We have a revamped UX for the Account Details page, where exporting transactions will be natural, so we will get back to this topic after we land that
Super important that we get CSV export so we can then import the data into any number tax calculation packages. Don't want to rush anyone, but not having this feature puts NEAR at a disadvantage relative to other chains.
Any suggestions on how to work around the missing CSV export?
@jimoquinn There is still no clear path on how to achieve the export given that some accounts have millions of transactions. I feel we might end up with a standalone tool that would pull the data from some data storage solution directly and Explorer will just guide on how to use that tool.
Any suggestions on how to work around the missing CSV export?
Use PostgreSQL directly: https://github.com/near/near-indexer-for-explorer/#shared-public-access
Guys, seriously. All the problems you have raised here were already identified and solved by others before. Just look at the mother of block explorers, Etherscan:
First, at the bottom of every table that shows transactions, there is a simple link for easy accessibility of the feature:
Still, the page that shows up then is a generic one, meaning it doesn't make a difference from where the user came (except of the wallet address, of course). The reason is not for simplicity of implementation, but because the export function is highly restrictive and puts up a lot of constraints that need to be expressed in one simply place. Those restrictions seem not to be very user friendly, in fact the user feels like he is made to jump through a lot of hoops, but they are there to solve all the problems we are talking about here:
- a solid CAPTCHA implementation that implements protective measures against both mass data extraction as well as DoS attacks
- an arbitrary time frame, in the Etherscan case defaulting to the full current month, in order to make the regular export and collection of monthly transaction reports a breeze (I would suggest to default the timeframe to the full last month though, as the end of the current month is very likely still in the future :P)
- a limitation to the earliest (first) 5000 lines in the given timeframe which alleviates for all load, bandwidth and file size problems
- a hint for the user that it might makes sense to specify even smaller time frames to avoid problems in times of high load; better safe than sorry, and it for sure avoids a lot of user support cases
Give all that, it boils down to a "SELECT FIRST 5000 ..." statement and we are there. As said, somewhat restrictive in the perspective of the user, but we have our reasons.
Even when a user wants to have a report for say the last year (my case, just by coincidence), she only has to repeat the export 12 times and then deal with the 12 files (in the case of import into a tax solution like Koinly, it would be sufficient to just do 12 imports, in other cases there might be the need to concatenate the lists somehow). Might be cumbersome, but hey, it is possible at all!
Let's be very clear. The NEAR eco-system's current inability to offer any means of transaction export translates into ignoring the substantial need of investors to report their profits and losses to the IRS or other authorities. For a lot of folks this might be a show stopper, and if not, it still sets NEAR miles apart from established, likely EVM-based networks.
Make up you mind. Please.
What if you queue the requests for exports up on a queue or topic and then email a link to the user when the export is done to download the file? Do you have something like memcache or maybe a Presto cluster setup? Or just allow 1000 - 5000 transactions at a time. It is better than nothing. A tool that would just interrogate the blockchain slowly and produce a report might be a workaround. I stopped using NEAR because it makes my taxes complicated.
I stopped using NEAR because it makes my taxes complicated.
There you have it. It becomes more an more crucial that NEAR gets the must-have capability of transaction reporting via e.g. CSV export.
@frol can you please review and comment on my post (the lengthy one of May 5th)? TIA.
Edit: Woah, I just realized that you are from Kharkiv, Ukraine. Sorry for being pushy, I can imagine you have to deal with way more important things than export functions these days... Take care, bro!
@sriedel1970 @bardionson Thanks for your inputs they definitely inform us on what is needed, so we can prioritize things better. As I said before, we are working on a full Explorer revamp, and this CSV feature will definitely be addressed once we land other features: FT and NFT assets with history of those operations, activity list that includes validator rewards, etc
@sriedel1970 Thanks for being mindful ❤️
@sriedel1970 We will definitely evaluate how other solutions deal with this problem. We could even consider live-populate Google Sheets documents. E-mailing and having sane limits are also good points!
Great news, thanks so much, Vlad. The NEAR community will definitely appreciate those features.
Although my assessment would be that they'd prefer to have a very basic, but early export function much over the power and convenience functions you mentioned, when this will take some time. You know your time line best, just take this into consideration.
Again, stay safe. My feelings are with you and the Ukraine.