bjorn
Import Assistant for CSV/JSON
#41
There usually is a number of transaction types and data fields that simply need to be correlated. At the moment this is done in code only. I would like this to be done in a more flexible and easier to adjust way.
I wish for a graphical import helper for the next other format because I see many wallets out there and already there was changes on an import format, this might not be the last change or other language for an export. This should make it easy for ordinary users to assign what columns contains what data and then iterate on the contents to clear any interpretation.
So e.g. select the column(s) for the timestamp and then clear what the format is supposed to be, so that the app will get the correct interpretation. Select the type of transaction and then assign all possible entries to available transaction types of the app. and so on…
The user must be enabled to save this combined description for reuse alongside the hard coded ones. The data should be stored per user, not just per portfolio. In a way it could be reused for another portfolio. Also this should enable users to provide the configuration for a certain exchange / wallet to other users and the developers, so integrating extensive information for validation of certain fields and missing implementations for e.g. transaction types should help.
Bounty: 500'000 sats
Nice idea, does this make example data obsolete?
I think you meant to ask whether it makes the built-in CSV formats obsolete? Since I'd still want to avoid asking the user for manual setup for known formats, at best some of these hardcoded formats could be replaced by the necessary configuration.
Defining a generic import mapping for CSV files to transactions will not be easy though. I'm sure sometimes it will work out great, but quite often the CSV exports have quirks or differences that may be hard to account for. That said, I definitely see the advantage of this for cases where it does work.
The user must be enabled to save this combined description for reuse alongside the hard coded ones. The data should be stores per user, not per portfolio. Also this should enable users to provide the configuration for a certain exchange / wallet to other users and the developers, so integrating extensive information for validation of certain fields and missing implementations for e.g. transaction types should help.
Regarding storing this per-user, for me that raises the question of where this should be. Currently the only settings that are per-user are a few file system references (things lie last portfolio, last export location, etc.). Not state I'd want to share between PCs and hence this is stored in ~/.config/raccoin/state.json.
In contrast, I'd probably want to be able to check in the CSV configs into my repository and it could be convenient if it is simply part of my portfolio file. Maybe related to how we implement #43. I could also imagine being able to import/export such CSV file definitions using a dedicated file format.
does this make example data obsolete?
No. But it may support both developers that need information and people that do not wish to share their private data. I saw your felt ideal. That is a nice approach (tumble data but keep the structure and write a description of the format as well) as well if it works, for now I did not succeed with it to get the result I imagined.
Defining a generic import mapping for CSV files to transactions will not be easy though.
Maybe I did not understand the mechanics of raccoin. My impression is: it reads CSV files, matches them with known formats (header strings etc.) and based on that also assumes known formats for each row. It does not even recognize when the language is changed. It also wont be working if the number format is not the expected one, so even preparing CSV with Excel might get difficult. Especially what the content (mainly columns) is and what format its cells. My wish as a user simply is a wizard that asks me those details. I may tell it more precise than this
- this column is the date, the first 4 characters are the year, the characters 6-7 is the month, the characters 9-10 are the day
- this column is the time …
- this column is the txid for bitcoin when it is an onchain tx or the nodeid when it was lightning, type hex
- this is the payment reference, type text
- this is a private note, type text
- this is the fee amount in msats
- this is the sent native amount, format is integer
- this is the sent amount in fiat, format is currency US-style float with "." as delimiter and 3 characters as ISO-like code for the currency at the end.
- this is the type of transaction
- …
- header looks like this
I think when someone has a working configuration for the newest fedimint / ecash / taproot wallet export file, they can help others with it by simply sharing the resulting description file. This includes helping any developers. Just do not integrate an automatic data exchange, some things look more trustworthy when done manually.
The approach by @vv01f may also result in a description file usable, just I doubt it being as effective as users determined to make it work. And I would like to be wrong on this.
The state.json is a nice collection of recent usage. Maybe such configuration files not holding private data can be stored next to it in ~/.config/raccoin/ in *nix or e.g. %AppData%\raccoin\ on Windows. Better ideas are welcome, just it should be possible for normal users to exchange the description files for their exchanges.