CoinTaxman
CoinTaxman copied to clipboard
provinzio
Kraken: Action Types in CSV Exports
Summary
The current implementation for reading the Kraken CSV exports is according to the documentation of the "type" field: https://support.kraken.com/hc/en-us/articles/360001169383-Explanation-of-Ledger-Fields
However, it seems like the CSV exports have changed and do not correspond to the definition above anymore. This leads to completely different behavior for some action types.
For the exports that I have, I can see the following behavior:
To-do list
- [x] Fix staking / unstaking
- [x] Staking rewards: Recognize reward entries
- [ ] Staking rewards: Stake rewarded coins correctly (not FIFO)
- [x] Update logic for deposits / withdrawals
- [ ] Resolve coin names, e.g. XBT/BTC (for matching withdrawals / deposits)
- [x] Kraken API: Try inverse coin pair, e.g. for EUR/USD
- [x] Kraken API: Update logic to support virtual sells
- [ ] Margin trading: Closed positions
- [ ] Margin trading: Settled positions
Actions that currently do not work
Staking / Unstaking
A combination of "withdrawal"/"deposit" and two "transfer" rows. The current implementation considers "transfer" as airdrop. Depending on how the CSV exports looked before, it could be more or less complicated to guarantee backwards-compatibility. Also related to https://github.com/provinzio/CoinTaxman/issues/57. For https://github.com/provinzio/CoinTaxman/issues/87, a similar behavior was observed, although slightly different (four "transfer" rows). For ETH, replace asset with XETH and asset.S with ETH2.S.
Staking
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| "" | refid 1 | date time 1 | "withdrawal" | "" | "currency" | asset | - amount | fee | "" |
| "" | refid 2 | date time 2 | "deposit" | "" | "currency" | asset.S | amount | fee | "" |
| txid 1 | refid 1 | date time 3 | "transfer" | "spottostaking" | "currency" | asset | - amount | fee | balance |
| txid 2 | refid 2 | date time 4 | "transfer" | "stakingfromspot" | "currency" | asset.S | amount | fee | balance |
Unstaking
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| "" | refid 1 | date time 1 | "withdrawal" | "" | "currency" | asset.S | - amount | fee | "" |
| "" | refid 2 | date time 2 | "deposit" | "" | "currency" | asset | amount | fee | "" |
| txid 1 | refid 2 | date time 3 | "transfer" | "spotfromstaking" | "currency" | asset | amount | fee | balance |
| txid 2 | refid 1 | date time 4 | "transfer" | "stakingtospot" | "currency" | asset.S | - amount | fee | balance |
Staking Rewards
According to the Kraken documentation, staking rewards should be listed as "reward". However, they are actually logged as "deposit" and "staking" actions. The "deposit" row would probably mess with https://github.com/provinzio/CoinTaxman/issues/4 and "staking" is currently not known and throws an error. We should be careful when adding the "staking" operation: We want to add the rewarded coin to staking, not just any via FIFO. For ETH, replace asset.S with ETH2 (not ETH2.S as for ETH staking above).
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| "" | refid 1 | date time 1 | "deposit" | "" | "currency" | asset.S | reward | fee | "" |
| txid | refid 2 | date time 2 | "staking" | "" | "currency" | asset.S | reward | fee | balance |
Margin Trades
Margin trades are opened with "margin" and closed with "margin"/"settled", in-between there may be "rollover" fees for the open position. I could not find any "margin trade" as documented on the Kraken website. The row "margin" is currently not known and throws an error. "settled" and "rollover" rows are detected but also throw an error because margin trades are not supported yet (related to https://github.com/provinzio/CoinTaxman/issues/52). For now, I would suggest to skip these lines with a warning, but still process the rest of the CSV.
Closed Position
Note that the CSV export only contains the base currency for the position (e.g. ZEUR for BTC/EUR). There is no way of finding out for which crypto the position was opened (BTC in this case). Also, there is no way of linking the opening and closing actions ("margin" types) as they have different "refid" values.
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| txid 1 | refid 1 | date time 1 | "margin" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 2 | refid 1 | date time 2 | "rollover" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 3 | refid 1 | date time 3 | "rollover" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 4 | refid 2 | date time 4 | "margin" | "" | "currency" | base asset | gain/loss | fee | base balance |
Settled Position
There is no way of linking the opening and settling actions as they have different "refid" values. The sign of "amount" depends if the position was settled with gains or losses.
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| txid 1 | refid 1 | date time 1 | "margin" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 2 | refid 1 | date time 2 | "rollover" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 3 | refid 1 | date time 3 | "rollover" | "" | "currency" | base asset | 0 | fee | base balance |
| txid 4 | refid 2 | date time 4 | "settled" | "" | "currency" | base asset | base amount | fee | base balance |
| txid 4 | refid 2 | date time 4 | "settled" | "" | "currency" | asset | - amount | fee | asset balance |
Adjustment / Sale
Two further types could occur in the CSV export according to the documentation: "adjustment" and "sale". I don't think we need to implement them for now as they cover only very special actions.
Actions that work
The following actions work with the current implementation and are just for information.
Deposit / Withdrawal
Two "deposit" / "withdrawal" rows.
Deposit
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| "" | refid | date time 1 | "deposit" | "" | "currency" | asset | deposited amount | fee | "" |
| txid | refid | date time 2 | "deposit" | "" | "currency" | asset | deposited amount | fee | balance |
Withdrawal
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| "" | refid | date time 1 | "withdrawal" | "" | "currency" | asset | - withdrawn amount | fee | "" |
| txid | refid | date time 2 | "withdrawal" | "" | "currency" | asset | - withdrawn amount | fee | balance |
Spend / Receive (Trade via the Buy Crypto button)
A combination of "spend" and "receive" rows. Example for buying crypto with fiat:
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| txid 1 | refid | date time 1 | "spend" | "" | "currency" | base asset | - base amount | fee | base balance |
| txid 2 | refid | date time 1 | "receive" | "" | "currency" | asset | amount | fee | balance |
Trades
A combination of two "trade" rows. Example for buying crypto with fiat:
| txid | refid | time | type | subtype | aclass | asset | amount | fee | balance |
|---|---|---|---|---|---|---|---|---|---|
| txid 1 | refid | date time 1 | "trade" | "" | "currency" | base asset | - base amount | fee | base balance |
| txid 2 | refid | date time 1 | "trade" | "" | "currency" | asset | amount | fee | balance |
Hi @shredEngineer @wearymanateevedaknotstabooresisting, I'm asking you since you worked on previous Kraken implementations: Do you happen to have old ledger CSV exports that include staking, unstaking and staking rewards? Kraken apparently has changed the action types in the ledgers.csv and I want to ensure backwards compatibility before implementing the changes. Also, if you have any records of margin trades, that would be helpful. Thank you very much!
@Griffsano Sorry, I don't have staking or margin trades in my data. :/
Hi all, Is somebody working on this one? I get an error for type staking on my kraken report. See my error below. Happy to hear from you soon! Happy new year!
2022-01-02 14:11:04,251 book ERROR /home/pi/CoinTaxman/account_statem ents/ledgers.csv: 13: Other order type 'staking' is currently not supported. Ple ase create an Issue or PR.
Traceback (most recent call last):
File "/home/pi/CoinTaxman/src/main.py", line 44, in <module>
main()
File "/home/pi/CoinTaxman/src/main.py", line 32, in main
status = book.read_files()
File "/home/pi/CoinTaxman/src/book.py", line 904, in read_files
self.read_file(file_path)
File "/home/pi/CoinTaxman/src/book.py", line 859, in read_file
read_file(file_path)
File "/home/pi/CoinTaxman/src/book.py", line 555, in _read_kraken_ledgers
raise RuntimeError
I don't know if I can allocate time for this right now, but I do know that we would need some sample data beforehand. :) So if you could provide us with some real exported data on this, that would certainly help. I'm no Kraken power user myself, so I don't have exports with staking in it.
Will be happy to share. How can we do that best? I have created a csv, with no fees, amount and balance. Still dates are there and txIDs.... So sharing this in public may be a bit odd.... Any idea is welcome.
Txids=replace with something Date=if you feel uncomfortable with them change them to other dates Amount and Fee would be still nice but you can anonimize them if you want
Ok. I have removed txids and altered the dates. This is just a snapshot of the data....
I will be happy to test any fix you push.
"txid","refid","time","type","subtype","aclass","asset","amount","fee","balance"
"","QCCNPU5-JCHILF-COMBWG","2021-02-10 10:06:39","deposit","","currency","ZEUR",500.0000,0.0000,""
"","QCCNPU5-JCHILF-COMBWG","2021-02-10 10:08:19","deposit","","currency","ZEUR",500.0000,0.0000,500.0000
"","TOUPQD-AQOLA-AYA2VU","2021-05-11 18:55:10","trade","","currency","ZEUR",-200.0000,0.3200,299.6800
"","TOUPQD-AQOLA-AYA2VU","2021-05-11 18:55:10","trade","","currency","DOT",6.2500000000,0.0000000000,6.2500000000
"","TCSZJV-5YGSR-E3OQ6Q","2021-05-18 17:46:21","trade","","currency","ZEUR",-100.0000,0.1600,199.5200
"","TCSZJV-5YGSR-E3OQ6Q","2021-05-18 17:46:21","trade","","currency","DOT",4.1666666600,0.0000000000,10.4166666600
"","BUHHIWP-UBPSTP-CO65OD","2021-05-18 17:49:42","withdrawal","","currency","DOT",-10.4166666600,0.0000000000,""
"","RUI4U34-RQ42JX-Q334OO","2021-05-18 17:52:20","deposit","","currency","DOT.S",10.4166666600,0.0000000000,""
"","BUHHIWP-UBPSTP-CO65OD","2021-05-18 17:52:28","transfer","spottostaking","currency","DOT",-10.4166666600,0.0000000000,0.0000000000
"","RUI4U34-RQ42JX-Q334OO","2021-05-18 17:52:46","transfer","stakingfromspot","currency","DOT.S",10.4166666600,0.0000000000,10.4166666600
"","RUIGB6Q-GDOWY2-RRO72B","2021-05-20 01:15:51","deposit","","currency","DOT.S",0.0044380200,0.0000000000,""
"","ST3YXL2-5KJ7G-6WTIXW","2021-05-20 12:07:57","staking","","currency","DOT.S",0.0044380200,0.0000000000,10.4211046800
"","TP7MRH-JKLNG-OYGXHJ","2021-05-23 19:36:03","trade","","currency","ZEUR",-100.0000,0.1600,99.3600
"","TP7MRH-JKLNG-OYGXHJ","2021-05-23 19:36:03","trade","","currency","DOT",5.5555555500,0.0000000000,5.5555555500
"","BUHIYDN-EEDPTT-5MZ2OQ","2021-05-23 19:53:14","withdrawal","","currency","DOT",-5.5555555500,0.0000000000,""
"","RUIUE4T-FKBS3P-6CRCQV","2021-05-23 19:56:27","deposit","","currency","DOT.S",5.5555555500,0.0000000000,""
"","BUHIYDN-EEDPTT-5MZ2OQ","2021-05-23 19:56:41","transfer","spottostaking","currency","DOT",-5.5555555500,0.0000000000,0.0000000000
"","RUIUE4T-FKBS3P-6CRCQV","2021-05-23 19:56:50","transfer","stakingfromspot","currency","DOT.S",5.5555555500,0.0000000000,15.9766602300
"","RUINS55-CEAUJA-XGFBE5","2021-05-23 01:05:35","deposit","","currency","DOT.S",0.0140788800,0.0000000000,""
"","STSTDNK-FJMJY-W3A22T","2021-05-23 04:09:48","staking","","currency","DOT.S",0.0140788800,0.0000000000,15.9907391100
"","RUIVIHC-PFD4VQ-GJZ5K6","2021-05-23 01:05:28","deposit","","currency","DOT.S",0.0157602700,0.0000000000,""
"","ST6EP53-OLJO3-V7RAD7","2021-05-23 04:18:19","staking","","currency","DOT.S",0.0157602700,0.0000000000,16.0064993800
"","RUIADK5-7RA3R7-CSKSUB","2021-06-02 01:05:42","deposit","","currency","DOT.S",0.0210345700,0.0000000000,""
"","STFECFO-HNIES-7YWRIJ","2021-06-02 04:16:28","staking","","currency","DOT.S",0.0210345700,0.0000000000,16.0275339500
"","RUIIBQS-HIN67L-TOUO7J","2021-06-03 01:05:34","deposit","","currency","DOT.S",0.0157962600,0.0000000000,""
"","STNF5CY-XY37V-YETWHE","2021-06-03 04:17:24","staking","","currency","DOT.S",0.0157962600,0.0000000000,16.0433302100
"","RUIUGZ4-Z53SNL-2E63XM","2021-06-07 01:05:50","deposit","","currency","DOT.S",0.0210829800,0.0000000000,""
"","ST7R3DW-UUQPB-OYYCHT","2021-06-07 04:26:32","staking","","currency","DOT.S",0.0210829800,0.0000000000,16.0644131900
"","RUIT3NO-FTZXB6-4BC7HI","2021-06-10 01:05:42","deposit","","currency","DOT.S",0.0158325700,0.0000000000,""
"","STSZLWF-WJB2U-TJVUH7","2021-06-10 04:20:39","staking","","currency","DOT.S",0.0158325700,0.0000000000,16.0802457600
"","RUIQIO3-CO5FSQ-M7W4YS","2021-06-10 01:05:54","deposit","","currency","DOT.S",0.0211314300,0.0000000000,""
"","STB22YP-ZJBEV-6WZNCR","2021-06-10 04:26:08","staking","","currency","DOT.S",0.0211314300,0.0000000000,16.1013771900
"","RUICG6X-5VOBWX-EZEZVZ","2021-06-10 01:05:42","deposit","","currency","DOT.S",0.0158689500,0.0000000000,""
"","STMRQ6Z-QI43Z-4REUM5","2021-06-17 04:26:17","staking","","currency","DOT.S",0.0158689500,0.0000000000,16.1172461400
"","RUIPP26-ZMQAB3-JBHIG3","2021-06-21 01:05:49","deposit","","currency","DOT.S",0.0211800100,0.0000000000,""
"","STLXB5D-IIUK3-P4X5RK","2021-06-21 04:50:34","staking","","currency","DOT.S",0.0211800100,0.0000000000,16.1384261500
"","RUIMZKX-CDI5G5-ZKM6OY","2021-06-24 01:05:51","deposit","","currency","DOT.S",0.0159054000,0.0000000000,""
I'm not sure my problem fits here, but I get the following error for a kraken import:
ledgers.csv row 7: Parameters for refid QYTM757-VSJIHJ-JGLBQP (Deposit) do not agree: coin.
The assertion fails at this line:
book.py", line 650, in _read_kraken_ledgers assert ( AssertionError: coin
And it stems from the following two values 'EUR.HOLD'=='EUR'.
Any ideas?
CoinTaxman is trying to match kraken internal deposit and withdrawal operations together. It expects two refids (QYTM757-VSJIHJ-JGLBQP) to have the same operation type, change amount and same coin. In your case it seems like you have two lines in your csv with the same refid but different coins.
... and it looks like kraken is using weird symbols.... try to replace EUR.HOLD with EUR for a quick fix. Does this work for you? Do you have other coins in your account statement with the suffix .HOLD?
Just to clarify, I do indeed have two lines with the same transaction and this is what it looks like
- [missing txid] QYTM757-VSJIHJ-JGLBQP date deposit currency ZEUR 11 0.1
- txid001 QYTM757-VSJIHJ-JGLBQP date deposit currency EUR.HOLD 11 0.1 10.9
Do you have other coins in your account statement with the suffix .HOLD?
Somehow strange, it only appears for deposit and spend and only in a limited time frame. After that time frame it turns into ZEUR. I'll try changing it to EUR and will report.
Btw have a look at core.py. The variable kraken_asset_map is meant for this kind of conversion. :) I'll hope that this fixes your problem, so that we can add EUR.HOLD to the list for correct conversion.
It ran through successfully! Thanks for the tip! You can add it to the kraken_asset_map. Or I can add it and do a PR.