Populate tx_status_cache before it's used, not after. Only compute short_id when requested

[nabijaczleweli]

trafficstars

The endInsertRows() call triggers the main calls to get_data_for_role(): in the current order, the try/except goes to the exception every time

In the updated order, it doesn't do it ever

We have full control over what's in the cache, and we always fully populate it, so we can get rid of the fallback

[nabijaczleweli]

On the wallet from #9958/#6625, the overall time to pre-compute this is about 90ms. When instrumented as

from timeit import default_timer as timer
GLOBULE = 0
GLOBULE_COUNT = 0
class HistoryNode(CustomNode):

    model: 'HistoryModel'

    def get_data_for_role(self, index: QModelIndex, role: Qt.ItemDataRole) -> QVariant:
        start = timer()
        ret = self.get_data_for_role_(index, role)
        end = timer()
        global GLOBULE
        global GLOBULE_COUNT
        GLOBULE += end - start
        GLOBULE_COUNT += 1
        return ret

    def get_data_for_role_(self, index: QModelIndex, role: Qt.ItemDataRole) -> QVariant:
        # real impl

with the data zeroed at top of refresh() and read out at bottom, I see, before:

1.4012  132348
1.4551  132348
1.4161  132348

after:

3.9735  132348
3.8653  132348
4.1097  132348

Just measuring refresh e2e, without the instrumentation above (but excluding the ~2.4s wallet.get_full_history() takes, so measuring from after if transactions == self.transactions: return): before

6.713636453729123
6.914032700937241
6.382696805987507

after

3.733156899921596
3.6829894930124283
3.5451462320052087

So this is a ~2x speed-up.

[nabijaczleweli]

The short_id patch wins another 4% on average:

3.512978475075215
3.5296032559126616
3.4849150171503425

[ecdsa]

why is this cache not part of wallet.py ?

[nabijaczleweli]

One has to assume it's because the model views a snapshot of the data, so the cache is bundled with the snapshot and also for performance?

[nabijaczleweli]

Is this missing anything?

[f321x]

For me this throw tons of exceptions on startup (seemingly multiple for each txid):

Traceback (most recent call last):
  File "/home/user/code/electrum-fork/electrum/gui/qt/history_list.py", line 83, in lessThan
    item1 = self.sourceModel().data(source_left, ROLE_SORT_ORDER)
  File "/home/user/code/electrum-fork/electrum/gui/qt/custom_model.py", line 94, in data
    return node.get_data_for_role(index, role)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/user/code/electrum-fork/electrum/gui/qt/history_list.py", line 132, in get_data_for_role
    status, status_str = self.model.tx_status_cache[tx_hash]
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyError: 'd4decca1cc2f2c110056864021a49c861b121e0dc52a7e9e36fd1d532bc2a9e7'

Seems like there is some situation in which the cache is not properly populated before calling get_data_for_role().

[nabijaczleweli]

I can't repro this but fall-back re-added, performance appears unaffected