Add a timestamp to balances and cheques

[holisticode]

In Swap, we store balances and cheques in a StateStore.

If a balance or a cheque never gets settled, it will stay forever there. It also means that over time, we might get entries which get stale because the peer disappeared or we dropped it and we don't communicate with it.

It is rather a difficult problem to resolve what to do with these entries.

Having at least a timestamp helps identify how old a LastExchanged entry is, in order to help evaluate what to do with it.

In the future, even an archiving of "older" balances or cheques might be considered.

[holisticode]

@janos 's opinion highly welcomed

[janos]

I would consider this a very reasonable requirement, even very important.

[holisticode]

Actually I wanted your opinion on archiving entries, and how to do it, because you have extensively investigated LevelDB.

However, I now realize that this ticket is about adding a timestamp only, to be able to analyze balance and cheques data. We can then decide about archiving later.

[janos]

I see, different prefixes for archived keys and regular ones in StateStore is the way to go. There already is the Iterate method with prefix and moving data from one key to anther can be done in a batch.

For optimization, there could be even a timestamp "index", prefixed keys ordered by timestamps so that iteration on discovering staled balances and cheques would be less costly.

[mortelli]

few questions here:

• is there a size limit to each store?
• would we be able to tell if the data we need is archived? how?

[holisticode]

• would we be able to tell if the data we need is archived? how?

=>

different prefixes for archived keys and regular ones in StateStore is the way to go

[mortelli]

maybe i'm not following here, but:

suppose i want a cheque for peer p. how do i know if this is stored in the regular store or the archived one? wouldn't the keys be named by date?

[holisticode]

You'd probably first search the "live" keys, and if not found, the "archived" ones.

But keep in mind that we did not decide yet if we are going to archive at all. That's why we need the timestamps in order to make informed guesses :)

[janos]

We should see what kind of queries there would be. All cheques and balances (live and archived) can be under a single key prefix, but that they contain a field to mark them as archived. And to keep an "index" by their timestamps for iterations by their age.

I think that it would be good to know how archived entries would be handled before organising the data.