xous-core
xous-core copied to clipboard
betrusted-io
PDDB key changed notification not working as expected
While implementing the vault server, the key changed notification isn't firing when I expect it to. Need to investigate.
Hello Bunnie,
This is not related, but I experience something kinda worrying, I have 3 PDDB in vault, with 3 different passwords, i've been using it since 0.9.8 release Today I noticed that one of the database now contains another database entries... while I did not do anything except using it, what do I miss and how come that data with 2 different passwords crossed over from one base to another?
I went through sleep/poweron/sleep poweron and unmount, mount, sleep, poweron, umount and such but that's about it.
NB: I have not yet open an issue since i need to investigate the code and i need to try to reproduce somehow, should I?
The tricky bit about using multiple secret Bases is that by default the vault will write into the most recently opened one.
So, let's say you have Bases A and B
You open A, and create an entry "foo" in it. You open B, and create an entry "bar" in it.
Now you should have "foo" in A, and "bar" in B.
Now, you close basis A. Then, you open basis A again. You attempt to write to "bar". It will create an entry in A (most recently opened) for "bar", that can have an entirely different content from the entry for "bar" in B. We'll call this "bar"-prime. The "bar" entry in B is not altered.
Now the situation is you have:
"foo" and "bar"-prime" in A "bar" in B
This behavior is a bit surprising. However, the reasoning for this architecture is that the secret Bases should be thought of as masking layers on top of a base set (and not as disjoint databases that behave independently of each other).
So, let's say you were to have a contact with "John Doe" that is not secret, but occasionally you discuss sensitive matters. You may have "John Doe" in basis A; later on, if you want to have a conversation that is sensitive with "John Doe" you would open Basis B and have the sensitive conversation with "John Doe" and have any chat logs stored in Basis B. Then, you would close Basis B. The result is you now see the chat log in Basis A only, which is some non-sensitive stuff you've conversed with "John Doe". You may consult Basis B later on for the sensitive information, but as long as that Basis is closed, the sensitive information is not visible.
If you had to surrender the device and passwords for inspection, you could just surrender passwords for Basis A. It is more plausible that you have given all the relevant passwords to the inspector if they can see a conversation they were expecting to see with your known contact "John Doe". This consequently gives you the option to forget the existence of Basis B and transitively the existence of any sensitive data within Basis B is effectively erased.
It does get complicated if you have to manage multiple, non-overlapping secret Bases, but the general intent of the design is that as you open more Bases you open them in successive layers of sensitivity, grouping together all the data that may be used in that mode.
First of all thank you, for Novena (which i got too :)) and now for precursor, basically for being productively inspiring. Ok, gotcha, i need to rethink my basis based on that assumption, I misunderstood I guess. Sorry for polluting this issue and thank you for the thorough response, I was not sure where to ask (irc? discord? other?)
Thanks for raising the issue. I will point out that the PDDB is currently in unknown territory when it comes to user experience and explaining what it's trying to do. I am not aware of any famous or common software interaction patterns that I can use as a crutch to pull users through the plausible deniability experience. Thus, we're going to have some growing pain as we understand how users interact with the software and what we need to do to improve that experience. To that end questions like this help me understand the gap between my intent and the user's expectations.
As for where to bring up issues, github is a reasonable forum; probably, I'd say this would have been better served in a stand-alone issue. I encourage people to open issues even on small things because it's easy enough to close them if they are irrelevant or off-topic but generally it helps to create a searchable record. We also have a matrix workspace which you can join through this link: https://matrix.to/#/#precursor.dev:matrix.org. There's a few rooms/topics available, so you don't have to feel like you're shouting at everyone for small issues.
so to comment, this makes perfect sense and is this documented somewhere on the wiki (re explanation on basis above)? I might have missed it. Some comments:
- is that PDDB format documented somewhere? I'd like to attempt writing a package for it for my own use in Go & C as an exercice and to understand the properties.
- could we maybe put a little 'r-|rw' next to the DB that is being open
- now in the example above you mention the bar and "bar"-prime, which one will be displayed? would we see 2 "bar" entries (the one from A and the one from B)?
- is there any "fsck_pddb" integrity checker? in case of corruption or something like that? (I have not checked the code I don't read rust yet) maybe a very naive question...
There isn't a fsck_pddb checker (yet). Maybe that's a tool that should be written. There are consistency checks in the mount process, but it doesn't try to fix any problems it finds, it just reports them.
now in the example above you mention the bar and "bar"-prime, which one will be displayed? would we see 2 "bar" entries (the one from A and the one from B)?
You would see the version belonging to the most recently opened Basis.
There isn't a fsck_pddb checker (yet). Maybe that's a tool that should be written. There are consistency checks in the mount process, but it doesn't try to fix any problems it finds, it just reports them.
now in the example above you mention the bar and "bar"-prime, which one will be displayed? would we see 2 "bar" entries (the one from A and the one from B)?
You would see the version belonging to the most recently opened Basis.
That means that if:
- Mount basisX
- Create entry "bar" (aka barX)
- Mount basisY
- Create entry "bar" (aka barY) (Now "bar" is barY on display, correct?)
- Re-Mount basisX (unmount/mount)
- "bar" is barX now correct? If i edit it will only modify barX
The issue i was facing come on opening and editing an entry in another basis that was just recently open.
Would that make sense to display the most recently basis on top(maybe already the case or with a *) in the "manage basis" menu?
That is correct, that is how it should behave. Basically, the list of Bases are maintained as a Vec, and for the vault, we specify that you should consult the last entry in the Vec for the Basis to use for writing data.
When it comes to reading data, the algorithm works to loop through the Basis list in reverse, looking for the first available key, and only returning "no key found" if none of the Bases report a match.
Yes, there was an intent to maybe do something like a A > B > Sys notification in the top status bar to clarify the current Basis order, but, that feature was forgotten. Maybe open an issue in xous-core, it will help remind me to implement that. It's a good idea, it just never got done.
There isn't a fsck_pddb checker (yet). Maybe that's a tool that should be written. There are consistency checks in the mount process, but it doesn't try to fix any problems it finds, it just reports them.
Cool there is already some i can borrow to make some tests.. maybe an opportunity to learn rust..
Can I use the backup.pddb to extract the decrypted pddb and work outside the device for playing with the data format and trying to understand how this tool would be implemented ? Maybe attempt a standalone first as my rust is 0 so.. i need to quickly learn rust.
Could that help?
now in the example above you mention the bar and "bar"-prime, which one will be displayed? would we see 2 "bar" entries (the one from A and the one from B)?
You would see the version belonging to the most recently opened Basis.
English is terrible sometimes.
Basis is as in "vector basis". When referring to one or more basis the plural is bases. I capitalize the Basis because the plural form Bases makes you think of bases as in a center for operations, like a military base, or a foundational base as in a database, which is not the same meaning. The key point is that a basis is orthogonal, so you can have several of them overlaid on top of each other and tease them apart by convolving against the cryptographic key via the decryption cipher.
I wish the plural was actually something like basisies but unfortunately that is not the case.
There isn't a fsck_pddb checker (yet). Maybe that's a tool that should be written. There are consistency checks in the mount process, but it doesn't try to fix any problems it finds, it just reports them.
now in the example above you mention the bar and "bar"-prime, which one will be displayed? would we see 2 "bar" entries (the one from A and the one from B)?
You would see the version belonging to the most recently opened Basis.
I see the following in the description of backalyzer.py :
and perform a consistency check of all the Basis structures, dictionaries and keys. It can also be instructed to dump all the data.
I guess that's already fsck_pddb correct? :)
Kind of...it's an offline fsck if anything. The main thing is it's not capable of editing the record to fix problems -- you can diagnose them, and it's flexible enough that in the worst case we could do something horrible like brute-force every block to search for old journal entries given a set of keys, but as of now it can't do the key thing that fsck does (in my mind) which is resolve filesystem structures that are corrupted by an ungraceful powerdown. Right now it just triggers a panic and I'm patching over issue piecemeal as they happen.