Broken encryption when direct dialing

[fr0zn]

The snippet below demonstrates a scenario where encryption is not upheld. In this example, there are three clients involved. The first client creates a database and inserts a Post entry into it. The payload of the Post entry is encrypted specifically for client3. Subsequently, client2 contacts client1 and synchronizes the post. At this point, one would expect the post to be encrypted. Finally, client3 also contacts client1 and synchronizes the post.

When fetching the data, the expected behavior sometimes functions correctly, while other times it does not:

I don't know what going on here, is this an internal race-condition?

---

Running the following POC with rm peerbittest; ts-node-esm documentstorelate.ts:

import { field, variant } from "@dao-xyz/borsh";
import { Program } from "@peerbit/program";
import { Peerbit } from "peerbit";
import { DeleteOperation, Documents, Observer, PutOperation, SearchRequest } from "@peerbit/document";
import { X25519Keypair } from "@peerbit/crypto";


function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

@variant(0) // version 0
class Post {
	@field({ type: "string" })
	id: string;

	@field({ type: "string" })
	message: string;

	constructor(id: string, message: string) {
		this.id = id;
		this.message = message;
	}
}

@variant("posts")
class PostsDB extends Program {
	@field({ type: Documents })
	posts: Documents<Post>;

	constructor() {
		super();
		this.posts = new Documents();
	}

	async open(): Promise<void> {
		await this.posts.open({
			type: Post,
			index: { key: "id" },
			canAppend: async (entry) => {
				await entry.verifySignatures();
				const payload = await entry.getPayloadValue();
                console.log('GOT PAYLOAD')
				if (payload instanceof PutOperation) {
					const post: Post = payload.getValue(
						this.posts.index.valueEncoding
					);
					console.log('PUT POST', post)
					return true;
				} else if (payload instanceof DeleteOperation) {
					return false;
				}
				return true
			}
		});
	}
}

const client1 = await Peerbit.create({directory: "./peerbittest/client1"});
const client2 = await Peerbit.create({directory: "./peerbittest/client2"});
const client3 = await Peerbit.create({directory: "./peerbittest/client3"});

const store = await client1.open(new PostsDB());

const post = new Post('ID1', "hello world")

await store.posts.put(post, {
	encryption: {
        keypair: await X25519Keypair.create(),
        reciever: {
            // Who can read the log entry metadata (e.g. timestamps)
            metadata: [
				// client1.identity.publicKey,
				// client2.identity.publicKey,
				// client3.identity.publicKey
            ],

            // Who can read the references of the entry (next pointers)
            next: [
				// client1.identity.publicKey,
				// client2.identity.publicKey,
				// client3.identity.publicKey
            ],

            // Who can read the message?
            payload: [
				// client1.identity.publicKey,
				// client2.identity.publicKey,
				client3.identity.publicKey,
			],

            // Who can read the signature ?
            // (In order to validate entries you need to be able to read the signature)
            signatures: [
				// client1.identity.publicKey,
				// client2.identity.publicKey,
				// client3.identity.publicKey
            ],

        },
    },
});

async function printPosts(store:any) {
    const responses: Post[] = await store.posts.index.search(
        new SearchRequest({
            query: [], // query all
        })
    );
    console.log(responses)
}


console.log('Dialing client2 with client1')
await client2.dial(client1.getMultiaddrs());

console.log('Dialing client3 with client1')
await client3.dial(client1.getMultiaddrs());

//////////////////////
const store2 = await client2.open<PostsDB>(store.address)
// await store2.waitFor(client1.peerId);
//////////////////////

//////////////////////
const store3 = await client3.open<PostsDB>(store.address)
// await store3.waitFor(client1.peerId);
//////////////////////

await sleep(5000)

console.log('Store1:')
printPosts(store)
console.log('Store2:')
printPosts(store2)
console.log('Store3:')
printPosts(store3)

await sleep(5000)

console.log("END")

[marcus-pousette]

Thanks for creating this issue.

As of now, you can query nodes for unencypted version of data, which means that you necessarily does not have to be able to decrypt the payloads to receive decrypted copies of the documents.

If you include canRead fn in your open fn

 async open(): Promise<void> {
        await this.posts.open({
            type: Post,
            index: { key: "id" },
            canAppend: async (entry) => {
                await entry.verifySignatures();
                const payload = await entry.getPayloadValue();
                console.log('GOT PAYLOAD')
                if (payload instanceof PutOperation) {
                    const post: Post = payload.getValue(
                        this.posts.index.valueEncoding
                    );
                    console.log('PUT POST', post)
                    return true;
                } else if (payload instanceof DeleteOperation) {
                    return false;
                }
                return true
            },
            canRead: async (publicKey) => {
                return publicKey?.equals(client3.identity.publicKey) || false;
            }
        });
    }

you can control what publickeys can search for documents, and one who sits on the documents will decrypt them if the canRead check will pass. (canRead by default is () => true which means everyone can read, even though the payloads are encrypted). The reason for why there is a separate way for peers to retrieve documents outside the "encryption at rest" which is what you define during .put is that there are scenarios where you want to protect the data at rest, but leak things when certain conditions are met.

In your example, I also modified the print lines to this

console.log('Store1:')
await printPosts(store)
console.log('Store2:')
await printPosts(store2)
console.log('Store3:')
await printPosts(store3)

so that the prints are in order.

With the changes above the results will be:

Store1 will receive results because, it created them, which means it has an unencrypted local copy. Store2 will receive no results Store3 will receive results because it can decrypt the payload.

Though:

Issues/Bugs I have identified with Peerbit codebase when looking into this, which I ASAP look into.

• When using the canRead, and a node gets rejected. The node will wait until timeout for responses (that will never come). This is why the response times are so long if you make the change above. #128
• The documentation is lacking regarding canRead and that documents can leak by default if you are not modifying the canRead function. #129 * canRead, should perhaps be mandatory to define.
• canRead should also be a callback that not only takes the publicKey of the query, but also the query itself, or some other information, so that more advanced behaviour could be created. Perhaps there should be to be a filter of what queries are performed, but then also a filter for controller what documents are returned. Perhaps there should be a canSearch (before querying) and canRead (after results has been obtained, but before returning them) #130
• By default, you open stores are as a replicator, but if peers put documents in them that you can not decrypt, should you reject in the canAppend (?). The search index duties are not conditioned on what documents you can encrypt/decrypt but the hash of the commit. #131

[marcus-pousette]

All problems have now more or less been solved with the latest release

https://peerbit.org/#/modules/encryption/?id=encrypted-document-store

Only one issue left that I have not fully settled on:

• What should the default behaviour of canRead and canSearch be? False? Throw a Type error if not provided?

Sorry for the long delay of fixing these issues.