[Discussion] Support non allow-listed catalysts

[nchamo]

The situation today

As of today, there is a list of catalyst servers that is allow-listed by the DAO. Content servers sync with other servers on that list and only those. When a non allow-listed server starts, it will sync with whitelisted servers, but any deployments made to them will not be picked up by any other servers.

The proposal

We could potentially change this by changing this DAO maintained list with a more dynamic list of known servers. This would imply:

1. Allowing servers to register with each other
2. Make each server expose a list of other known servers

By doing this, any new server could use the whitelisted list as a starting point to know all active servers and register with them, therefore becoming a part of the network. Then each server would sync with all servers on the network.

The consequences

This approach would require a lot of changes to how DCL works today, but mainly:

1. We would need to change the way that the synchronization works: A. We would probably want to announce new deployments, instead of asking for them to each known server B. We would need to handle the lifecycle of the list of known servers
2. The explorer relies on the fact that catalyst servers all run the same version, that will no longer be possible to ensure
3. We need to enable the explorer to easily connect to other servers
4. Right now malicious servers can be simply removed from the whitelist and no client will connect to them. Maybe we would need to create a way to report servers, as a way to keep all users safe
5. We would need to have a good communication channel to announce updates, and provide support

[menduz]

1. We would need to change the way that the synchronization works: a. We would probably want to announce new deployments, instead of asking for them to each known server b. We would need to handle the lifecycle of the list of known servers

Should we start thinking more technically about that implementation?

2. The explorer relies on the fact that catalyst servers all run the same version, that will no longer be possible to ensure

I can only think about very well documented API contracts to handle that scenario. Thoughts?

3. Right now malicious servers can be simply removed from the whitelist and no client will connect to them. Maybe we would need to create a way to report servers, as a way to keep all users safe

I can think about a bunch of allowed nodes accepting any "untrusted" pushed deployments. That node is also trusted by the rest of the DAO nodes, and acts like a firewall of messages to avoid DoS on the rest of the nodes. At the end of the day, I believe those configurations will emerge naturally if we create a scheme flexible enough.

4. We need to enable the explorer to easily connect to other servers

We committed to do it.

5. We would need to have a good communication channel to announce updates, and provide support

Updates like version updates? I know mechanisms to schedule protocol updates like the ethereum timebomb or protocol versions. In contrast to resilient API contracts (point 2) which may not require updates.

[nchamo]

Should we start thinking more technically about that implementation?

Well, I think that before going into implementation details, we should discuss the general approach. I can think of two different ways to handle this:

Option 1 It's the option described above. There would be no roles regarding content servers, and the whitelisted servers will only work as a discovery entry point to know all servers currently on the network. When a server starts, it will automatically join the network by registering to already known servers. When a deployment is made to a specific server, then it will make sure to let the others know.

Option 2 We could define two different roles: whitelisted servers and non-whitelisted servers. Non-whitelisted servers will not know about the existence of each other, and they will simply sync with the whitelisted ones. So if server A (non-whitelisted) gets a new deployment, then it re-enacts that same deployment on one of the whitelisted servers. All other servers will automatically sync with it, therefore propagating all changes.

I can only think about very well documented API contracts to handle that scenario. Thoughts?

Yeah, I agree. But I think that the main problem will be new features. Currently, there isn't a good versioning schema. We only have a git commit associated with the release. Once this is fixed, the explorer could have a min required version for each feature

I can think about a bunch of allowed nodes accepting any "untrusted" pushed deployments. That node is also trusted by the rest of the DAO nodes, and acts like a firewall of messages to avoid DoS on the rest of the nodes. At the end of the day, I believe those configurations will emerge naturally if we create a scheme flexible enough.

My main concern is not a DoS actually. My main concern is with the explorer. For example, a malicious server could serve corrupted content on purpose. Currently, if someone finds out that this is happening, then that malicious server would be removed from the DAO whitelist and no client would connect to it anymore. If we enable clients to connect to any server, then we would lose that protection layer. Maybe the DAO could have a list of blacklisted servers?

Updates like version updates? I know mechanisms to schedule protocol updates like the ethereum timebomb or protocol versions. In contrast to resilient API contracts (point 2) which may not require updates.

Yeah, I mean new catalyst versions. This is definitely relates to point 2, but I just want to mention that there should be a good communication channel to provide support & announce new versions

[menduz]

Option 1 Option 2

I think Option 1 has fewer moving parts and state-related problems (nodes won't know they are whitelisted or not).

Option 2 sounds more peer-to-peer, but the broadcasting of messages is not peer-to-peer, it is a directed graph broadcasting directly to the whitelisted servers. Since the whole deployment message and the contents are verifiable I would rather broadcast the update to all connected peers, or choose Option 1.

To summarize: Full P2P Option 2 sounds good, otherwise I'd bet on Option 1 to start.

My main concern is not a DoS actually. My main concern is with the explorer. For example, a malicious server could serve corrupted content on purpose. Currently, if someone finds out that this is happening, then that malicious server would be removed from the DAO whitelist and no client would connect to it anymore. If we enable clients to connect to any server, then we would lose that protection layer. Maybe the DAO could have a list of blacklisted servers?

Content is not corruptible, the CID (filename) of the content is merely a hash over the content data. And the whole deployment signs the list of files, we have a very decent trust chain to verify the deployments.

const x = require("ipfs-only-hash");

export function cid(buffer: Uint8Array): Promise<string> {
  return x.of(buffer, { cidVersion: 1 });
}

[nchamo]

To summarize: Full P2P Option 2 sounds good, otherwise I'd bet on Option 1 to start.

Content servers already know if they are whitelisted or not. Given the current state of the project, I think that option 2 would be easier to implement. However, I personally also prefer option 1, since it's more generic, and there is no need for roles or special cases.

Content is not corruptible, the CID (filename) of the content is merely a hash over the content data. And the whole deployment signs the list of files, we have a very decent trust chain to verify the deployments.

Every server validates deployments, so there is no harm possible when propagating new deployments. However, my concern is that a malicious server could affect explorer clients. For example, let's say that Hash(File) = H. A server could be modified so that when a client wants to download the file with the hash H, another file is returned. I'm not aware of a validation happening after all content servers have synced, so it could go undetected.

So my point is that right now, if a case like this is detected, the server would be removed from the DAO whitelist and no explorer would connect to it. If we were to allow the explorer to connect to any server, we would lose that protection level.