[ENG-42] P2P communication

[oscartbeaumont]

This is a draft but I though I would open a PR as I wanna get it cleaned up and merged as soon as possible. The PR will only function for P2P communication over a location network. Support for communicating over the internet will come at a later date.

TODO major:

• [x] Documentation comments on structs, traits, etc
• [x] Error handling
• [x] Remove all Rust warnings on build
• [x] Shutdown handling - close conns, unregister from mdns + global
• [x] Implement discovery stack idea to aid separation of concerns
• [x] Make helper which can receive from socket & rmp decode with error handling!
• [x] Logging system
• [x] Change P2PData from typedef into struct.
• [x] Sending mock sync messages -> Like Brendan's CRDT layer is going to
• [x] Peer Online/Offline working -> This is nearly done, just hook up with UI
• [x] Update Linear with current progress
• [x] Persist P2P Identity to disk in the node_config.json file
• [ ] Attempt to reestablish lost network connections
• [x] Rename p2p2 to p2p directory
• [ ] Fix slow pairing over network -> due to looping synchronously and trying IP's. -> Instead make a guess then async loop.
• [ ] Sanitise events (so when multiple connections are opened/closed they aren't always emitted to user)
• [ ] Documentation

TODO minor:

• [ ] MDNS and Global discovery should merge PeerCandidate's instead of overriding.
• [ ] Investigate moving from SHA-1 to SHA-256 or SHA-255 for PeerId's
• [ ] Move from DashMap/DashSet to Mutex uglyness -> Also remove the tokio::time::sleep hack that I left in.
• [ ] Rename server.rs nm-server.rs and put in on the NetworkManager struct
• [ ] Fix Typescript errors due to cross crate types
• [ ] Only allow joining the same if not newer version of Spacedrive + have UI warning on both devices for failure reason
• [ ] Does Tauri work with ctrlc cause it's import it is run on shutdown
• [ ] Make Cloudflare only advertise IPv4 address of Fly.io

[linear[bot]]

ENG-42 P2P Communication

Space tunnel:

• [X] Prometheus Metrics
• [X] Restrict log output
• [X] Client certificate authentication
• [ ] Server certificate authentication - Client should check server matches what they expect
• [ ] Make Cloudflare only advertise IPv4 address of Fly.io → Document this
• [X] Error handling - Remove all `.unwrap()`
• [X] Deploy Redis
• [X] Storing and reading client announcements from Redis
• [ ] Fetching available discovery servers via HTTP from the static JSON DB
• [ ] Allow querying multiple announcement in the same query
• [ ] Proxy network packets between devices
• [ ] Documentation

P2P:

• [ ] Client communication protocol implemented

  • [X] Pairing process - Find and connection without being blocked by system??
  • [ ] Unpairing process
  • [ ] Sync message
  • [ ] Request/Return small file

• [ ] Advertise IP's from STUN to Spacetunnel

• [ ] Allow using multiple STUN servers

• [ ] UDP hole punching

• [ ] Reestablishing lost network connections

• [X] Move data off the Server as it is a confusing name.

• [ ] ServerCertVerifier logic

• [ ] ClientCertificateVerifier logic

• [ ] Test how Drop ing a channel could break stuff and stop it from doing that.

• [ ] Investigate moving to SHA-256 or SHA-254 for PeerId

• [ ] Binding the UDP server to a port set by the user.

• [ ] Limit number of connections that can be open at a single time

• [ ] Handle device having too many network addresses to fix in DNS record

• [ ] Investigate certificate expiry

• [ ] Data Compression

• [ ] Documentation

Merging into Spacedrive:

• [ ] Loading files over Rust bridge
• [ ] View NetworkManager.get_state() in settings panel for debugging.
• [X] Build pairing UI
• [ ] Build unpairing UI
• [ ] Events for showing when device is connected or not
• [ ] Is the `ctrlc` handler called when Tauri shuts down??
• [ ] Debug button  in UI - Export OS, discovery servers, UUID’s, etc

-— Future stuff ---

Space tunnel:

• [ ] Prevent leaking metrics to public
• [ ] Investigate certificate renewal → Maybe allow each server to advertise multiple certificates
• [ ] Move sdtunnel from @oscar's personal account to company account
• [ ] Setup autoscaling & mutliregion deploy in Fly.io
• [ ] Setup healthchecks in Fly.io
• [ ] IPv6 support → We are limited by Fly.io for this one.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
spacedrive-landing	✅ Ready (Inspect)	Visit Preview	Aug 8, 2022 at 1:01PM (UTC)
spacedrive-storybook	✅ Ready (Inspect)	Visit Preview	Aug 8, 2022 at 1:01PM (UTC)
spacedrive-web	✅ Ready (Inspect)	Visit Preview	Aug 8, 2022 at 1:01PM (UTC)