spacedrive icon indicating copy to clipboard operation
spacedrive copied to clipboard

Published 20 hours ago • verified publisher icon spacedriveapp

[ENG-42] P2P communication

Open oscartbeaumont opened this issue 3 years ago • 2 comments

This is a draft but I though I would open a PR as I wanna get it cleaned up and merged as soon as possible. The PR will only function for P2P communication over a location network. Support for communicating over the internet will come at a later date.

TODO major:

  • [x] Documentation comments on structs, traits, etc
  • [x] Error handling
  • [x] Remove all Rust warnings on build
  • [x] Shutdown handling - close conns, unregister from mdns + global
  • [x] Implement discovery stack idea to aid separation of concerns
  • [x] Make helper which can receive from socket & rmp decode with error handling!
  • [x] Logging system
  • [x] Change P2PData from typedef into struct.
  • [x] Sending mock sync messages -> Like Brendan's CRDT layer is going to
  • [x] Peer Online/Offline working -> This is nearly done, just hook up with UI
  • [x] Update Linear with current progress
  • [x] Persist P2P Identity to disk in the node_config.json file
  • [ ] Attempt to reestablish lost network connections
  • [x] Rename p2p2 to p2p directory
  • [ ] Fix slow pairing over network -> due to looping synchronously and trying IP's. -> Instead make a guess then async loop.
  • [ ] Sanitise events (so when multiple connections are opened/closed they aren't always emitted to user)
  • [ ] Documentation

TODO minor:

  • [ ] MDNS and Global discovery should merge PeerCandidate's instead of overriding.
  • [ ] Investigate moving from SHA-1 to SHA-256 or SHA-255 for PeerId's
  • [ ] Move from DashMap/DashSet to Mutex uglyness -> Also remove the tokio::time::sleep hack that I left in.
  • [ ] Rename server.rs nm-server.rs and put in on the NetworkManager struct
  • [ ] Fix Typescript errors due to cross crate types
  • [ ] Only allow joining the same if not newer version of Spacedrive + have UI warning on both devices for failure reason
  • [ ] Does Tauri work with ctrlc cause it's import it is run on shutdown
  • [ ] Make Cloudflare only advertise IPv4 address of Fly.io

oscartbeaumont avatar Jul 12 '22 10:07 oscartbeaumont

ENG-42 P2P Communication

Space tunnel:

  • [X] Prometheus Metrics
  • [X] Restrict log output
  • [X] Client certificate authentication
  • [ ] Server certificate authentication - Client should check server matches what they expect
  • [ ] Make Cloudflare only advertise IPv4 address of Fly.io → Document this
  • [X] Error handling - Remove all `.unwrap()`
  • [X] Deploy Redis
  • [X] Storing and reading client announcements from Redis
  • [ ] Fetching available discovery servers via HTTP from the static JSON DB
  • [ ] Allow querying multiple announcement in the same query
  • [ ] Proxy network packets between devices
  • [ ] Documentation

P2P:

  • [ ] Client communication protocol implemented

    • [X] Pairing process - Find and connection without being blocked by system??
    • [ ] Unpairing process
    • [ ] Sync message
    • [ ] Request/Return small file

  • [ ] Advertise IP's from STUN to Spacetunnel

  • [ ] Allow using multiple STUN servers

  • [ ] UDP hole punching

  • [ ] Reestablishing lost network connections

  • [X] Move data off the Server as it is a confusing name.

  • [ ] ServerCertVerifier logic

  • [ ] ClientCertificateVerifier logic

  • [ ] Test how Drop ing a channel could break stuff and stop it from doing that.

  • [ ] Investigate moving to SHA-256 or SHA-254 for PeerId

  • [ ] Binding the UDP server to a port set by the user.

  • [ ] Limit number of connections that can be open at a single time

  • [ ] Handle device having too many network addresses to fix in DNS record

  • [ ] Investigate certificate expiry

  • [ ] Data Compression

  • [ ] Documentation

Merging into Spacedrive:

  • [ ] Loading files over Rust bridge
  • [ ] View NetworkManager.get_state() in settings panel for debugging.
  • [X] Build pairing UI
  • [ ] Build unpairing UI
  • [ ] Events for showing when device is connected or not
  • [ ] Is the `ctrlc` handler called when Tauri shuts down??
  • [ ] Debug button  in UI - Export OS, discovery servers, UUID’s, etc

-— Future stuff ---

Space tunnel:

  • [ ] Prevent leaking metrics to public
  • [ ] Investigate certificate renewal → Maybe allow each server to advertise multiple certificates
  • [ ] Move sdtunnel from @oscar's personal account to company account
  • [ ] Setup autoscaling & mutliregion deploy in Fly.io
  • [ ] Setup healthchecks in Fly.io
  • [ ] IPv6 support → We are limited by Fly.io for this one.

linear[bot] avatar Jul 12 '22 10:07 linear[bot]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
spacedrive-landing ✅ Ready (Inspect) Visit Preview Aug 8, 2022 at 1:01PM (UTC)
spacedrive-storybook ✅ Ready (Inspect) Visit Preview Aug 8, 2022 at 1:01PM (UTC)
spacedrive-web ✅ Ready (Inspect) Visit Preview Aug 8, 2022 at 1:01PM (UTC)

vercel[bot] avatar Jul 12 '22 10:07 vercel[bot]