Hackathon: Examples of common distributed design patterns

[jhellerstein]

For now these should go in examples or the book. Eventually these should be in libraries of importable modules.

Local wall-clocks, etc

• [ ] heartbeats
• [ ] timers/timeout
• [ ] nonce generation
• [ ] counter

Local ordered data structures

• [ ] local FIFO queue
• [ ] local stack
• [ ] local priority queue

Distributed clocks

• [x] Lamport clock
• [x] Vector clock #435

Networking

• [ ] causal delivery
• [ ] reliable delivery
• [ ] multicast
• [ ] randomized gossip

MPI-Style Collective Communication

• [ ] tree aggregation
• [ ] tree dissemination
• [ ] barrier
• [ ] scatter (partition collection across all recipients)
• [ ] gather (assemble partitions into whole at one node)
• [ ] allgather (assemble partitions into whole replicated at all nodes)
• [ ] reduce (distributed)
• [ ] allReduce (distributed)

Storage

• [ ] write/read to/from file formats
• [ ] write/read to/from db formats

Network Services

• [ ] invoke REST endpoints
• [ ] provide REST endpoints

[MingweiSamuel]

#330

[MingweiSamuel]

Termination detection (#928)

[shadaj]

Spark examples: https://spark.apache.org/examples.html

[shadaj]

1 billion rows challenge: https://github.com/gunnarmorling/1brc