bushel - a reconstruction of livejournal for the 2020s

wip spec, if you dont know what this is, its probably not ready for you yet

Bushel is an implementation of the classic ``webring'' of old times, when a smallish group of people would collaborate to share content while maintaining their own websites and online identities.

In a classic webring, each node would generate its own styled HTML site and share data via RSS or Atom. Every node that is consuming data has to parse the Atom feeds, figure out the peer's encodings, and then turn those back into datastructures for the local site.

Bushel instead provides a simpler distributed datamodel that is hopefully more maintainable in the long term. Bushel uses the [irmin] git database to directly share filesystem based data structures across the webring. This avoids having to roundtrip via XML, and allows direct sharing of typed versioned datastructures across nodes owned by different people. Examples of such datastructures include blog posts, wiki entries, social media feeds, git pushes, events and so on. Every datastructure is versioned with migrations so that older feeds can always be upgraded to newer feeds.

Motivation

Working with open source communities in the modern world is a tremendous undertaking due to the sheer number of channels and lines of communication we have to deal with for even simple interactions. A single feature might begin life in an email thread on a mailing list, then migrate to several GitHub issues, show up on Hacker News for some random criticism, be pushed to a git repository or three, be released onto a package manager, get threads on a Discourse forum, be published as a research paper, have some Jupyter notebooks released as a tutorial, and then a video of a conference talk released.

The above is a pretty common example of one person's workflow, but open source rarely happens solo these days. Instead, small groups of people collaborate across these tasks, and this is where tracking the information flow gets complex and manual across all the communication medium.

Goals

Bushel is intended to provide a framework for:

• individual open source contributors to quickly write about their work, and mirror it to some of these communication mediums
• groups of contributors to share content via bidirectional links (e.g. release announcements or blog posts about work done together) with the ability to work in private if they desire
• organisations to track feature flows among bigger groups of individuals working together across many projects
• projects to pull together the contributions of individuals and credit them appropriately, while providing a coherent feature flow to others.

Design

The authoring workflow requires minimal work for the users by:

• generating content mirrored from online sources and convert them into a markdown format ready for human editing
• providing markdown shortcuts to easily reference other sources and projects in the webring database (e.g. @avsm or @avsm/bushel)
• letting humans quickly edit those autogenerated files from their perspective (e.g. a project might writeup a github release differently from an individual that is proud of a particular feature they worked on)
• ensuring all html endpoints are version controlled like a wiki, e.g. so users reference either the latest version of a blog post or a particular revision.
• bidirectional links mean that if you reference someone elses content, their site is rebuilt to reflect your link.

The serving workflow requires a custom OCaml server instead of just static pages, since:

• nodes offer GitHub oAuth to offer per-group privacy settings for content that shouldn't immediately be public. This allows, for example, draft blog posts to be easily shared among authors and then go public at the same time.
• some content can be private for a group indefinitely, for example design discussions. Although open source is eventually released, most small group design is easier when done in private without the whole Internet giving their opinions.
• content can initially be served via HTTPS, but eventually will have bridges to other mechanisms such as a CLI, and GraphQL endpoints of the datamodel for custom clients.

Schema

TODO filesystem schema for authoring