beaker
beaker copied to clipboard
beakerbrowser
The philosophy of Beaker's platform
Is Beaker a browser, or a new platform built on Web technology?
If it's a browser, we have to be careful about adding APIs which break from other browsers because it makes the stack more difficult for existing vendors to adopt. If it's a new platform, we're free to evolve the model away from the traditional stack with new APIs and program models.
Prior to the 1.0 beta, I had approached Beaker as a browser. We spent 2-3 years developing under that model, and I ultimately concluded it was the wrong approach. Peer-to-peer applications have an entirely different set of needs than hosted applications because there's no remote device with the freedom to "break the sandbox." The platform needs to be complete on the users device.
This need has already manifested in a couple ways, for instance the ability to break CORS to fetch() or the ability to capture page screenshots. It's also a key question for "userlanding" beaker's builtin interfaces such as the diff/merge tool, the editor, and so on.
I think we should view the beaker APIs in the same way that other browsers viewed their extension APIs -- as a wholly new model that we're free to design. I considered starting from the extensions APIs (see also) but that would only make sense if those APIs were exactly what we need, and those are mostly geared toward manipulating the UI of the browser experience.
Obviously this is a balancing act and we should be thoughtful about how we do this, but I'd rather risk going too big than not going big enough.
One other sidenote -- mathias and I are talking about cooking up an ES-module approach to beaker's APIs. We'd leave in the current beaker global but start migrating to this new model, something like:
import { hyperdrive } from 'beaker://api/hyperdrive.js'
import { PeerSockets } from 'beaker://api/peer-sockets.js'
import { screenshot } from 'beaker://api/shell.js'
This has two upsides: 1) it will put userland modules and platform APIs on the same "footing", 2) it will make versioning the APIs easier once import maps land.
Mozilla describes a browser as such:
A web browser takes you anywhere on the internet, letting you see text, images and video from anywhere in the world.
And Wikipedia as such:
A web browser is a software application for accessing information on the World Wide Web. When a user requests a web page from a particular website, the web browser retrieves the necessary content from a web server and then displays the page on the screen.
This closely resembles the current functionality and usage of Beaker, even in the scope of the distributed web and with advanced developer tools and social elements mixed in. I think it is not difficult to argue that hyperdrives used to distribute web pages are still "websites" and part of the "World Wide Web", as they rely heavy on hyperlinking. In the descriptive sense, by the standard of a user or developer, Beaker is a browser.
So long as Beaker can be used as a traditional HTTP browser, Web APIs should remain intact in context of HTTP pages and scripts. The same logic ought to apply to Hyperdrive websites, for the sake of consistency when writing cross-protocol pages and scripts.
Similarly, Beaker APIs should follow browser convention if they are to be directly accessible from web pages. An import from a vendor URI restricts cross-protocol usage, and cannot be directly tested the same way that a global variable can. E.g.:
// No way to test that the runtime supports the `beaker:` URI
import { hyperdrive } from "beaker://api/hyperdrive.js";
// Global variables can be easily tested
if (window.beaker) {
// Beaker APIs are accessible
} else {
// Fallback
}
As a side note, I think the camel case beaker namespace is a misstep. By convention, global namespaces should use Pascal case, whether constructors or not. I should think Beaker would be more appropriate, if such a namespace is even required. Global Hyperdrive, Peersockets, etc. would be fine as well, assuming you are not overly concerned about global namespace pollution and a slightly higher chance of future collision.
If the intention is to move away from traditional browsing entirely, such that Beaker could not be descriptively called a "web browser", so be it. But please be wary of developer expectations and canonical terminology. By using terms like "web", "navigate", "sites", "pages", etc. in proximity, coupled with usage of the full HTML-JavaScript-CSS framework, Beaker implicitly promises to be, on some level, a browser.
In my opinion, Beaker either needs to undergo a radical paradigm shift and start using web technologies in a very different context, or commit to being a web browser.
@metanomial Some of your observations about the import issue aside (fair point, we'll need to discuss) this is ultimately a question about whether a browser is also an applications platform. I believe it already is, and there have been a number of attempts to turn the browser into a stand alone apps platform before (Chrome Apps, extensions). Our intention is not to move away from browsing but to recognize that we need to expand on the applications model of the browser to make p2p apps work.
In a practical sense, this manifests as questions like, "should there be a way for apps to capture screenshots of pages?" Traditionally you'd say no because- why adopt the security risk (even with perms) when a backend could do that anyway? But we don't have that backend available, so if the use-case is common and valid, we should say yes.
The assumption of existing browsers -- that pages are just the frontend to applications -- doesn't hold in Beaker. We shouldn't adopt that assumption in our development if it's not right for this model.
I remember your post about the browser as an OS, I think it's the philosophy Beaker should keep. Actually I would go literrally for that and make it a full OS, you already have the file system, you could build everything from there a Beaker package manager, you also have the shell and so on.
Imagine running beaker install <hash>
Idk, I'm just saying you should go big.
Well, I think the central presence of the editor, file browser, terminal and forking/merging interfaces from the main UI have moved Beaker beyond just a "browser" - even if I think calling it a "browser" still makes perfect sense. I realize this discussion is about APIs - but like you say - APIs have to be aligned with the strategy of the platform.
You have added some APIs for customizing the terminal. Imagine if you also had APIs for integrating with the editor, file browser, perhaps even the forking/merging UI. Editing has become a big deal on the Web - and yet every site has a different editor. Imagine if Beaker's editor became an important part of editing throughout the network - you go there to post tweets, to write blog posts, edit an e-mail, to leave comments. (And sites could perhaps offer virtual files to edit - in addition to the access to the underlying hyperdrive.)
To me, this would prevent Beaker from becoming merely a shell for other applications - even if that might be ultimately unavoidable. I don't think that you have to just repeat the Web. In fact, I think you're more likely to succeed if you don't. (I personally think people are more likely to switch networks to take advantage of novelties - think of how Instagram's filters were originally the draw, not the network itself.)
I'm not just talking about the editor, of course. I'm suggesting that having a rich set of tools that interacts with a website does a few things:
- Editing and creating with a website becomes as automatic as reading it.
- The tools become familiar across the network - and we've seen the benefit of this with, for example, the standardization of mobile gestures. There is pressure to keep that familiarity intact.
- Developers save time when they don't have to roll their own tools.
- The more that people interact with the underlying documents/code, the more pressure there is to have open code. (Very similar kind of pressure explains why so many companies use Github - they desire the free debugging and contributions.)
I'm not exactly sure the best strategy to take along these lines, but also imagine having an editor that can eventually handle image editing, music creation, video splicing - perhaps by integrating with tools outside the browser. (One of my criticisms of desktop browsers is that they do a poor job integrating with other software - whereas mobile browsers are very good at sharing to text message or moving a file over to another app.)
I'm not saying that Beaker OS would dash my hopes or anything - I really trust your instincts. As a user of 1.0, I would be remiss if I didn't pipe up about this, tho. I think Beaker is poised to show browsers how little imagination they've had recently.
+1 @YerkoPalma
@kickscondor I love where your head's at. In the sense that an OS makes tools work with each other, I think that's 100% the right way to be thinking about it. I'm going to chew on all of those ideas.
I think from a developers perspective, go big. Any complex application needs to be completely redesigned from the ground up to work on the peer to peer web anyways. I think even adding new functionality is better than trying to apply a metaphor that doesn't quite work onto P2P. Also, we all know that many of the modern browser's features developed along with technology of the day, and are massive compromises for the sake of backwards compatibility.
The one caveat is that trying to keep the "normal user" experience as browser-like as possible might be a good idea. I know we area talking about developer experience mostly, but it might be an idea to consider that normal users don't want to have to adapt as quickly - they want it to just work like something they already know how to use. I don't think that means you can't make big changes, just that we want developers to be able to place a "browser-like" face on any apps they build. Let users explore the extra features at their leisure, and be forced to learn as few new things as possible upfront. Beaker does a good job of this now, but just something to keep in mind as we move forward.
That Beaker is and will increasingly be an applications platform seems like a foregone conclusion.
Better question: "Is Beaker's goal to extend the HTTP web or to build a new Hypercore web?"
If the goal is to extend the HTTP web, it would be crazy to move away from Web APIs and browser conventions and do its own thing where standalone applications are concerned. Chrome Apps, hosted and packaged, are dead. Going forward, PWAs are going to be the cross platform solution. Browsers are already application platforms, and are on course to become more comprehensively so over the next few years.
Instead of trying to exist in parallel to Web APIs designed for HTTP, expand upon the Web APIs. If the goal is to extend the HTTP web, the Hypercore Protocol team ought to be drafting and releasing extended specs that define Web API behavior for Hypercore sites and PWAs.
But, if the goal is to create a completely new and independant Hypercore web, I agree with @YerkoPalma and @matthewauld. Go big! Why carry around all of the historical baggage of modern browsers? Totally overhaul the system. You won't need to worry about backwards compat or conventions.
It would be tragic if in 10 years peer-to-peer and distributed tech has taken off; and Beaker is just the Internet Explorer of the peer-to-peer browsing world, unable to ever settle down on committing to standards versus doing its own thing.
Involvement in standards should be a goal IMO. I think it would help to see Beaker as part of an ecosystem of new browser development work, along with other projects like Brave and many others possibly yet to be announced.
What we think of as a browser today is naturally becoming more of an applications platform over time so if Beaker could see itself as part of that process as opposed to separate I think that would be a useful model since that would allow Beaker to position itself and its technology in a way that accepts that it is moving ahead and is implementing new APIs and approaches, but still with an eye to be part of a standards ecosystem alongside other browser-like platforms. Taking the standards changes to align where necessary too.
For the CORS differences, it seems clear the security models of hyper requests are very different from HTTP and as such it makes sense that these properties change, so far as any browser environment implementing hyper-like protocols would likely undergo a similar view here.
Builtin modules are part of very current standards discussions for browsers at the moment, although are a little stalled right now by a lack of a clear path to align these standards between browsers. Taking a lead here could be a strategic decision that in turn could provide a fruitful example for standards work to take inspiration from. Even if it changes later, such a break doesn't seem too high risk.
Allowing universal patterns would still be good though where appropriate, but certainly Beaker-specific library code should exist too. For example with export maps or top level await it can be possible to conditionally load these modules only when in beaker environments while still having universal code, even supporting eg alignment to some builtin protocol format as a future break via fallbacks in exports / feature detections.
I have recently gotten interest in Beaker (and P2P technologies), so apologies if I get something conceptually wrong.
As a Web enthusiast and hobbyist, I think it feels important for me that Beaker avoid doing too much of “its own thing”, as that has shown to be really ineffective at attracting adopters.
[ASIDE] One thing I find unfortunate is the choice for Beaker-branded APIs. I think APIs should be agnostic of browser branding, so that other browsers can be created supporting the same technologies and not have to “imitate” existing browsers in brand name for APIs.
I think it’s important for APIs to incorporate well with the existing Web environment. For example, instead of coming up with its own “proprietary” beaker.foo APIs, I think it should try to leverage existing well-understood Web API concepts to achieve similar things.
For example, the beaker.hyperdrive API could be incorporated into the existing fetch APIs. Admittedly, fetch embraces many concepts from HTTP, but at the same time, it tries to be agnostic of it (so that browsers can interact with e.g. FTP).
- Instead of
await beaker.hyperdrive.readFile("hyper://...")andawait beaker.hyperdrive.readdir("hyper://..."),await fetch("hyper://...")could be used (directories would be encoded as JSON arrays). - Instead of
await beaker.hyperdrive.stat("hyper://..."),await fetch("hyper://...", {method: "HEAD"})could be used (stat information would be available as response headers; links could be analyzed through a request header). - Modifying files could be done through
"PUT","POST","DELETE", etc.
Note that the fetch API stays agnostic of what a “method” is supposed to mean for anything non-HTTP, leaving it up to the underlying protocol. “Methods” are treated as arbitrary, mostly opaque strings by the spec, so Beaker could be able to use its own custom methods for more exoctic operations (“exotic” compared to the HTTP Web, I mean).
I think it’d be neat of hyper: URLs could respect the URL spec, at least to an extent, so that people could use new URL("foo/bar", "hyper://...").
There's been a lot of energy around a fetch() API lately for hyper:// so that may happen in the nearish future. Shout out to @rangermauve who's been carrying the banner on that.
I honestly wouldn't worry too much about the beaker API space in the long term. I'd think of them like vendor prefixes. Either Beaker will work its way "into the fold" of the Web, leading those APIs to get standardized/replaced, or Beaker will always be something separate, in which case the prefix makes sense. Ideally the former will happen.
I know vendor prefixes are not well-loved because they require breaking changes to move away from, but existing browsers have a much more coordinated process with each other to avoid them, and we just don't have that available to us. It's not ideal but it's not a deadly issue.
On the plus side, the more people are displeased with our beaker prefix, the more political will there is for those APIs to become replaced with Web standards with no prefix!
One other side note about the original post I wrote here --
Until Beaker has, you know, been realized, almost everything I write is a part of an exploration process. This question of whether Beaker is a browser or something else (a platform? an OS?) is reflective of us trying to understand how to execute on the ideas of the dweb and then explain our execution to people.
At this particular moment, I'm leaning towards Beaker being a weird browser.
For the record, I'm currently iterating on a fetch API here: https://github.com/RangerMauve/dat-fetch
Currently integrated it in Agregore, and I'm planning on getting it into an http gateway and at some point getting the same methods into Gateway Browser and maybe Beaker. 😁
I think that beakers APIs are pretty cool and do stuff that couldn't be easily encapsulated in a fetch interface. Especially all the social stuff. :O it'll be cool if all this stuff gets mainstream adoption down the line.
I honestly wouldn't worry too much about the
beakerAPI space in the long term. I'd think of them like vendor prefixes. Either Beaker will work its way "into the fold" of the Web, leading those APIs to get standardized/replaced, or Beaker will always be something separate, in which case the prefix makes sense. Ideally the former will happen.
I think the problem I have with this is somewhat twofold.
- It seems to me like it means it becomes difficult for people to adopt P2P Web technologies, because they would need to modify their websites to adjust for the newest browser concepts and APIs.
- If Beaker does become popular (in a broader sense), it might become difficult for it to remove or replace these namespaces.
As much as I think experimenting is a really good thing, I feel like one of the problems with the distributed Web community is the lack of some kind of standardization, making it difficult for people to adopt it.
It seems to me like you can’t design (part of) a platform intending for it to be replaced if you want for it to be a tool towards helping popularize its paradigm (the “dweb”, as you say).
I think that beakers APIs are pretty cool and do stuff that couldn't be easily encapsulated in a fetch interface.
Yes, of course! I used fetch as an example. What I meant is that I think avoiding introducing new concepts (at least when possible) is a good thing.
Until Beaker has, you know, been realized, almost everything I write is a part of an exploration process
I feel like I’m looking forward to see how this kind of exploration turns out, but I think it shouldn’t try to be too distinct if it hopes to be considered as an option by more people.
It seems to me like a lot of websites would work well without many modifications in a distributed environment, which might make it attractive to people. If someone has to learn all of these new concepts and APIs that don’t work on regular browsers, they might be discouraged from endorsing it.
not really sure if this is the right thread to add my thoughts, but after playing with Beaker for a while as browser/microblog platform one thing I'd love to see a (longer term) plan for is integrating with the 'regular' web ... if I wanted to abandon Facebook but still have my 'regular' friends be able to see/subscribe to notifications a hyper:// URI to my microblog isn't going to help, but a proxy solution via (eg) a regular domain name and a proxy (say in partnership with someone like Cloudflare who could certainly scale and are also doing interesting things with edge compute) would create a 'best of both worlds'
