Interactive Notifications API

[EternityForest]

Introduction

Describe the challenge or problem on the web you are proposing we address.

Smartwatches and other small devices are unsuited for displaying current websites. Rendering HTML on ESP32-class devices is not well supported.

There is also poor support for cross-platform desktop widgets.

I'm proposing a standard for very small amounts of embeddable web content, which can run either locally, or on a constrained device.

Use Cases (Recommended)

Clearly define one (or, ideally, more) ways in which this challenge or problem is blocking real-world applications. Please be as specific as you can.

• Push interactive web content to smartwatches, shelf labels, and other constrained devices
• Create local system notifications that allow for instant access to features and information, such as a TV remote or weather widget
• Enable web pages to interact with embedded displays on game controllers, presentation remotes, and similar.

Goals (Optional)

A list of what this proposal must do in order to be considered successful.

• Allow a website to serve a persistent notification or widget object with basic capabilities like buttons, menu navigation trees, text inputs, icons, and number ranges

• Allow for discovery and selection of devices on which to render the widget, including connected devices, system notifications, and making the it available as a desktop applet

• Allow the server to push new content to the applet, and the applet to communicate back to the server

• Allow a service worker to persist on the device for as long as it's widget or notification does.

Non-goals (Optional)

A list of aspects of this challenge/problem or related challenges/problems that this proposal does not need to address.

• In depth control of aesthetics and styling. The web applications are meant to be agnostic to any details about the display device, which could be anything from a 640px widget to a tiny monochrome OLED.

• Scripting on the small devices themselves

Proposed Solution

Without being too prescriptive, articulate a specific way (procedurally, algorithmically, declaratively) that a new or current web technology solves an existing problem or challenge.

A format is created that can describe the complete menu tree and widgets of a small applet, not containing any HTML at all, but a small set of basic widgets suitable for smartwatches and the like.

This file can specify a web socket URL on which to send user input, and receive new values for widgets. The system may disconnect this socket and try again later if it detects excessive activity.

The file can also describe hyperlinks(To other UI descriptions, or to real HTML), and has enough capability to represent rich text(About equivalent to Markdown).

A JavaScript API is created which is used to tell the browser about a URL containing such a widget. Two versions are available, one that simply informs the browser which displays its own UI, and one that can return a list of connected devices and try to send things to a specific one.

The JS API requires a secure context as usual, but the JSON itself does not, unless pointed to by a secure page, because we want to be able to use these on the local network

An MDNS based protocol is created to enable this discovery.

Various transports are possible between the host page in the browser and the device that physically displays the content.

A new BLE protocol can be created to install a widget on a Bluetooth device like an e-reader, and allow the device to proxy HTTP and websocket data through the host, to the device.

Examples (Recommended)

Share one or more examples of how your proposal would be used by developers.

A smart TV would like to provide flexible ways to control it. It acts as a local web server which can serve widgets. Your phone and smart remote both discover the device and show the appropriate notification widget.

A diary app website provides serves a widget that creates an always-accessible instant text box to record your feelings. On their main page, they use a JS API to ask you if you would like to install the widget.

The browser allows routing it to your desktop, your notifications, your watch, etc, and auto starting at system start.

A game has extra information to display on a keyboard.

A news website would like to make its feed readable on an e ink reader connected via Bluetooth. It prompts the user to install it's widget somewhere, she selects her e-reader.

Now, the e-reader can at any time access the given widget URL, so long as the phone is nearby and has Bluetooth. Any time this feature is triggered from the reader, a notification will pop up on the phone, so that nobody can install things to devices and forget they are there.

As a future extension, device-specific features could be discovered and enabled, so that, for example, a connected weather station could support the "com.foocorp.weather2000" protocol, and a web site could access it by asking to store it's widget on the station, and intercepting it's data with the service worker.

Alternate Approaches (Optional)

Could this challenge be addressed another way? Including other options you considered can help others understand the limitations or adverse side effects of other approaches.

Privacy & Security Considerations

Perform a self-review. You may think of additional privacy/security considerations other than the ideas listed. If you find no considerations after careful review: "No considerable privacy or security concerns are expected, but we welcome community feedback."

• Almost all powerful web features can always be used for tracking, but I don't expect major concerns as there is nothing done without user consent, and the feature doesn't do anything without a visible notification on every single use, which persists until dismissed.

Let’s Discuss (Optional)

This is a great place to list out questions you'd like others to weigh in on or areas you need feedback on.

[AdamSobieski]

Hello @EternityForest, here are some hopefully useful hyperlinks:

Working and Community Groups

• https://www.w3.org/WoT/wg/
• https://www.w3.org/2014/secondscreen/
• https://www.w3.org/community/webscreens/
• https://www.w3.org/community/webtiming/

Recommendations and Drafts

• https://www.w3.org/TR/presentation-api/
• https://webtiming.github.io/timingobject/
• https://notifications.spec.whatwg.org/

[EternityForest]

Further interesting related links:

https://adaptivecards.io/

Microsoft's Adaptive Cards standard is an example of purely declarative, logicless content that leaves look and feel to the host application.

[AdamSobieski]

If I understand correctly, you're broaching something like using the Web of Things to send JSON-based adaptive cards to things.

Let us consider the examples of the Stock Update card or Weather (Compact) adaptive card. For both of these examples, the underlying data, stock market and weather data, vary over the course of time. It could be that as the underlying data changed, new cards would need to be transmitted to things, or that some new solution for dynamic streaming data might be considered. Brainstorming, a set of URLs could be included in adaptive cards to provide things with sources of dynamic streaming data.

Push interactive web content to smartwatches, shelf labels, and other constrained devices

A preliminary approach might involve utilizing Web of Thing descriptions, more specifically action affordances, to enable (1) requesting from a thing which well-known (URI-based) adaptive card schemas, and/or versions thereof, that it recognized, and (2) transmitting cards to it. It might also be useful, in particular if lists of recognized schemas were to be lengthy, to enable (3) querying whether it recognized a specific adaptive card schema. Noteworthy is that things could asynchronously update their firmware and then recognize new adaptive card schemas, and/or versions thereof.

Focusing on the "interactive" aspect of the use case, a second approach might involve bidirectional communication. Perhaps the Web browser on a smartphone or other computing device could provide a Web of Things description and affordances to the thing intended to receive adaptive cards. Then an end-user could push a button or click a hyperlink in an interactive card and this would result in notifying the Web application on the smartphone or other computing device of the button press or hyperlink click. Another approach to this might involve Web of Thing event affordances.

With respect to smartphone and smartwatch interoperability scenarios, I also found the Web Bluetooth Draft Community Group Report. It should be possible to map these abstract ideas, points (1), (2), and (3) above, to a Web Bluetooth approach.