Extension (plugin) support

[The-Compiler]

There should be support for writing plugins which are more complex than the simple (non-js) userscripts. Python plugins are the way to go.

FAQ

What's the current state of this issue?

~~September 2018: I'm working on this as a student research project. The main focus is designing a clean plugin API which can then be used to move internal qutebrowser parts to "plugins". As soon as this work is done (late 2018), the next step is gradually opening up the plugin API to third-party plugins.~~

~~March 2019: The research project went quite well (paper), but also was quite academic - not that much time was actually spent on implementation. As a result, there's now an initial extension API used internally in qutebrowser, but it's far from finished and not open to third-party extensions yet.~~

~~I likely won't have much time to continue working on extensions until my bachelor thesis is finished in mid-June (see the announcement mail for details), but after that I plan to launch another crowdfunding focused on long-term donations (Patreon/OpenCollective). This should allow me to work 2-3 days per week on qutebrowser for a longer time, primarly focused on finishing extensions.~~

~~September 2019: See this announcement mail for a longer recap of how the research project went. I finished my Bachelor thesis and had my final exams in late August. This means my studies are now finished and I finally have more time for qutebrowser on my hands again. Like outlined in that announcement mail, I'm now employed around 16h/week, while I plan to spend the rest of the time on qutebrowser and some freelancing work. At the moment, I'm planning the next crowdfunding campaign (also see the related survey) and getting things set up. After things are rolling, there are some things which are currently more urgent than plugin support: Making sure things continue to work with the latest Qt 5.13/5.14 versions and taking care of a lot of contributions. Once that backlog is down and some other work I started is finished, I finally plan to get back to extension support again, this time with enough resources to get things to a working state!~~

October 2019: See the recent roadmap mail for more details on what's going on in qutebrowser development - this definitely isn't forgotten, there are just some more important things to look at first. I also launched the next crowdfunding which will allow me to work on qutebrowser part-time (without having to write a research paper this time :laughing:) and hopefully accelerate things a lot.

How can I get updates?

Subscribe to this issue. You might also want to subscribe to the qutebrowser-plugins repository and the qutebrowser-hsr-sa repository where the related university documentation will land (also check the releases if you're curious).

What about JavaScript plugins / WebExtensions?

Support for WebExtensions is currently out-of-scope for qutebrowser. It's hard to even evaluate to which extend support for them would be possible, but if possible, it's going to be a lot of work. Partial support for often used plugin APIs (i.e. some WebExtensions) might happen some day, but currently it's too early to say if/when. The main focus is on a Python plugin API right now. Related QtWebEngine issues which might make this easier: QTBUG-51185, QTBUG-61676.

Also, a Python plugin API will allow for more freedom, since we treat plugins as trusted code.

Why don't you just allow people to inject Python code into qutebrowser?

Firefox had the issue of having a plugin API which tightly integrates with its internals, and had to drop it because it made further Firefox development harder, and bigger changes impossible. Also, every Firefox update broke a lot of plugins. This is not what I want things to end up like with qutebrowser.

In theory, you can do some things via a config.py file, like done in the qutenyan project. However, this is entirely unsupported and discouraged.

Do you call them plugins or extensions?

They used to be called plugins - however, since that term is often used to refer to NPAPI/PPAPI plugins (like Flash) in the context of web browsers, they were later renamed to extensions.

Plugin ideas

• uMatrix: #28
• More sophisticated adblocking: #29 (host-blacklist based adblocking is implemented in qutebrowser's core)
• HTTPS everywhere: #335
• Ghostery/Disconnect
• Certificate Patrol
• (random) user agent switcher
• Mouse gestures (can be done via easystroke to some degree)
• Remove google redirects
• Integration with LanguageTool
• Filling forms (e.g. password management with KeePassX/gnome keyring/passwordstore/... - as in #180, also form filling as in #695)
• HoverZoom
• speed dial
• periodic auto-reload
• detailed page loading bar (like otter/opera)
• automatically spawn mpv on video pages
• Find video URLs on a page, like Media Sniffer or Video DownloadHelper
• copy URLs of all pages or write them to a file
• DownThemAll
• Google Cloud Print (devguide)
• Rewriting URLs to their .onion equivalents based on a list
• Chromecast integration? (#1319)
• autoscroll
• decentraleyes
• URL rewriting like PureURL for Firefox (removing things like utm_foo)
• Open certain links (like youtube) in a handler application automatically instead of following it
• Adding a "real" address bar
• Adding a bookmarks bar
• Yanking a short URL (e.g. youtube has a shortlink ref for youtu.be links)
• Restarting failed downloads after a timeout (with bad internet connection)
• alternative stylesheet selector
• A way to show md5/sha/etc. for downloads or even enter the correct one.
• Plasma Shell browser integration (#3996)
• Exposing the plugin API as an RPC interface usable from other languages
• Integrating Wallabag or other bookmark managers
• Back/Forward by domain instead of page
• Controlling Spotify/Youtube via MPRIS: #5023
• Middle-click scrolling (like on Windows), also see #7215
• Noiszy (Background tab creating "browsing noise")
• also see existing dwb userscripts
  • Arch wiki
  • tlvince/dwb-userscripts
  • dwb-extensions on GoogleCode
  • built-in extensions
  • dwb-extensions repo
• Hooking into notifications
• Showing the current mode in something like i3bar
• Prompting for redirects (?), see #4534
• TabFS
• Mouse navigation similar to keynav, see #6169

Abilities that plugins should have

• Interact with the WebElements on the page (e.g. make non-clickable links clickable) - or maybe only do this via JS, e.g. Linkify.
• Add widgets to the statusbar. (e.g. RSS or search engine symbol)
• Register new commands (with capital letter, like vim)
• Register new settings (in a special [plugin.foo] section probably)
• Subscribe to Qt signals (QWebPage/QWebView)
• Add new completions, e.g. with an external bookmark manager like cym13/bookmark
• Create widget-tabs when #724 is done, so someone could theoretically write an RSS reader or mail client or whatever
• Adjust page content, like Detox (and many others)
• Maybe make it possible to interact with the command input so that adding www. and .com to an URL like in #1749 would be possible.
• Maybe bind keys with prefixes, e.g. g# with any number
• Use acceptNavigationRequest, e.g. to automatically open mpv if a YouTube link is opened.
• Custom URL schemes
  • IPFS (#1314)
  • Gopher (#321)
  • dat
  • ~~FTP for QtWebKit (#235)~~
• Custom completions (see #3376)
• There might be some more ideas in https://omar.website/tabfs/, as a power user focused interface to the web
• provide custom hinting logic (some ideas in https://github.com/qutebrowser/qutebrowser/issues/674)
• Cloudflare Privacy Pass (#7945)
• Provide custom format strings for commands and tab/window titles, and replace/modify existing format strings (eg a possible solution to https://github.com/qutebrowser/qutebrowser/issues/7685)

Libraries

• Yapsy
• straight.plugin
• stevedore
• pluggy
• PluginBase
• A simple plugin framework
• Plug n' play
• anki.hooks
• pike

Resources

• Fundamental concepts of plugin infrastructures
• The little manual of API design
• Python API checklist
• http://sedimental.org/plugin_systems.html
• Initial WebExtension support in Epiphany

Javascript plugins

• Javascript: QJSEngine
• PyV8 would probably work as well for javascript extensions (and maybe be easier/more pythonic), but it's another dependency and a different javascript engine.

It's probably not possible to directly use QtWebEngine's JavaScript engine, as that's tightly connected to QWebEnginePage actually showing a website.

Possible API inspirations

• WebExtensions API in Chrome/Opera/(soon) Firefox
  • Mozilla Wiki
  • Chromium docs
  • Opera docs
• splinter driver API
• dwb's javascript API
• Spyder IDE:
  • https://github.com/spyder-ide/spyder/blob/master/spyder/api/plugins.py
  • https://github.com/spyder-ide/spyder/blob/master/spyder/otherplugins.py
  • https://github.com/spyder-ide/spyder/tree/master/spyder/plugins
  • https://github.com/spyder-ide/spyder/wiki/User-plugins
  • https://github.com/spyder-ide/spyder/wiki/Writing-Plugins
  • https://github.com/spyder-ide/spyder-plugin-cookiecutter

Other interesting links

• ghacks-user.js and its wiki for privacy related features.
• Another user.js, dito.

First ideas

Separate namespace for plugins

There should be a new qutebrowser.pluginapi namespace. Everything outside of that namespace is to be considered to be "private" for plugins, and will likely break at some point.

Only some stuff (like possibly utils.urlutils) will be exposed directly - everything else will be exposed through a wrapper, for various reasons:

• I can change stuff without breaking plugins, if I update the wrappers accordingly.
• The wrappers will be easier to expose to a javascript plugin API
• They will be less complex than the qutebrowser internals, so it should be easier to get started

If something is missing from the wrappers, it should be trivial for a plugin author to open a feature request to have it added - this should be encouraged in the docs.

Qt signal based workflow

I don't think using something like pluggy will be neccessary (especially because pluggy is still expected to undergo changes, and is not packaged).

Instead of that, there's a simple function (say, plugin_init) which gets called when a plugin is loaded. The plugin gets a PluginContext object with some version information and a signals attribute.

The signals object then has many signals (perhaps also in sub-objects), like started, cmd.entered, etc. etc. Such a signals object also exists per-tab and per-window. The plugin will get the window-specific object with a global new_window signal (and something similar for new tabs).

Those events/signals are the main entry point for plugins. There will also be APIs for certain tasks (like adding an icon to the statusbar), but the majority of tasks should be possible using those signals.

For most actions, an API to call commands nicely (i.e. as functions rather than as strings) would probably already suffice.

Exposing plugin signals to userscripts

This is related to #902 - the signals mentioned above should be written to userscript FIFOs so those can react on the events - similar to herbstluftwm's hooks and herbstclient --idle.

This could possibly be implemented as a plugin by itself, which binds to all available signals - some layer to transform the signal arguments to strings will be needed either way.

A line written to the FIFO could look like this:

category sender signalname arg1 arg2 ...

• category: For extensibility - currently always signal
• sender: A dotted path of the sender of the signal, i.e. which signal object from above emitted the signal:
  • global: A global signal, like started
  • win0: A signal from the window with win ID 0
  • win0.tab1: A signal from the tab 1 in window 0
• signalname: A dotted path of the signal, e.g. started or cmd.entered, etc.
• arg1, arg2: Arguments which were passed to the signal, which need to be converted to a string in some way. Either there's a plugin which handles this argument for argument, or there's type-checking:
  • str: passed through
  • int, float: str() called on them
  • QUrl: url.toString() called on them
  • window objects -> win0
  • tab objects -> win0.tab1

Resources from BPL

resources

• http://www.se-radio.net/2008/07/episode-104-plugin-architectures/
• https://github.com/The-Compiler/qutebrowser/issues/30
• http://www4.in.tum.de/~blanchet/api-design.pdf
• http://eli.thegreenplace.net/2012/08/07/fundamental-concepts-of-plugin-infrastructures/
• http://martyalchin.com/2008/jan/10/simple-plugin-framework/
• https://www.sublimetext.com/docs/3/api_reference.html
• https://ep2015.europython.eu/conference/talks/the-hook-based-plugin-architecture-of-pytest
• https://www.sk89q.com/2010/03/a-first-class-extensible-python-plugin-framework/
• https://trac.edgewall.org/wiki/TracDev
• http://docs.openstack.org/developer/stevedore
• http://aroberge.blogspot.com.es/2008/12/plugins-part-1-application.html
• https://pymotw.com/2/abc/
• http://virtualplants.github.io/latest/dev/peps/pep_plugins.html

libraries & apps

• https://pypi.python.org/pypi/extensions
• https://www.riverbankcomputing.com/software/dip/download
• https://github.com/kovidgoyal/calibre
• http://yapsy.sourceforge.net/
• http://pluginbase.pocoo.org/
• https://github.com/benhoff/pluginmanager
• https://github.com/daltonmatos/plugnplay#readme
• https://github.com/ccoughlin/NDIToolbox
• https://github.com/dae/anki/
• https://github.com/ironfroggy/straight.plugin
• https://pypi.python.org/pypi/pluggy
• https://github.com/edgewall/trac
• https://github.com/HackEdit
• http://docs.enthought.com/envisage
• https://github.com/openstack/stevedore
• https://picard-docs.musicbrainz.org/en/appendices/plugins_api.html
• https://github.com/PyUtilib/pyutilib/ <----

[noctuid]

automatically spawn mpv on video pages

Just to contribute to this idea for a possible future plugin, it would be awesome to be able to bind a key to manually invoke mpv (or another player with :spawn) on the direct link (not the {url}) of a playing video. With pentadactyl I do this with the firefox media sniffer plugin currently.

[cwmke]

Just thought I would throw a couple possibilities/alternatives into the mix for long term feature ideas. Ublock Origin, Umatrix, Random-Agent-Switcher. Also, pass support would be amazing. http://www.passwordstore.org/ Thanks for this incredible project. I'm really excited for its future.

[lamarpavel]

@cwmke The integration of password managers has come up in several suggestions before and it is unlikely that one specific password manager will be integrated into qutebrowser, rather there might be an abstract interface that can be used with many different password managers.

Until then, there are user scripts for this purpose, for example this one.

[The-Compiler]

I added some thoughts to "First ideas" above - any feedback would be much appreciated!

[traverseda]

Has any progress been made in this direction?

[The-Compiler]

No, not yet. Tests, the QtWebEngine backend, config revolution and per-domain settings are the current main focus. After that I'll look at this, but it's probably going to take a few months.

[traverseda]

Seems like reasonable priorities. Just though I'd drop in and remind you that this issues exists.

A lot of projects, like django, allow you to include apps that are basically just python code. You can get a long way with monkey patching, and just providing some entry points.

Is there a reason you don't provide a quick and hacky solution? Load and run python scripts from a directory or something?

[The-Compiler]

Yes, because I don't like quick and hacky solutions :wink:

I don't want third party plugins which break with every release, there's already Firefox for that :wink: - so I'd like to do things right.

[traverseda]

Fair enough

[btall]

:+1: Ghostery/Disconnect and Remove google redirects

[RomanSC]

• Default adblock needs have ability to add third party block lists such as uBlocks, otherwise it's not adequite enough to cross out adblock: https://github.com/gorhill/uBlock/tree/master/assets

[The-Compiler]

@RomanSC: I think you mean the ability to use adblock-style lists rather than just hosts, because all of the default block lists are already third party, and you can add your own :wink:

See #29 for that, though it's not a priority for me currently, as the simple host-blocker we have now works well enough for 95% of the cases.

[deniss-s]

I would add HTTPS Everywhere to the list of useful plugins.

[haasn]

Allow me to add FireGestures to the list of my really-want-to-have-this plugins (mostly due to muscle memory, and because I often want to alternate between using the mouse and the keyboard depending on what is more convenient at the time) :)

Edit: The third party program easystroke completely replicates all of FireGestures' functionality for me. If anybody else is missing mouse gestures, I can highly recommend it!

Edit 2: Seems like easystroke is a bit buggy overall, and can often crash X. Instead, I've adjusted to having my tab bar on the side and using the mouse wheel on the tab bar area.

[moon-chilled]

Here's what I thought a simple plugin might look like, as inspiration for when/if plugins are actually implemented. Aside from that, here are some methods, not really organized, that might should be in here:

• currpage.add_css_rule(tag, ruleslist) # str, dict
• currpage.del_css_rule(tag, key, value) # useless, maybe, because we can currpage.add_css_rule(tag, {myrule: default})
• currpage.move(direction) # (left|right)
• currpage.forms[0].fill(content) # str
• currpage is an instance of Page(), and pages is a list of those?
• settings.store/fetch(value) # stored in DB? json? App can create its own files? All values that can be stored must be declared, and maxsize declared at beginning of plug, or in manifest.json type thing? Maybe not manifest.json, but have the user confirm (e.g. "this plug may take up to X MB of space")?
• qb_command(command)

[The-Compiler]

So I disagree with some of that, and agree with some other points of that - but we're really still at the brainstorming state here :wink:

Here is how I'd perhaps imagine a plugin reloading a given site periodically:

"""Module docstring automatically used as plugin description."""

from qutebrowser import pluginapi

@pluginapi.hook('new-tab')
def handle_new_tab(tab):
    if tab.url() == QUrl('http://example.com'):
        pluginapi.utils.setup_timer(60 * 1000, tab.refresh)

The hooks will then be named just like the same hooks for other purposes (like autocmds).

[haasn]

You could borrow a page from the mpv lua API, which I think works quite well for many purposes.

mpv scripts can basically be invoked in three different ways:

1. Through a keybinding or script command they add at load time (which calls a custom function)
2. Through callback from a defined event (similar to the hooks)
3. Triggered by changes to an observed variable
4. (Less used) Through a timer callback, which they can either set at load time or perhaps manually toggle on a keybinding

The 3. could be merged into 2. in some sense, though, but I still think the ability to explicitly watch named variables (mpv calls them “properties”) can be useful for some purposes. For example, I could imagine a simple history beancounter (assuming we didn't have one already) work by watching for changes to the URL property.

In summary, in addition to thinking what events a plugin could hook, it might also be good to think about what properties a plugin could read/watch.

[cwmke]

The WebExtensions route seems like a good move to me since most of the requested extensions already exist for other browsers. This would open up compatibility with a large group of existing extensions. Firefox is moving their extensions system towards this as well.

[The-Compiler]

@cwmke I agree WebExtensions support would be awesome, but I don't see it happen anytime soon unless someone finds out how to easily implement it in QtWebEngine. In either case, I definitely want a Python extension API as well.

[haasn]

The problem with WebExtensions is that:

1. The API is very limited from what I understand
2. Directly interfacing with qutebrowser elements would be hard from a WebExtensions addon

So in the interest of providing the ability for plugins to, say, define their own commands and do whacky things, a python API would be mandatory (IMO).

But WebExtensions might still be useful for something like μBlock₀ which essentially only interfaces with the underlying browser component - and not so much the python interface.

[The-Compiler]

Yeah, I think the main motivation for WebExtension support is that you could run some (all?) existing Chromium plugin - "native" qutebrowser plugins definitely will be in Python.

[cwmke]

@The-Compiler is correct in terms of what I am thinking. We would be able to run most (probably not all) Chromium and shortly, Firefox extensions.

My concern is that by focusing only on Python plugins qutebrowser will be very limited in plugin choices since there will unquestionably be a smaller developer pool to do the creating compared with Chromium and Firefox. This is not a nock on the developers or Python plugins at all since native browser plugins tend to be great. :-)

What creates this concern are similar browser projects like Epiphany, Midori, and Konqueror/Rekonq that have limited plugin support.

Either way I will continue to be a very happy and satisfied qutebrowser user. These are just the remainder of my personal thoughts on the matter.

[behrmann]

Having looked around a bit, I agree that some JS-API for plugins would be grand. Unfortunately for the Chrome extension API there seems to be movement only a subset in QtWebEngine for now.

I would like to see uBlock working in qutebrowser. It looks rather cleanly implemented to me and separates general and platform specific code. I guess it would be a much better solution for adblocking and related wishes than any homegrown solution and it would solve issue #29 in a single stroke. If there would be a JS-API for plugins the the platform specific code for qutebrowser might even be upstreamed, which would be good PR for qutebrowser, too.

[Xananax]

I'm coming late to the party, but I just want to add my +1 to the idea of having js plugins instead of python plugins. I'm not gonna repeat the reasons why, we all know the pros. However I'm not sure I understand the cons properly. ~~Whether python or JS, an interface of limited surface will need to be provided to secure plugins in a sandbox, so it's not like implementing python will be straighforward.~~ edit: I misread the op, actually python is going to be much more straightforward.
All I've read is that it is "harder" and might not be "cleaner", and that makes it hard to comment constructively, as I am unable to assess how hard vs the benefits. Can anyone point out some reading material so I can document myself?

[traverseda]

I would much prefer to write plugins in python. I'm not terribly interested in this project beyond vimperator+firefox outside of that requirement.

Of course greasemonkey style userscripts are valuable too.

[The-Compiler]

After talking to @Xananax in IRC, the main misunderstanding was that he thought I wanted to sandbox/restrict plugins in some way, which is indeed difficult/impossible with Python.

That's not the goal however - I view plugin code as trusted code with full privileges, either vetted by qutebrowser devs or by users.

I definitely want a Python API too (it was one of many reasons to write a new thing instead of contributing to an existing vim-like browser!). Nowadays I think a JS API only makes sense for some degree of WebExtensions compatibility (Chrome extensions), and that's definitely more in the future than a Python API is.

[triclops200]

I would also like to indicate support of python plugins.

[s1lvester]

Okay, so I'm going off on a little rant here: Clearly this project has a) huge potential and b) a lot of interest form the community (hence the number of Issues, PRs, Stars and so on). We need a Plugin API so the community can scratch their individual itches and the core devs can focus on big picture stuff. @The-Compiler Please make this a priority and give us an API.

[The-Compiler]

@s1lvester I don't disagree about needing this, but qutebrowser is a lot of work, so it'll take some time until I get to bigger issues which are quite a bit of work to get right, like this one. It's definitely on my radar (and hey, even I'd like it sooner rather than later!), but currently the last few QtWebEngine issues and the new config system (with per-domain settings) are still more important. Are you aware of userscripts?

[moon-chilled]

I'm pretty sure he wants to finish perfecting webengine first, so that plugins will be interfacing with the final product, so the API can be more stable.