companion feat: surface instances/modules

This continues #3694 #3695

I am not keen on the name 'instances' (largely because until recently I was working on removing the term from the codebase). So I am very open to other names for these if there are any better options.

The aim is to move all surface implementations into this system.
I am calling them 'surface instances', because each 'instance' will handle multiple surfaces, there isnt a 1:1 relationship there.

In the modules page, it now labels modules as being a connection or a surface (I am not 100% happy with this, input is welcome):

Then under the surfaces tab-group, is a new 'instances' tab:

This is modelled on the connections page, and shares various chunks of code.

The config for the instances is exactly the same as for connections (the ability to specify config fields is not currently supported):

~~And the actual running of these instances is not implemented, so they crash immediately.~~
The operation of these modules is starting to work. There is a streamdeck prototype that is mostly functional for usb surfaces, ~~but it does feel slower.~~ I was being stupid and forgot to enable the faster jpeg library

For DX/UX of the surface modules, they use a slightly different manifest format (for now, that may change), and can be imported through the same ui, and pulled from the store in the same way.
For dev, they can be loaded from the same dev folder as connection modules.

There is a rough sketch of how the api for the modules might look, but I realised that doing that without being able to run the modules in the correct threading model meant throwing the implementation away and starting again.
With this ui and thread management now in place, the module api can be prototyped and implemented exactly how it will work once finalised.

While doing this, I have taken care to prepare for more types of modules, so that if/when we want a third type, it will be much simpler to implement the few specific pieces instead.

Remaining steps:

Setup the developer portal for these surface modules :heavy_check_mark:
Port many of the current surfaces to modules, to prove that they work :heavy_check_mark:
Implement the api inside of satellite too (could use bundled modules for now) POC :heavy_check_mark:
Figure out the tcp streamdeck flow :heavy_check_mark:
Figure out the streamdeck firmware update checks :heavy_check_mark:
~~Consider if the satellite-api should be implemented as a module~~ TBD, but should be a follow up
Figure out the build packaging; I think that we should be shipping at least the elgato module in the builds :heavy_check_mark:
How to generate and sync udev rules :heavy_check_mark:
Finalise the api and release 1.0. At this point, every surface should be ported to a module with a 1.0 release using the 1.0 api.
Make this new implementation the default. Remove the old surface implementations, and tidy up :heavy_check_mark:
Migrations using the old config fields to auto-add instances :heavy_check_mark:
Handle import/export
The end?

Oct 13 '25 22:10 Julusian

I think this is largely ready to merge as a first chunk. For anyone interested in doing so, please give any feedback on this PR. I am still open for feedback on the naming. That is going to be quite crucial to pin down before it gets in front of many users.

Before merging I am tempted to hide it behind a settings flag, to hide it from view until it is ready for general use. I don't want to put it into develop, as I think this should be in 4.2 and it will get too tangled with the graphics overhaul in develop.

It is not a 100% complete api, but it handles usb surfaces (proven by https://github.com/bitfocus/companion-surface-elgato-stream-deck). ~~There is no support for tcp streamdecks in the api yet.~~ Some bits are still placeholders (no support for module config), but this will be filled out.

~~At this stage, I am calling the api used still 'experimental', as it needs some more testing with more modules.~~

Oct 19 '25 22:10 Julusian

I greatly welcome surfaces as independent modules. I also would not call them instances, actually this is not even the technical correct term. I think this should be quite similar (as similar as possible) to the device modules. That means the user can or has to download and install a surface module in order to being able to use a surface type. Then for each individual surface one instance of the according module is added, but instead instance we could just call it surface, like we call the instances of a device module a connection.

It should not be possible to activate or deactivate the module but instead the individual surfaces, just like with the connections. This may not be super useful with USB, but for network or midi or serial surfaces it would be a good step.

What is the current state now? Are you preferring one store or independent ones? I tend to prefer one store. Actually in #2387 I suggested to merge surfaces and connections into one module type (with different functionality) and I still think that this could be a far goal but for now that should only be a possible thought. Currently I think two types of modules are perfectly fine.

Nov 15 '25 20:11 dnmeid

I think that approach will be challenging to do; because we need the module to be able to tell us about surfaces which can be opened, and because sometimes a single 'hid device'(or equivalent) can provide multiple surfaces. For these reasons, I have opted to have one thread/process which handles every surface of that type.

To elaborate on the 'tell us about surfaces', I mean that:

For some (elgato SDS) we need something to perform the discovery and filtering. While we could handle this with some more json, it will complicate things and will limit us to supporting mdns for discovery.
- Perhaps someone wants to build a 'cloud emulator' service, in which case this discovery would be listing the emulators that have been setup in the cloud dashboard.
The 'xencelabs' module will expose one or more surfaces depending on what is paired to the dongle. we don't know ahead of time what they will be, or if they will appear.
This is currently built primarily for HID, and so does tries to avoid the modules needing to do any hid scanning (as that is slow and costly), but doesnt do the same for serial, so loupedeck has to do its own scanning based on when usb changes are detected.
The videohub-panel module is a server (this used to be a service in companion) so doesnt know ahead of time what might connect.

An alternate approach would be to require the modules to declare in their json the vendor+product id pairs that they support, so that some more explicit usb paths can be provided, but that wont help for any network surfaces. I am tempted to do this anyway, to allow us to optimise the events being passed around.

another problem with not having a visible 'instance' layer, is how do we choose which module version to use? we probably could limit it so that only one can be installed/loaded at a time, but that is in effect what this instances section of the ui does.

That said, I don't like the 'instances' term, and am not keen on this extra level of ui/complexity in front of users, but I dont see a way around it (other than to make this diverge quite a bit from how connection modules are managed).

There is also the likelihood that some of these 'instances' will want some configuration fields, which this gives us a place to put, and I wouldnt be surprised if someone wants to run multiple of one for some reason (again thinking to something like that cloud emulator idea). Perhaps that ties into #2849, as users will want to pick which surface modules to run on which daemons.

What is the current state now? Are you preferring one store or independent ones? I tend to prefer one store.

Currently in this PR all the old code remains, with the new modules being an opt in thing to enable each module and disable the appropriate setting to avoid fighting.
But I am likely to remove all the old logic very soon, as it adds some complexity to handle both in various places (but it has been really useful for testing)

Actually in https://github.com/bitfocus/companion/discussions/2387 I suggested to merge surfaces and connections into one module type (with different functionality) and I still think that this could be a far goal but for now that should only be a possible thought. Currently I think two types of modules are perfectly fine.

I still think that trying to merge these will introduce a lot of complexity for little benefit. I can only think of a couple of modules that could benefit from this.

Nov 15 '25 22:11 Julusian

another problem with not having a visible 'instance' layer, is ...

I hope there is not a misunderstanding. I think we should have a visible instance layer. The module itself has the code and knowledge how to handle a surface type. Modules can be installed or deinstalled and upgraded at the module page from the store. Once a module is present Companion will be able to detect one or multiple instances of that surface like today and they are shown in a surface list, like the connections are shown in their list. I'm talking here about how this is looking in the UI. Under the hood I don't say that we should have one instance of a module-class running per surface, only one instance of a surface-class per surface.

So the differences between a surface and a connection in terms of module workflow would be: no individual version per surface (could actually be done but maybe the pitfalls make it not worth the effort), some surface-types can be automatically detected and added (what in theory we could also provide for some connections).

My major concern is to make the workflows of surfaces and connections as similar as possible, so users can easily operate both once they understand the concept and don't have to learn different workflows for two module types.

There will probably still be more edge cases to solve with surface modules. One module that provides multiple surfaces is one of them. Can we imagine something like the common UDP listening socket problem, that we have for multiple VISCA modules, for surfaces? I know that other control applications don't give direct acces to hardware to plugins, but provide so called resources, basically a proxy to the hardware. That way different modules could use the same hardware. And it is also one possible way of emulating hardware or rewiring or remoting the connection between hardware and the module.

Nov 16 '25 01:11 dnmeid

I hope there is not a misunderstanding. I think we should have a visible instance layer.

Then I think Im confused, as it sounded like you want to drop or combine some layers, of which that one is the one I think is most confusing to users

The module itself has the code and knowledge how to handle a surface type. Modules can be installed or deinstalled and upgraded at the module page from the store.

Currently, this uses the same modules page as for connections, and each module is labelled if it is a connection or surface module.
So we have the same rules on multiple versions, upgrades etc.

My major concern is to make the workflows of surfaces and connections as similar as possible, so users can easily operate both once they understand the concept and don't have to learn different workflows for two module types.

I think that is what I have done; its the same to install/manage modules and to create/manage the instances. But then it does keep going with more that will be familiar from already existing of having the surfaces table and other bits.

So the matching part is setting up which surface types you want to use, equivalent to setting up your connections you want to use. Then it does diverge.

These surface modules will be included in the 'offline module bundle'. One difference I am planning is to embed a version of the stream deck module (maybe some others), so that they continue to work on a fresh install. We shouldnt do this for all of them, as most are disabled by default anyway.

Once a module is present Companion will be able to detect one or multiple instances of that surface like today and they are shown in a surface list, like the connections are shown in their list. I'm talking here about how this is looking in the UI.

This sounds like that we will auto-create the instances based on what modules are installed?

As a related note; back when I added the 'remote sources' and 'discovered surfaces' tables, I did look at doing one/both of those within the current surfaces table. But I quickly found issues in how to present that, because a connection may or may not have a serialnumber associated with it, and even if it did that serialnumber could be active separetely to the connection.
So it quickly became messy and confusing over whether something should be one or two rows, which all depended on the exact state; or to show each sds connection always as its own row separate from the actual surface portion.
Throw in that the order of this table matters and surface-groups and it becomes too much.

My point with that is that I am worried that trying to merge two differing concepts for the UI is hard and messy. It is always much easier to present in the ui similarly to the backend structures (or at the very least the structures sent to the ui).

It should not be possible to activate or deactivate the module but instead the individual surfaces, just like with the connections.

This is challenging to do in many cases. (I did start looking at this separately before this PR).

For some, we don't make the connections, the library gives us already opened hid devices. So we could reject them, but we will still be holding the maybe exclusive hid handle.
Some surfaces don't have real ids, instead we have to fake them. This needs some more work/thought on how to handle properly as each does their own thing. Typically this involves some amount of counting how many are open to avoid id collisions, so ids arent particularly stable.
Anything network based, we need to open a connection to query the id. we cant reliably cache that, as the id could change. Also, for some of these (stream deck) we already have an enable/disable toggle on the connection

I do think we should look to support enabling/disabling per surface, but it will depend on support for that from the module, it wont be available for every module.

Regarding one process per surface or per module, I strongly feel that per module is the best approach.

I am using processes here largely for the memory/crash isolation and cleanup properties (like we have for connections), not for performance.
As far as Im aware noone has complained about performance in the current setup of not using any threading.

One per module keeps things simpler (we can let the module handle some discovery tasks). Inside these processes is also some util (like the pincode images generation), so that we can avoid the overhead of passing those images over ipc, but at the same time that does cost memory, so we don't want to spawn too many of these.

It is also worth pointing out that in the current approach, using the same modules in companion-satellite is fairly simple https://github.com/bitfocus/companion-satellite/pull/230. The current abilities of the surfaces from these modules is a subset of what the satellite api is capable of

Nov 16 '25 12:11 Julusian

A couple more thoughts that I just had;

I have continued with this today, and the old implementations are now removed.
Changes are still solely inside this branch (other than general refactoring and reworking which was done on main)

I believe that the surface instances page is mostly code shared with the connections page, so is a very familiar page and will continue to be so.

It would pose some challenges for discovery, but I suppose the remote surfaces table could be combined into the surface instances table. On other words, make it so that an instance is either usb/inbound connections, or is set to make an outbound connection to a surface. That would better match how connections behave, and cuts out a table, but has challenges.

Now that I think about it, that outbound structure would be great for working with midi.. (for a module that understands panel layouts), as it could easily produce config fields to input the paths. Maybe I need to review naming to make it be less network focused.

Nov 17 '25 00:11 Julusian

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@julusian/image-rs@2.0.2 ⏵ 2.1.1	⁺²		⁺¹	⁺⁵
	@companion-surface/host@1.0.0

View full report

Nov 30 '25 19:11 socket-security[bot]

Well... I tried it and

The current version fails with an infinite restart loop on Windows (yarn dev)... because the filename is being interpreted as a bad URL by ESM Loader. It was very difficult for me to debug -- but eventually I traced it to spawn and hit a dead-end. Changing 'entrypoint` to a URL string just caused another problem, so I gave up...

2025-12-01T02:10:26.088Z debug Instance/ProcessManager Instance "xkeys" command: ["E:\\XX\\companion\\.cache\\node-runtime\\win32-x64-22.21.1\\node.exe","E:\\XX\\companion\\companion\\dist\\Instance\\Surface\\Thread\\Entrypoint.js"]      
2025-12-01T02:10:26.090Z info Instance/Child/xkeys Process started process 27272
2025-12-01T02:10:26.102Z info Instance/ProcessManager Starting instance: elgato-stream-deck 
2025-12-01T02:10:26.103Z debug Instance/ProcessManager Instance "elgato-stream-deck" command: ["E:\\XX\\companion\\.cache\\node-runtime\\win32-x64-22.21.1\\node.exe","E:\\XX\\companion\\companion\\dist\\Instance\\Surface\\Thread\\Entrypoint.js"]
2025-12-01T02:10:26.107Z info Instance/Child/elgato-stream-deck Process started process 25672
2025-12-01T02:10:26.339Z verbose Instance/Child/xkeys stderr: node:internal/modules/esm/load:187
    throw new ERR_UNSUPPORTED_ESM_URL_SCHEME(parsed, schemes);
          ^

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'e:'
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:187:11)
    at defaultLoad (node:internal/modules/esm/load:82:3)
    at ModuleLoader.load (node:internal/modules/esm/loader:815:12)
    at ModuleLoader.loadAndTranslate (node:internal/modules/esm/loader:594:31)
    at #createModuleJob (node:internal/modules/esm/loader:624:36)
    at #getJobFromResolveResult (node:internal/modules/esm/loader:343:34)
    at ModuleLoader.getModuleJobForImport (node:internal/modules/esm/loader:311:41)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:664:25) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

The code runs in WSL, and just to make me miserable, the packaged version (i.e. yarn dist on Windows) runs in Windows even though debug output shows the same windows-style filenames and webpack is supposed to be generating modular output, i.e. ESM (but maybe it isn't?? or maybe the way companion & shared-lib are packed is different in dev?)...
Unfortunately, I find the UI rather confusing. Partly it's the name "Instances", which you definitely need to change and partly its that the separation in the UI of the "modules" page from "connections"/"instances" has always been confusing. I think you "fixed" it for "Connections" by allowing the user to add new modules directly on that page, but on the surface instances page the "All Available" filter doesn't work. One thought: get rid of the "Modules" page altogether and filter the modules appropriately on these other two pages...
Minor bug: Mirabox and ContourShuttle are shown in the list of "Instances" on the Instances page (left panel) but the modules are not installed (Red triangle w/ tooltip "Unknown Module" in the left-hand table, and the modules not shown in the right-hand panel). Shouldn't they have been auto-installed since they were enabled previously? (Though here too is a very minor problem -- I have no need for Mirabox, I think it's just enabled by default. It's easy enough to delete or ignore.)

The main quandary is what to do about the name "Instances". We all agree that it needs to be changed... by far, the simplest approach would be to consider what it is: it corresponds to what is currently called "enabled" for a particular product line and/or brand (Enable XKeys, Enable ContourShuttle, etc. which include all models by that manufacturer), so it could be called something like "Brands" or "Products" or "Device Types" (rather loosely). To make it even better:
In keeping with the idea that the page should be organized similarly to connections, just add the modules list to the right-panel of the "Configured Surfaces" page and label it something like "Add Surface Brand" (or product line... or type). The panel could either be arranged like Connections is currently done (gets hidden when the user selects a "configured" surface), or in a tabbed pane, as in the Buttons page (in fact, Connections could be converted to a tabbed pane too, to make it clearer how to get back to the "Add Connection" panel).

FWIW: Note that the current naming is exactly backwards: the thing you call "Instances" is the class and the thing you call "Configured" are instances of the classes. I'm sure you can explain on a deeply technical level why it's not really so, but that's the way it looks to someone who didn't write the code -- and that's my point!

Another possibly clarifying thought: the user never adds a surface manually (except for emulators) -- they plug them in and then Companion registers them it the product is enabled -- and that is a good feature -- so if the right-panel tab just contains the "Add Surface Brand" panel, then once a brand is added, connected devices will be recognized and a user can click on it to configure it, etc. As already mentioned, in most other aspects this follows the behavior of the Connections panel. Note also that using the "Installed Only" filter shows what's currently installed, so there's no need for the left-hand table in the current "Instances" window; The various controls could easily enough be added to the line for installed modules in the right-hand panel.

Well, that's a bunch of thoughts anyway...

Dec 01 '25 05:12 arikorn

1 ) Cool.. I haven't tested this on windows at all of course, but I guess its a similar issue to what you found with the companion build process, so Ill look into that 3&4) Because there isn't a 'stable' version of some of the modules published, those modules get a little grumpy and aren't auto-installable. I suppose that might be a bug to look at, to not offer them at all when in this state? 3 ) > One thought: get rid of the "Modules" page altogether and filter the modules appropriately on these other two pages...

The reason these are separate is that they are doing very different tasks. The modules page is all about managing which versions you have installed, with the connections page about instantiating those modules (with the shortcut to install the latest version if needed) and configuring those connections.
If we were to merge them, I think it would clutter the 'add connection' list (which I guess wouldn't just be 'add connection' anymore) and that right panel would need to also swap out to manage the versions of a module (unless that was moved to a modal or something?
But also, the 'import' buttons on the modules page work for both connection and surface modules, which I think would be confusing if put anywhere else. The system would happily let you import a surface module from the connections list and either not provide guidance on what you imported, or to redirect to you a completely different area of the application. (this is why the change I made to put the modules page under the connections in the nav only lasted for a few days).
Now this might not be the best conclusion, but in my experimenting this is what I have found, resulting in the arrangement that we have.

5 ) > "Brands" or "Products" or "Device Types"

The current idea I have is 'integrations', but I am not certain.
Something to remember here is multiple modules support more than one brand/manufacturer, and could even not translate to a product as such. (I am considering moving the satellite api to be one of these modules)

6 ) Part of my resistence to this is to try and keep the flow the same for connection and surface modules. But also I worry that the right panel will also become too cluttered/confusing to jam that in there. Especially if it is supported to absorb both the instances and modules portions. Because there is the list of current instances, a way to add new ones, a page to configure an instance, a list of modules and a page to manage versions of a module.
Thats 5 more views to do things that while semi-related to the left panel, the left panel is of no use while doing them. But seeing the list of instances while you add/configure some is useful.

Connections could be converted to a tabbed pane too, to make it clearer how to get back to the "Add Connection" panel

I converted these away from tabbed panels a while back, because I found the UX weird. There would often be just one tab, and like with the button editor, when you switched away from some tabs they would disappear. Additionally, when we collapsed down to a single column on mobile, we didnt have a plan for how that should be affected, so the right panel would get dumped underneath the left which made for an awful experience. But I guess the other approach for mobile would be to add the left panel to the list of tabs when on mobile.

FWIW: Note that the current naming is exactly backwards: the thing you call "Instances" is the class and the thing you call "Configured" are instances of the classes. I'm sure you can explain on a deeply technical level why it's not really so, but that's the way it looks to someone who didn't write the code -- and that's my point!

Well the problem is that a few years back connections used to be called instances. So I have been trying to avoid that word to minimise risk of user confusion, and was working to remove it from everywhere until starting on this.
But even now, it is still an instance of a module, so it depends on at what level you look at it.

Note also that using the "Installed Only" filter shows what's currently installed, so there's no need for the left-hand table in the current "Instances" window;

Except you can have an instance installed but not 'configured'. If you import the offline module bundle you will get every type of connection (and soon surface) module we have installed, but they will only be enabled when you explicitly 'add' it. And right now, none of the surface modules have any configuration fields, but that will change at some point and needs somewhere to go.
For now it is limited to one instance of each module, but in the future I expect that some module will want to allow multiple instances.
And this follows the same module version rules as we do for connections, so users do need to explicitly update the module versions in use

Some of this is planning for future functionality that I expect to crop up, so may not make the most sense right now. But right now it also mirrors the connections page pretty closely, so while conceptually the page is confusing, using it is hopefully familiar

Dec 01 '25 10:12 Julusian

@Julusian replied to my suggestion:

One thought: get rid of the "Modules" page altogether and filter the modules appropriately on these other two pages...

The reason these are separate is that they are doing very different tasks. The modules page is all about managing which versions ... ...and that right panel would need to also swap out to manage the versions of a module (unless that was moved to a modal or something?

Yeah, that was my thought -- the way it's done on the Connections page. That said, another option, that might be simpler for all, is to move the "Modules" page back to it's short-lived tenure under Connections, and restrict it to Connections modules, then keep the so-called "Instances" page to manage the Surface Modules. Just call them "Connection Modules" and "Surface Modules" or something else that's inclusive enough. Maybe "Input Modules"? or "User-Input Modules"? (You could even rename "Surfaces" "Input Devices", come to think of it.) Anyway, it seemed to me that the "Instances" page did pretty much everything that that "Modules" page does, except that the left and right panels are swapped. (OK, an the table, which is still a bit confusing -- the only "table" for loaded connections modules is exactly what I suggested above: you just turn on the "Installed-only" filter in the Connections page and you get a list with all sorts of cogs and doodads for configuration.)

5 ) > "Brands" or "Products" or "Device Types" The current idea I have is 'integrations', but I am not certain.

The problem I'm having with all of these names is that they could be applied equally to "configured devices" and the "modules", so it leaves the user guessing which is which. My current favorite, as above might be "Input Modules" or "User-Input Modules" (and the other page could be "known surfaces", or better: "my surfaces"! ... but "configured" is fine).

6 ) Part of my resistence to this is to try and keep the flow the same for connection and surface modules.

Wait, isn't that what I'm campaigning for 😀 -- to make the Surfaces page look/behave like the Connections page!

I converted these [Connections page, etc.] away from tabbed panels a while back, because I found the UX weird. There would often be just one tab, and like with the button editor, when you switched away from some tabs they would disappear.

My suggestion would be to have two permanent tabs: the modules list and the configs panel. Right now when a user selects something in the left pane (connections page) the configs pane covers over the modules list and the astute user will notice the X in the top-right corner. It definitely has a certain elegance to it, but I worry that it's a bit subtle.

And on the Surfaces page, the right panel is just begging for content when nothing is selected...

Dec 02 '25 07:12 arikorn

move the "Modules" page back to it's short-lived tenure under Connections, and restrict it to Connections modules, then keep the so-called "Instances" page to manage the Surface Modules.

Then how will you install prerelease versions of a surface module, or uninstall versions? Currently that lives in that modules page.

Wait, isn't that what I'm campaigning for 😀 -- to make the Surfaces page look/behave like the Connections page!

Perhaps I should start by pointing out that the Surface Instances page is almost exactly the same as the Connections page, and are both largely comprised of the same couple of react components.
So in its current form, those pages are the same. The main difference is the extra 'configured surfaces' page.

Anyway, it seemed to me that the "Instances" page did pretty much everything that that "Modules" page does, except that the left and right panels are swapped. (OK, an the table, which is still a bit confusing -- the only "table" for loaded connections modules is exactly what I suggested above: you just turn on the "Installed-only" filter in the Connections page and you get a list with all sorts of cogs and doodads for configuration.)

Not at all.

In the modules page, you install/uninstall modules. It is the only place where we let you install prerelease versions of a module. Installing a module is essentially just downloading files onto disk, and tracking what has been installed. We dont run the code until an Instance is created.
You can also import modules; primarily the offline bundle.

In the instances page, you 'instantiate' those modules, and can configure them. (For surfaces, none currently have any config other than the label&version. And there is a semi-broken logic to limit to one instance of each module).
Here we do allow auto-installing the latest version of modules on demand, to make it easier to get something setup. (We intentionally limited this to just the latest and installed versions, to avoid a dropdown presenting too many options that the user wont know what to pick. vmix has over 20 versions, we dont need to list them all to every user)

So while you can install the latest version of a module from the instances page, you don't get the full flexibility to pick older versions. Deprecated modules and versions are also hidden.
From the modules page, you can install versions, but you can't 'instantiate'/activate them

Maybe we could do the module management inside the connections/surface instances pages. A modal might work best. Basically just lifting that right table in the modules page into a modal. (I think a modal because it wont relate at all to the left panel, and you are more likely to want to go back to what you previously had in the right panel) That would also require:

A way to toggle to show deprecated modules, as they are currently hidden from this list unless already installed (I think there is only one currently search 'Hippotizer'; there is one result, and also a deprecated result with the same name)
Move the import buttons here. It would probably be pretty easy to limit the 'import module package' one to check the module type. Is it ok that the 'import offline module bundle' will be importing modules of both types? I dont think we should be limiting that So actually, I'm not opposed to this right now. It might be good to do, but isnt tied to this PR as it largely reworking the existing UX flow for connection modules.

Dec 02 '25 11:12 Julusian

Thanks @Julusian, I think your conclusion is essentially what my original thought/suggestion was, so I'm all for it of course. My second suggestion, to limit the Modules page and move it under Connections, would not require a reworking of the connections UX flow, but would allow the flow with surface modules to be more like the Connections flow. I think if the "Instances" page is made more like the Modules page, and the right-panel of the Configured page is made similar to the Connections page, it would cover the current functionality and keep the two UX flows parallel (apologies if I'm ignoring something obvious!!)

Dec 02 '25 19:12 arikorn

My second suggestion, to limit the Modules page and move it under Connections, would not require a reworking of the connections UX flow, but would allow the flow with surface modules to be more like the Connections flow.

The problem is that if there is no 'modules' page for surface modules, then there will be no way to uninstall surface modules or to install prerelease ones.
Unless that got built that into the instances page, but that would mean the workflows for connection and surface modules would differ.

I think if the "Instances" page is made more like the Modules page, and the right-panel of the Configured page is made similar to the Connections page, it would cover the current functionality and keep the two UX flows parallel (apologies if I'm ignoring something obvious!!)

Again, this would mean the workflows would differ. Maybe not substantially, but the layout would be significantly different so I expect users wouldnt notice that they are actually the same.

Perhaps something that would help is to think about the contents of the configured page as being something that an instance produces, and equivalent to the actions/feedbacks/variables that connections produce.

Dec 02 '25 20:12 Julusian

The problem is that if there is no 'modules' page for surface modules,

Not sure if we're misunderstanding each other... My suggestion is to convert the "Instances" page into a second "Modules" page -- its content and layout would be the same but filtered for surface modules. Then the table currently in the left panel can either go away altogether -- as it or its functionality would be moved to the right-panel of the Configured page. -- or it could be moved to the top of the right panel or tabbed into it or shown when nothing is selected... (or it could be put in the Configured page under the "known surfaces" table instead of the right-panel) In my, perhaps naïve, view the table currently in the Instances left-panel replaces the "Settings > Surface" page, which I've already thought would fit perfectly in the Surfaces right-panel... Well, it's not a big deal in the end... I just wanted to clarify the idea.

Dec 02 '25 20:12 arikorn

Perhaps it worth explaining my motivation in these comments briefly: I have, too many times, been totally mystified as to why Companion wasn't recognizing my Contour Shuttle, only to eventually realize that it was disabled in a page buried in a completely different part of the UI. This is why I wanted to put those settings in the "Configured" page right-panel -- so the enabled state is immediately obvious on the page affected by it -- and why, in this feature, I'm campaigning for the equivalent to be done.

That said, the "Instances" page is a big step in the right direction, I just think we can do one step better with no negative consequences.

Dec 02 '25 23:12 arikorn

My suggestion is to convert the "Instances" page into a second "Modules" page -- its content and layout would be the same but filtered for surface modules.

right ok, I wasnt quite understanding what you meant here. I think I do now.
Still dont think I agree with it though

Then the table currently in the left panel can either go away altogether -- as it or its functionality would be moved to the right-panel of the Configured page. -- or it could be moved to the top of the right panel or tabbed into it or shown when nothing is selected... (or it could be put in the Configured page under the "known surfaces" table instead of the right-panel)

My worry here is still that the 'surface instances' page is comprised of 3 panels that all belong together that I dont think will be intuitive to squeeze into the empty space of the 'configured' page.
I dont think putting it below will be nice, on my dev setup 75% of screen height is occupied from the handful of streamdecks I have tested with recently, and my main setup the table is 150% of the screen height. I think it will be too hidden there

I'm also a little worried that users may not realise that surfaces in the configured list can be configured at all.. there used to sometimes be a cog to open a modal, then it changed to this split view with the right column indicating they can be configured. But there are other ways to make it a little more visible

In my, perhaps naïve, view the table currently in the Instances left-panel replaces the "Settings > Surface" page, which I've already thought would fit perfectly in the Surfaces right-panel...

Yeah that is correct, but this instances page has much more complex ux flows, and I simply think will feel too squeezed into that space.
But perhaps this actually means those settings shouldnt move into the right panel of this page, because it doesnt strictly belong to any particular one of these pages... I hadnt thought about that

Also, putting the settings into this on one page is making me worried this will feel cluttered (considering it used to be a very simple page). Doing both of these would put it at 5 views that could be in that right panel (which is more than even the buttons view, if presets counts as 1)

I have, too many times, been totally mystified as to why Companion wasn't recognizing my Contour Shuttle, only to eventually realize that it was disabled in a page buried in a completely different part of the UI. This is why I wanted to put those settings in the "Configured" page right-panel -- so the enabled state is immediately obvious on the page affected by it -- and why, in this feature, I'm campaigning for the equivalent to be done.

I hadnt thought about that. But I'm still just worried that when the user needs to interact with it that it will feel squeezed in there. It will be a bunch of navigating between the 3 instances views while setting things up there, during which the left panel wont be touched or looked at.
Maybe it wont be, or maybe it is useful to see the left panel while configuring the instances.
Now that I think about it, it will flow the same as using the connections page on mobile/narrow, so getting a feel for how it would behave doesnt require any code changes..

Dec 03 '25 16:12 Julusian

@Julusian wrote:

Doing both of these would put it at 5 views that could be in that right panel (which is more than even the buttons view, if presets counts as 1)

I don't think this is quite right but suspect at this point we'll need a "live" demo to really get a feel for it. How about a trade: you take a break from this PR and I'll try and mock something up -- hopefully in code --for discussion, meanwhile you can take the "freed time" to review my two most recent PRs. If it takes me more than a day or two, then I'll concede... (BTW, I see you just started on collapsible panels for presets... something like that could also be used to simplify the "instances table" here if it were in the Configured page right-panel.)

What do you think?

Dec 03 '25 21:12 arikorn

I don't think this is quite right but suspect at this point we'll need a "live" demo to really get a feel for it.

I count:

configure surface
general surfaces settings (moved from main settings area)
surface instances list
add surface instance list (unless that becomes a modal, further diverging from the ux/flow of connections)
configure a surface instance (unless that gets done collapsible panel style, further diverging from the ux/flow of connections)

I am not keen on the idea of using collapsibel panels inside/isntead of a surface instances table. I think it will be quite hard to use when everything can move, and when we have some surface instances which have enough config fields which would need scrolling. (From experience of massive code files, having to scroll constantly means I can never find what I am looking for quickly)

Tbh, this reshuffle can definitely happen after this PR is merged. We arent discussing anything changing in the core implementation. So I suggest waiting until this is merged before doing much playing around with ui (I am also frequently rebasing this branch on main, so any branch you make will have messy merge conflcits fast)

Dec 04 '25 00:12 Julusian

@Julusian wrote:

tbh, this reshuffle can definitely happen after this PR is merged.

Fair enough. I think we got here because the rearrangement made it clear (to me) that the page called "Instances" could be renamed (or perhaps replaced by) "Surface Modules" or "Input Modules" (plus, I guess, my little "motivation spiel"), but perhaps we ended up bit to far into the thicket...

FWIW, I only envision two or three content items in the right panel:

1 configure surface (i.e. click on something in left panel) 2. surface "instance" table (when no "configured surface is selected") 3. configure a surface instance (which could be accordioned or made a modal window, or done in the module page if we're limiting to one instance per module...)

For the other two:

"general surfaces settings (moved from main settings area)" -- I thought nothing would be left after this PR, except lockout settings, which could be put in a security page along with the admin password, which really doesn't fit where it currently is (my opinion of course) -- a link on the Surfaces page would be more than enough to make it more accessible.
"add surface instance list (unless that becomes a modal, further diverging from the ux/flow of connections)" -- this would be on the "Surface Modules" page. Though I assume if we subsequently wanted to get rid of Module pages, then it would end up here. Not sure we really want to eliminate them though, but if we did, it would be the same for both pages...so 🤷

I think we agree that Surface modules are not the same as Connection modules since it has an extra level, and so the UX/flow must necessarily diverge to some degree from the connections flow,... so I suspect what we're debating here is what is the least divergent!

OK. I'll do my best not to continue this thread. Thank you as always for considering my comments!

Dec 04 '25 07:12 arikorn

The settings page becomes this currently: Maybe more of it will dissappear over time (the api doesnt currently provide enough for the elgato-plugin to become a module, but maybe it should be)

"add surface instance list (unless that becomes a modal, further diverging from the ux/flow of connections)" -- this would be on the "Surface Modules" page.

I think that will be weird. You can see the list of current instances on one page, but add a new instance on a completely different page (adding one would presumably then redirect you back to the list page, to open the configuration panel)

I think we agree that Surface modules are not the same as Connection modules since it has an extra level, and so the UX/flow must necessarily diverge to some degree from the connections flow,... so I suspect what we're debating here is what is the least divergent!

No, I am saying that they are largely the same, just what they do that differs (in that one provides surfaces, one provides actions). But the instance and module stuff and what the user needs to be able to do with them is the same. The problem is that the ui presents this is a way that does not appear intuitive, at least not to existing users.

Maybe that is the difference which is the problem. The connections page is put very front and center, but the surface instances one is more hidden. But that is by design, as users will rarely need to touch the surface instances, but more frequently touch the connections.

Dec 04 '25 19:12 Julusian

(the api doesnt currently provide enough for the elgato-plugin to become a module, but maybe it should be)

Shouldn't "Enable Elgato Plugin Support" be a setting in the StreamDeck module? Is it relevant to other surfaces?

("Watch for usb devices" could be put on the top line next to "Rescan" -- might even make more sense there?)

Point being, of course, that maybe it doesn't need to be a panel...

I think that will be weird. You can see the list of current instances on one page, but add a new instance on a completely different page

This is partly addressed in my next comment:

No, I am saying that they are largely the same, just what they do that differs

Doesn't the Surface Module concept have one more layer than Connections? I.e., Connections has modules and instances but Surface Modules have modules, instances, and "configured surfaces"? It seems to me that that's what we're struggling with both in nomenclature and UI flow... and part of the "problem" is that to a user, Connection instances act more like "configured" surfaces than like "surface instances", even if under the hood it's the other way around...

Dec 05 '25 07:12 arikorn

After some time sitting with this, 'Surface Instances' was really bugging me. So I'm trying out 'Surface Integrations' this is a ui only change, internally they are still 'instances' which does lead to some slightly confusing naming, but I think this unavoidable unless we want to rename all the instances stuff to integrations. (some is referred to as connections)

But I think I am almost done on this now. Now that 4.2 is out, all I have left is to fix a couple of bugs and to make 1.0 releases of the libs and modules, then this is ready to merge. So I shall try and get this wrapped up and merged today, or tomorrow if not

Dec 15 '25 21:12 Julusian

Duplicate

Dec 16 '25 00:12 Julusian

Julian, I'm a bit confused by the comment immediately above. It starts with a "call-out" to me, but it looks identical to your original comment, which I replied to above. Did something get lost or added by accident?

Dec 16 '25 02:12 arikorn

sorry, github still showed it as a draft and I didnt notice it was already posted as it had been collapsed under the 'show more'..

Shouldn't "Enable Elgato Plugin Support" be a setting in the StreamDeck module? Is it relevant to other surfaces?

No, this is for the websocket protocol used by the plugin for the streamdeck softare. On a technical level, the implementations have nothing in common. Historically we have only allowed it so that we can either open streamdecks or this plugin can be used. But that linking will be going away (there was a request to do so in 4.2 also, but that didnt happen)

Doesn't the Surface Module concept have one more layer than Connections?

Is this current UI, yes, but technically, no. The 'configured surfaces' are roughly equivalent to actions and feedbacks.

Connection instances act more like "configured" surfaces than like "surface instances"

I dont agree with that. configured surfaces will appear as they are discovered, you cant manually add/remove them.
Surface Instances however, have the same label, version, module selection, enable/disable as connections; and need to have all those things (maybe not label really).

I see it as in the same way that you setup a module to provide you with the actions so that you can interact with a device, you setup the surface instance to provide you with the surfaces so that you can trigger buttons.

I definitely do productions where I don't touch the surfaces tab at all.

Maybe we should stop on this, I don't feel like we are making any progress to any changes..

But ill merge this today and maybe some beta users will get confused and complain and have ideas on what to improve :shrug:

Dec 16 '25 10:12 Julusian