wails icon indicating copy to clipboard operation
wails copied to clipboard

[V3] Refactor binding transport layer

Open APshenkin opened this issue 5 months ago • 17 comments

Description

This PR refactors binding layer + provides new Transport abstraction that allow user to replace default transport with WebSockets, custom protocols, or any other transport mechanism.

Motivation:

  1. System bindings uses HTTP transport for both request and response, but services bindings utilize JS callbacks
  2. Some usage scenarios doesn't allow to use HTTP transport (e.g. running external page with https and access to bindings)
  3. There is proposal to move default transport to Websocket (this needs continues investigation to identify how provide security guarantees)

List of changes:

  1. Drops JS callbacks for bindings in HTTP transport. Now all responses are returned via http. Events still uses JS callbacks, as HTTP doesn't support bidirectional connect.
  2. Replace GET /wails/runtime with POST /wails/runtime to handle cases with large data (#4428)
  3. Moved /wails/capabilities and /wails/flags under messageProcessor for object System to allow other transports handle those requests as well
  4. Inject all flags that were specified for the app in webview (previously only resizeHandleWidth and resizeHandleHeight for Windows were passed)
  5. Decouple logic of bindings transport from message processor.
  6. Introduce Transport interfaces, that allow to replace default HTTP transport for bindings handling

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue) https://github.com/wailsapp/wails/issues/4418 https://github.com/wailsapp/wails/issues/4428 https://github.com/wailsapp/wails/issues/1516 https://github.com/wailsapp/wails/issues/4686

Type of change

Please select the option that is relevant.

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [x] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration using wails doctor.

  • [ ] Windows
  • [x] macOS
  • [ ] Linux

If you checked Linux, please specify the distro and version.

Test Configuration

Please paste the output of wails doctor. If you are unable to run this command, please describe your environment in as much detail as possible.

 Wails (v3.0.0-dev)  Wails Doctor

# System

┌──────────────────────────────────────────────────┐
| Name          | MacOS                            |
| Version       | 26.0.1                           |
| ID            | 25A362                           |
| Branding      | MacOS 26.0.1                     |
| Platform      | darwin                           |
| Architecture  | arm64                            |
| Apple Silicon | true                             |
| CPU           | Apple M4 Max                     |
| CPU 1         | Apple M4 Max                     |
| CPU 2         | Apple M4 Max                     |
| GPU           | 40 cores, Metal Support: Metal 4 |
| Memory        | 128 GB                           |
└──────────────────────────────────────────────────┘

# Build Environment

┌─────────────────────────────────────────────────────────┐
| Wails CLI    | v3.0.0-dev                               |
| Go Version   | go1.24.0                                 |
| Revision     | 4a445ce218b58db4a16ee8fa84eac0d7c9a69932 |
| Modified     | false                                    |
| -buildmode   | exe                                      |
| -compiler    | gc                                       |
| CGO_CFLAGS   |                                          |
| CGO_CPPFLAGS |                                          |
| CGO_CXXFLAGS |                                          |
| CGO_ENABLED  | 1                                        |
| CGO_LDFLAGS  |                                          |
| GOARCH       | arm64                                    |
| GOARM64      | v8.0                                     |
| GOOS         | darwin                                   |
| vcs          | git                                      |
| vcs.modified | false                                    |
| vcs.revision | 4a445ce218b58db4a16ee8fa84eac0d7c9a69932 |
| vcs.time     | 2025-08-04T22:42:25Z                     |
└─────────────────────────────────────────────────────────┘

Checklist:

  • [ ] I have updated website/src/pages/changelog.mdx with details of this PR
  • [x] My code follows the general coding style of this project
  • x ] I have performed a self-review of my own code
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] My changes generate no new warnings
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] New and existing unit tests pass locally with my changes

Summary by CodeRabbit

  • New Features

    • Added support for custom transport layer implementations, enabling alternative IPC protocols like WebSocket.
    • Introduced WebSocket transport example with backend and frontend integration.
    • Added structured error handling framework for improved debugging.
  • Refactor

    • Refactored internal runtime message processing to support pluggable transport backends.
    • Redesigned IPC transport architecture for better extensibility.

✏️ Tip: You can customize this high-level summary in your review settings.

APshenkin avatar Nov 10 '25 16:11 APshenkin

Walkthrough

Introduces a pluggable IPC transport layer (Transport interfaces, HTTP and WebSocket transports), refactors runtime and message processing from HTTP-centric handlers to a RuntimeRequest-driven model, adds a typed errs package with codegen, and includes a WebSocket transport example with generated frontend bindings and runtime API changes.

Changes

Cohort / File(s) Summary
Dependency Updates
v3/go.mod, v3/examples/*/go.mod, v3/tests/*/go.mod
Removed indirect github.com/pkg/errors v0.9.1, bumped github.com/wailsapp/go-webview2 to v1.0.22 across examples, added github.com/gorilla/websocket v1.5.3 to main module.
Transport Abstraction & Implementations
v3/pkg/application/transport.go, v3/pkg/application/transport_http.go, v3/pkg/application/transport_event_ipc.go, v3/examples/websocket-transport/transport_websocket.go
Added a public Transport interface, HTTPTransport implementation, EventIPCTransport, and a WebSocket transport example implementation.
Application Startup & Options
v3/pkg/application/application_options.go, v3/pkg/application/application.go
Added Transport field to Options and rewired application startup to initialize and manage the configured transport, serve assets via transport, and wire event delivery.
Message Processing Refactor
v3/pkg/application/messageprocessor.go, v3/pkg/application/messageprocessor_*.go, v3/pkg/application/messageprocessor_args.go, (removed) v3/pkg/application/messageprocessor_params.go
Replaced HTTP writer-based handlers with a request-driven API using RuntimeRequest and Args; converted many process* methods to return (any, error); removed old QueryParams/Args file and added a new Args/MapArgs implementation.
Runtime API & Bundles
v3/internal/runtime/.../runtime.ts, v3/internal/runtime/.../calls.ts, v3/internal/runtime/.../dialogs.ts, v3/internal/assetserver/bundledassets/runtime.js, v3/internal/runtime/runtime.go
Added RuntimeTransport interface and setTransport/getTransport, updated runtime call path to prefer custom transport (fallback to HTTP), adjusted runtime bundle exports, and changed Core to accept flags.
WebSocket Example & Frontend Bindings
v3/examples/websocket-transport/* (Go: main.go, GreetService.go, transport; JS: websocket-transport.js, assets, generated bindings and index.js, README.md, assets/index.html)
Added complete WebSocket transport example: backend service, server setup, client transport JS, generated bindings (greetservice.js, index.js), HTML demo, and README documenting usage.
Error Framework (errs)
v3/pkg/errs/errors.go, v3/pkg/errs/error_functions.gen.go, v3/pkg/errs/utils.go, v3/pkg/errs/codegen/error_functions/main.go, v3/pkg/errs/README.md
New errs package with ErrorType constants, WailsError interface, utilities (Is/Cause/Has), and code-generated typed constructors/wrappers for categorized errors.
Event & Window Changes
v3/pkg/application/event_manager.go, v3/pkg/application/webview_window*.go, v3/pkg/application/window.go
EventManager dispatch simplified (removed per-window pre-dispatch), removed callback-style window JS response helpers (CallResponse/CallError/DialogResponse/DialogError) and Callback interface; runtime.Core calls updated to pass flags.
Asset Server & Bundling
v3/internal/assetserver/...
Moved runtime/transport serving into transport-driven middleware; runtime bundle export mappings adjusted to expose transport helpers and objectNames/clientId.
Docs & Changelog
docs/src/content/docs/guides/custom-transport.mdx, v3/UNRELEASED_CHANGELOG.md
New guide for custom transports and changelog entry.
Small Example Dependency Files
v3/examples/*/go.mod
Adjusted example go.mod files to reflect dependency changes (webview2 bumped, pkg/errors removed).

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant FE as Frontend (runtime.ts / JS)
    participant RT as Runtime API (runtime.ts)
    participant TR as Transport (HTTPTransport / WebSocketTransport)
    participant MP as MessageProcessor (server)
    participant APP as App handlers (process* methods)

    FE->>RT: call(object, method, args)
    RT->>RT: if customTransport set?
    alt custom transport available
        RT->>TR: transport.call(object, method, window, args)
    else
        RT->>TR: HTTP POST /wails/runtime (default transport)
    end
    TR->>MP: HandleRuntimeCallWithIDs(ctx, RuntimeRequest)
    MP->>APP: route by object -> processX(req)
    APP-->>MP: (result, error)
    MP-->>TR: response payload / error
    TR-->>RT: transport response
    RT-->>FE: resolve promise (result or error)

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~150 minutes

  • Areas requiring extra attention:
    • Message processor refactor (many files): ensure consistent use of RuntimeRequest/Args and correct error wrapping with errs.
    • Transport lifecycle and asset-serving integration (application.go, transport_*.go): verify Start/Stop, ServeAssets, middleware ordering, and event wiring.
    • Runtime transport switching (runtime.ts, bundled runtime.js, calls.ts): confirm fallback behavior, headers, and content-type handling.
    • Error framework usage: ensure all new errs constructors/wrappers are used consistently and that error chains preserve causes.
    • Removal of window callback methods: check for remaining call sites and JS expectations (generated bindings/runtime behavior).

Possibly related PRs

  • wailsapp/wails#4471 — overlaps on Window API and webview_window changes (removal/adjustment of callback methods).
  • wailsapp/wails#4100 — touches runtime exports and transport-related runtime APIs (setTransport/getTransport, bundled runtime changes).
  • wailsapp/wails#4066 — related transport/messageprocessor and error-handling refactors affecting the same subsystems.

Suggested labels

Enhancement, bindings, go, size:XXL, lgtm

Poem

🐇 A tiny rabbit hops on code so new,

Transports unbound — WebSocket sings through.
Errors now typed, requests flow neat and clear,
Flags in the core, bindings whisper near.
Hooray — pluggable paths for apps to steer!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.20% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title '[V3] Refactor binding transport layer' clearly and concisely summarizes the primary change: refactoring the transport layer for bindings with a focus on Transport abstraction.
Description check ✅ Passed The PR description covers the main changes, motivation, and includes references to related issues, but is missing some checklist items and didn't update the changelog. However, the core content is substantial and mostly complete.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c930250b005a9a3cf61f10d5434740477b95b87 and 747bc95b992c4a8041fd47e6acff8d5e7ddc2483.

⛔ Files ignored due to path filters (1)
  • v3/go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • v3/go.mod (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • v3/go.mod
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Run Go Tests v3 (macos-latest, 1.24)
  • GitHub Check: Run Go Tests v3 (windows-latest, 1.24)
  • GitHub Check: Run Go Tests v3 (ubuntu-latest, 1.24)
  • GitHub Check: semgrep-cloud-platform/scan

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Nov 10 '25 16:11 coderabbitai[bot]

This is an epic PR! I didn't realise you'd take on som many different issues in it 🚀 🎉 Please give me until the end of the weekend to review as it needs a solid look over.

It looks like there's a few merge conflicts. If they could be addressed that would be amazing 🙏

leaanthony avatar Nov 11 '25 11:11 leaanthony

This is looking really good. There was one thing I thought was worth discussing: you know how services can have a ServeHTTP method that can be used by frontend to retrieve files? I'm wondering if the transport interface could have something like Client() string and the runtime should/could fetch /wails/transport at init time to return whatever this method returns (if anything). We use this technique in services to call methods if they have been defined on the service. I was thinking this might be great to allow transports to provide Go config structs that would translate into dynamic frontend clients, like the address or port number could be set on the Go side. Ideally I'd like the developer to not need to worry about copy and pasting frontend transport code into their app.

Thoughts?

leaanthony avatar Nov 17 '25 18:11 leaanthony

@leaanthony oh, I didn't know that. Curious what are use-cases for it in services (why bindings are not enough).

Re Client() string I'm a bit confused by your proposal It returns string, but then later you are saying great to allow transports to provide Go config structs

Should it be then Client() map[string]any ?

Also If I understood your idea correctly, then there is TransportHTTPHandler optional interface, that you can define to provide any possible routes if you need them under default webview assetHandler (and default http uses it). Wouldn't it be enough? As then me as transport developer I can put any routes that I want there, if needed.

For me this is not fully clean I'd like the developer to not need to worry about copy and pasting frontend transport code into their app. If I as a developer implement custom transport, it's highly likely that I will have to implement its parts for both frontend and backend. So not clear what exactly will be copy/pasted. Can you provide expanded example?

APshenkin avatar Nov 17 '25 21:11 APshenkin

Curious what are use-cases for it in services (why bindings are not enough).

Think dynamic assets from v2 (outlined in tutorial here https://v3alpha.wails.io/tutorials/01-creating-a-service/#alternative-approach)

it allows you to have src=/some/route in html elements opposed to query and store then reference. Also allows for incremental implementation if you are migrating an existing web app you can mount your http service and it just works as a web app wrapper

Also If I understood your idea correctly, then there is TransportHTTPHandler optional interface, that you can define to provide any possible routes if you need them under default webview assetHandler (and default http uses it). Wouldn't it be enough? As then me as transport developer I can put any routes that I want there, if needed.

I haven't looked into the code fully I see it was based on my original work, there is a separate path for assets server that likely needs to be HTTP and I think the above comment could just attach to the asset server opposed to bindings transport

If I as a developer implement custom transport, it's highly likely that I will have to implement its parts for both frontend and backend

Agreed custom transport likely entails frontend and backend wiring

atterpac avatar Nov 17 '25 21:11 atterpac

@atterpac yes, this makes sence.

Re

there is a separate path for assets server that likely needs to be HTTP

It's still there. Optional interface AssetServerTransport with ServeAssets method

APshenkin avatar Nov 18 '25 13:11 APshenkin

Yeah, it's hard to explain in text without diagrams but I'll try a use case:

So the websockets example has 2 parts: the Go side and the JS side. I imagine all transports will have both as the frontend needs some kind of logic to call the Go part.

By default, the HTTP transport is used for bindings so the JS side will have e.g. fetch + post etc.

For websockets this code needs changing to connect to a host+socket. This JS code needs running before we can call bindings so during init.

The question is whether we can fetch this JS from the backend during init rather than manually adjusting the frontend code so the host and port aren't defined in JS, but Go.

Let's imagine for a minute that when the runtime starts, and requests /wails/flags it also requests /wails/transport.js. For now, let's say that it assigns this JS file to window.transport and has all the code needed to connect, call, cancel etc over websocket. This is extremely similar to the ipc.js from v2. The runtime startup would be the same regardless of the transport implementation.

Now that we are always requesting this file, in the message handler we can request the result of this call from the Go implementation of the transport, which is what I meant by having a method like JSClient() string which in our example would return the JS websockets code.

This means that bindings transports can be completely self contained in a Go package. Let's take the example further:

import "github.com/org/repo/transports/websockets"

func main() {

// .....
app := application.New(.....
// .....
   Transport: websockets.New(websockets.Confg{
      Host: "1.2.3.4",
      Port: "8080",
      // More config
   },
//.....

Under the hood, the config is set in the implementation, the application starts up, runtime calls /wails/transport.js, message handler calls the interface method to get the JS client string,.returns it to frontend now frontend can connect to the websocket server, using the host and port from the Transport config.

This process would need to be synchronous at the start or we'd need some mechanism on the frontend to wait for that to happen if it's async via a callback or event but we'd need that anyway (what if the server is down?).

I hope that all makes sense!

leaanthony avatar Nov 18 '25 18:11 leaanthony

As for the use cases in Services, think of anything that supports the service. There will be new docs uploaded soon and in that is a QR code generator tutorial. You add the service in Go, choose a route for it, then can request QR codes from the frontend using query Params, e.g. fetch('/services/qr?text=blah'). The speed is incredible. I hope that gives you some inspiration 😉

leaanthony avatar Nov 19 '25 20:11 leaanthony

@leaanthony ok, It seems I got it. Just to double check

You want to add method JSClient() string to transport that literally will return js code that will be returned when call /wails/transport.js ? Then in this code you would like to prepare the transport client and put it somewhere in global state, so that then wails runtime sdk will use this global state for doing calls. Proposal – put transport client in global.transport.

Am I right?

If so, I can try to play around it during the week

APshenkin avatar Nov 19 '25 22:11 APshenkin

I wasn't trying to be prescriptive though. I'm open to how it works, but the idea you could import a transport in Go, add it to your service and it just works sounds like amazing DX 😉 As for global on the frontend, it doesn't have to be, that was just an example. The idea is more that the runtime has a standard interface for making the calls and that gets fulfilled by whatever /wails/transport.js returns, which by default would be the standard http transport. Does that make sense?

leaanthony avatar Nov 20 '25 01:11 leaanthony

@leaanthony I spent some time looking into this option, and my personal opinion is that we should keep things as they are.

Adding JSClient would just introduce another way to solve the same task without providing clear benefits.

We can already pass any “args” or “transport params” through Flags. And if someone wants to serve a client from the transport layer, TransportHTTPHandler already allows adding custom routes.

I experimented with this in a separate branch — you can see the examples in this PR: https://github.com/APshenkin/wails/pull/2/files

Practically speaking, if I were implementing a custom transport for an app, I’d build it together with the app using the same tooling. Creating a separate /wails/transport.js would require an additional build pipeline just to serve that file from /wails/transport.js.

What do you think?

APshenkin avatar Nov 24 '25 23:11 APshenkin

@APshenkin - thanks for taking the time to look at that. I appreciate your efforts 🙏 I think there may be some cross-wires here though. Perhaps it would be easier for me to ask a question: If someone wanted to reuse the websocket transport (or any transport) in their project, how would they do that?

leaanthony avatar Nov 25 '25 10:11 leaanthony

@leaanthony If you want provide transport as a library/package, you can then: Few options:

  1. go module + npm package – might suit for cases, where app fetches static from external resource
  2. go module + serving required js via TransportHTTPHandler + adding await import(/whatever-path-you-specified-in-TransportHTTPHandler.js); to your app

or simplest – copy it in your project

APshenkin avatar Nov 25 '25 10:11 APshenkin

@leaanthony anything that blocks us to merge this PR? Happy to work on it.

if you still want to add JSClient, can do so, as heavy lifting for it is very low.

APshenkin avatar Nov 29 '25 11:11 APshenkin

@APshenkin thanks for your patience. I wanted to try and review the last files. It mainly looks good to me and thanks again for taking on such an epic PR :pray:

The thing I was trying to convey before was whether this code:

const { setTransport } = await import('/wails/runtime.js');

class MyRuntimeTransport {
  call(objectID: number, method: number, windowName: string, args: any): Promise<any> {
    // TODO: implement IPC call with your transport protocol

    return resp;
  }
}

const myTransport = new MyRuntimeTransport();
setTransport(myTransport);

could be automatic instead of something that people will need to copy and paste in. Like this could be in the runtime by default:

let transport = await fetch("/wails/transport")
setTransport(transport)

Which means your transport module would be able to service the frontend dynamically by handling /wails/transport (via message handler and forwarded to the transport implementation), meaning it could be entirely configurable in Go and there would be no need to update JS (most people won't care).

Does this make any sense? It's been a loooong day :sweat_smile:

leaanthony avatar Nov 30 '25 06:11 leaanthony

@leaanthony added. Tell what you think about it

APshenkin avatar Dec 01 '25 17:12 APshenkin

@leaanthony did you had a chance to take a look? Anything that I can help with this PR

APshenkin avatar Dec 06 '25 11:12 APshenkin

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

sonarqubecloud[bot] avatar Dec 07 '25 11:12 sonarqubecloud[bot]

@APshenkin - Thanks for your patience. I'm good for this to go as is. I think there might be some tweaks but that can come later and definitely don't want to block such an epic PR :+1: :pray:

leaanthony avatar Dec 07 '25 11:12 leaanthony