[board-server] Refactor board-server to use express

[wfaithfull]

The current node:http based request parser pattern that the board server uses has some disadvantages.

• Makes it quite hard for an outside observer to understand the available routes on the API, though this has been mitigated by @TinaNikou's work on the openapi spec, thanks Tina!
• Makes it very difficult to automatically generate an openapi spec from the code, so exposes us to the danger of the openapi spec drifting out of line with the actual API

I'm proposing that we reorganise the board server request handling code into straightforward handler-per-route style so that we can automatically generate the openapi spec from these and so that the request handling code is a bit more approachable for the uninitiated. I'd like run the board server on express so that we can take advantage of the routing layer semantics e.g.

app.get('/', (req, res) => {
  res.send('Hello World!')
})

This would let us organise each route with a function and integrate automatic generation of the openapi spec. Express, as far as I know, is just about the lightest HTTP server after node:http.

The 1000ft view

I want to drastically decrease the friction to get folks deploying their own board servers and integrating them into their applications. Therefore;

• [x] I want a self-contained containerised board server
• [ ] I want some really simple multi-language client SDKs so people don't have to even think about the board server API

Therefore;

• [ ] I want these client SDKs to be automatically generated from an openAPI spec

Therefore;

• [ ] I want to be able to automatically generate an openAPI spec for the board server API from the request handler functions / annotations

Therefore;

• [ ] I want the board server API to be organised as request handler functions with the routing layer abstracted

[wfaithfull]

@dglazkov would like to hear your thoughts on this

[dglazkov]

Can you help me understand why we want the SDK for the board server API? Most of it (aside from the /boards endpoint) is more like internal guts that shouldn't be super-interesting and is still in flux.

[dglazkov]

Overall, the idea of refactoring the routing to use something more robust SGTM.

[dglazkov]

@timswanson-google would love your thoughts, too

[wfaithfull]

Most of it (aside from the /boards endpoint) is more like internal guts that shouldn't be super-interesting and is still in flux.

I think this is great point, actually. What I really want to wrap up in a client SDK is primarily the complexity of the run API, along with a few other bits to make it really easy to invoke and run boards remotely. I currently have all this plumbing in the integration tests to call the run api - adapted from your code for testing it locally. I'd like to box this up into something that is very slick to bring into your own server-side application and start running boards.

const scriptedRun = async (
  boardName: string,
  script: { inputs?: Record<string, any>; expected: ExpectedResult[] }[],
  apiKey: string
) => {
  let next;
  for (const [index, { inputs, expected }] of script.entries()) {
    const inputData = {
      ...inputs,
      $key: apiKey,
      ...(next ? { $next: next } : {}),
    };

    const { statusCode, data: body } = await makeRequest({
      path: `/boards/@${account.account}/${boardName}.api/run`,
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: inputData,
    });

    assert.strictEqual(statusCode, 200);

    const events = body
      .split("\n\n")
      .filter(Boolean)
      .map((event) => {
        const jsonStr = event.replace("data: ", "");
        return JSON.parse(jsonStr);
      });

    assertResults(events, expected, index);
    next = getNext(events[events.length - 1]);
  }
};

[timswanson-google]

Disclaimer: this is a very uninformed opinion. I've never done any real JS server work

I think we should do this. I've also found the server code hard to parse.

When I first looked at the server code for Board Server, I found it a bit hard to read, but I feel like that's mostly because I was used to "path-first" server frameworks that use things like annotations or decorators to define routes (e.g. Bottle for Python, or net/http for Go 1.22+).

Express roughly follows this same pattern.

https://expressjs.com/en/starter/hello-world.html

So using Express for routing would have some benefits right away as far as cleaning up of the code. We could also replace some of our manual "middleware" layers for things like CORS with the Express counterparts that do the same thing.

https://expressjs.com/id/resources/middleware/cors.html

[timswanson-google]

Can you help me understand why we want the SDK for the board server API? Most of it (aside from the /boards endpoint) is more like internal guts that shouldn't be super-interesting and is still in flux.

I don't feel like this is a reason not to use a server framework. Ultimately, having something that's easy to understand and change is valuable especially for something that's in flux. And the code will always be interesting to someone who's trying to change it.

[dglazkov]

SGTM.

[wfaithfull]

Disclaimer: this is a very uninformed opinion. I've never done any real JS server work

I think we should do this. I've also found the server code hard to parse.

When I first looked at the server code for Board Server, I found it a bit hard to read, but I feel like that's mostly because I was used to "path-first" server frameworks that use things like annotations or decorators to define routes (e.g. Bottle for Python, or net/http for Go 1.22+).

Express roughly follows this same pattern.

https://expressjs.com/en/starter/hello-world.html

So using Express for routing would have some benefits right away as far as cleaning up of the code. We could also replace some of our manual "middleware" layers for things like CORS with the Express counterparts that do the same thing.

https://expressjs.com/id/resources/middleware/cors.html

I agree with this @timswanson-google. The thing that gets me is that currently we have leaky abstraction layers in the code. We are passing requests all the way down the call stack and responses all the way back up. There are examples at the moment where we are doing authentication-y bits as deep down as the database I/O code, which I think isn't a great separation of concerns - ideally we do auth in a filter-style pattern for example.

I think express is quite a light touch framework to manage the plumbing of our separation of concerns in this manner, and it should make the request handling code a lot cleaner and more approachable.

[wfaithfull]

@bertiespell and I tentatively started on this yesterday as a bit of a pathfinder to feel out how hard it's going to be. It feels achievable but it is definitely a substantial undertaking that may be better approached in backwards compatible stages given the pace of change.

[dglazkov]

Heads up, https://github.com/breadboard-ai/breadboard/pull/3412 adds a /me API endpoint.

[timswanson-google]

Hi Will, I think I'm going to pick this up as my primary focus. Please reach out with any questions or concerns.

The team has decided to move forward with conversion to Express, immediate priority. We're going to change in place, without any switching or flag control.

Scope is both board server and connection server. I think we're going to start with connection server, since it has a smaller API footprint.

@wfaithfull @dglazkov this will be my first exposure to Express, and overall I haven't done a lot of server-side Typescript in general. I might have some questions.

[wfaithfull]

Great! I don't pretend to have huge experience in using express or server-side typescript but I have a lot of proximate experience working around folks who do this day in day out.

Just relaying mine and @timswanson-google's conversation in discord here - #3340 introduced a parallel express version of the board server. This passes the current suite of integration tests and should be a good starting point. I think the tests I wrote still leave a bit to be desired, not least because they depend on one another, but also because they don't test the proxy endpoint.

However, a repeatable pattern is there.