PoC: Export model as JSON

[thomasrichner-oviva]

trafficstars

As discussed on Discord, here an quick proof of concept to generate schema information in a machine-readable fashion for code generation. This information would allow to at least generate the string constants that otherwise clutter client libraries.

In a next step one could add type-information and permission predicates.

Example Output:

[
  {
    "name": "user",
    "namespace": "default",
    "relations": [],
    "permissions": []
  },
  {
    "name": "platform",
    "namespace": "default",
    "relations": [
      {
        "name": "administrator"
      }
    ],
    "permissions": [
      {
        "name": "super_admin"
      },
      {
        "name": "create_tenant"
      }
    ]
  },
  {
    "name": "tenant",
    "namespace": "default",
    "relations": [
      {
        "name": "platform"
      },
      {
        "name": "parent"
      },
      {
        "name": "administrator"
      },
      {
        "name": "agent"
      },
      {
        "name": "tenant_administrator"
      },
      {
        "name": "admin_administrator"
      }
    ],
    "permissions": [
      {
        "name": "administer_user"
      },
      {
        "name": "create_admin"
      }
    ]
  },
  {
    "name": "administrator",
    "namespace": "default",
    "relations": [
      {
        "name": "self"
      },
      {
        "name": "tenant"
      }
    ],
    "permissions": [
      {
        "name": "write"
      },
      {
        "name": "read"
      }
    ]
  }
]

[github-actions[bot]]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[thomasrichner-oviva]

I have read the CLA Document and I hereby sign the CLA

[josephschorr]

@thomasrichner-oviva Want to continue on this PR?

[thomasrichner-oviva]

@thomasrichner-oviva Want to continue on this PR?

I want to, but probably will take some time. You can close it if you want and I'll rebase & re-open once I get to it again. Thanks for the support, it helped me to understand the underlying model and possibilities.

[josephschorr]

@thomasrichner-oviva Can you expand a bit more on how you were consuming this data?

[thomasrichner-oviva]

@thomasrichner-oviva Can you expand a bit more on how you were consuming this data?

Sure. We write an in-house client library for Zanzibar like permission systems, it adapts calls such as canModifyDocument(long actingUserId, long id) into the appropriate SpiceDB permission checks. In order to do so we need the strings of the names and relationships, i.e. the implementation looks something like:

canModifyDocument(long actingUserId, long id) {
    var check = Check.of(Constant.DOCUMENT, id, Constant.MODIFY_DOCUMENT_PERMISSION, Constant.USER, actingUserId)
   //....
}

If you take that idea further, generating the entire client library should be possible at some point, though just having the constants helps a lot for now. In order to keep the generating code a bit more decoupled from the SpiceDB DSL/implementation, some easy to read AST/definition in JSON makes sense to us.

[josephschorr]

@thomasrichner-oviva Would having a reflection-like API instead of a package suffice? Just trying to weigh various options

[thomasrichner-oviva]

@josephschorr can you elaborate a bit on how that would look/work?

Some ideas from the top of my head:

• provide tooling to spit out the AST of a schema, either a simple solution like the JSON above or maybe even something more standardized like what https://kythe.io/ outputs
• provide a 'stable' AST and its parser as a Go package for consumers, enabling the above without having to decide nor maintain it

There are multiple benefits of producing a machine readable AST for tooling:

• code generation e.g. clients
• custom linting & validation
• code completion etc. for editor

[josephschorr]

@josephschorr can you elaborate a bit on how that would look/work?

We'd implement a reflection package in pkg and then use that as part of a reflection API in gRPC. Callers could either use the gRPC API, or, if they want to include SpiceDB directly, call the pkg directly.

[thomasrichner-oviva]

Ah, I see. Yes that would probably also work.

From a usability perspective I'd say having a cli tool to read the schema and spit out the AST is much more convenient than to start a temporary instance, migrate the schema and then pull it from some API.

From my point of view for a start I'd be happy with a go pkg that I can import with something like a Parse(r io.Reader) (SchemaAst, error) method I can call.

[josephschorr]

From my point of view for a start I'd be happy with a go pkg that I can import with something like a Parse(r io.Reader) (SchemaAst, error) method I can call.

Technically, that already exists: https://github.com/authzed/spicedb/blob/main/pkg/schemadsl/compiler/compiler_test.go#L513

It does, however, return our core types which are subject to change as they are not an officially exported API

[thomasrichner-oviva]

From my point of view for a start I'd be happy with a go pkg that I can import with something like a Parse(r io.Reader) (SchemaAst, error) method I can call.

Technically, that already exists: https://github.com/authzed/spicedb/blob/main/pkg/schemadsl/compiler/compiler_test.go#L513

It does, however, return our core types which are subject to change as they are not an officially exported API

Indeed that's what I used for my PR here ;) https://github.com/authzed/spicedb/pull/657/files#diff-c048079c40c40411bbd1a335a6aca54cf8c610cde628a3b5349b8e7181e487a8R26

if you make that a bit nicer with a 'stable' model that would already be pretty nice. To be honest the next step of writing that in a standardised format (JSON, Kythe, ...) might also be better part of the zed CLI tool instead.

[josephschorr]

if you make that a bit nicer with a 'stable' model that would already be pretty nice. To be honest the next step of writing that in a standardised format (JSON, Kythe, ...) might also be better part of the zed CLI tool instead.

The big open question is around maintainability of the format vs something like a gRPC-based API. I'll have to give this some more thought. Have you investigated Kythe in any detail?

[thomasrichner-oviva]

if you make that a bit nicer with a 'stable' model that would already be pretty nice. To be honest the next step of writing that in a standardised format (JSON, Kythe, ...) might also be better part of the zed CLI tool instead.

The big open question is around maintainability of the format vs something like a gRPC-based API. I'll have to give this some more thought. Have you investigated Kythe in any detail?

Thanks for taking the time and considering it. Reg. Kythe I know that it is Google's attempt to solve exactly that problem, though I have not done a deeper dive yet.

[alsbury]

Any movement on this idea? Being able to generate code based on the schema would be very helpful right now.

[josephschorr]

@thomasrichner-oviva any updates or new thoughts?

[thomasrichner-oviva]

Nothing new from my side, we're currently still writing stuff by hand and have no plans to fix it this quarter.

[josephschorr]

@thomasrichner-oviva Any updates? I think the upcoming reflection APIs remove the need for this, but want confirmation before I close it

[thomasrichner-oviva]

@thomasrichner-oviva Any updates? I think the upcoming reflection APIs remove the need for this, but want confirmation before I close it

Let's close it for now :)