RFC: `*.edgeql` workflow

[colinhacks]

In addition to the current approach (reflecting EdgeQL and the EdgeDB typesystem into a query builder API), I propose a more GraphQL-like workflow.

*.edgeql files

Queries are written as plain EdgeQL inside *.edgeql files

// ./dbschema/queries/getUser.edgeql
select User {
  name, email
} filter .id = <uuid>$user_id

New npx command

An npx command that scans the project file tree for .edgeql files, reads the contents, and pings the database to retrieve type information about the queries' result type and parameter types.

Candidates:

• npx edgeql-js --generator queries: generates just queries
  • In this case npx edgeql-js --generator qb would generate the query builder and just npx edgeql-js would generate both
• Move all generation code to a separate edbgen repo
  • For *.edgeql files: npx edbgen --generator edgeql
  • For QB: npx edbgen --generator querybuilder
  • Support community-written generators too

For each .edgeql file, a new file is generated with the following structure.

// ./dbschema/queries/getUser.ts
export type getUser = {name: string; email: string;}
export const getUser = {
  edgeql: `select User {
      name, email
    } filter .id = <uuid>$user_id`,
  run: async function(client: Client | Transaction, params: {id: string}){
    // this will be either query, querySingle, queryRequired, or queryRequiredSingle depending on the introspected result cardinality
    return client.querySingle<getUser>(this.edgeql, params);
  },
  runJSON: async function(client: Client | Transaction, params: {id: string}){
    return client.querySingleJSON(this.edgeql, params);
  }
}

The generated query can be consumed like so:

import {createClient} from "edgedb";
import {getUser} from "./dbschema/queries/getUser";

const client = createClient();
await getUser.run(client, {id: "abc..."});
// { name: "Jules", ... }

await getUser.runJSON(client, {id: "abc..."});

Watch mode

There would also be a --watch mode to re-generate whenever a .edgeql file is updated.

Target

The target (.ts vs .js vs .mjs) is still determined using the same resolution algorithm as the current npx edgeql-js command.

Output directory

The output directory can be specified with the --out-dir flag. The default is ./dbschema/queries directory, as resolved relative to the project root. If two query files share the same name, an error will be thrown.

Perhaps there should be a mode that generates the .ts/.js file alongside the corresponding *.edgeql file, wherever it may appear in the project file system. This similar to how tsc works by default; still, it's messy and rarely used (most TS users use outDir) so it shouldn't be our default behavior.

Index file for convenience

For convenience, an index.{j|t}s file should be generated into the dbschema/queries directory that re-exports all generated queries.

./dbschema/query/index.ts
export * from "./getUser";
export * from "./getMovies";
export * from "./searchMovies";

This makes it easier to import all queries with a single import.

import {createClient} from "edgedb";
import * from queries from "./dbschema/queries";

const client = createClient();
await queries.getUser.run(client, {id: "abc..."});

Benefits

• Users can write plain EdgeQL, which is less verbose than the query builder, especially for shapes and operators.
• Users don't need to learn the equivalent query builder syntax for all EdgeQL operations
• Many users are familiar with this workflow from GraphQL
• Massive implementational simplification compared to the query builder: no headaches around TypeScript breakage, excessively deep, inference limitations, etc.

Cons

• You have to run a watcher
• No autocomplete or type checking until LSP lands
• Your queries aren't co-located with your code
• No complex params types. In the query builder we allow arbitrarily complex param types ine.params. The values are automatically serialized to JSON client side, passed as a JSON param, and re-cast in the generated query. This makes it possible to pass complex, strongly-typed objects directly into .run which is a big win for mutations.
• No select * (until it lands in EdgeQL). This isn't implement yet in QB but will be.

[haikyuu]

That's great. Most queries make more sense on the file workflow. And it makes queries independently testable and portable to use in a project with another language

[haikyuu]

No autocomplete or type checking until LSP lands Isn't it possible to introspect the type of a query using edgedb?

[colinhacks]

No autocomplete or type checking until LSP lands

I'm referring to the experience of writing the EdgeQL queries themselves in a *.edgeql file. Currently the IDE extensions we provide do highlighting but no autoformatting or autocompletion (they're not schema-aware either).

But the generated client would certainly be strongly typed.

[1st1]

I like the proposal. +1

Few comments:

npx edgeql-js --generator queries: generates just queries In this case npx edgeql-js --generator qb would generate the query builder and just npx edgeql-js would generate both

+1 for this flow.

There would also be a --watch mode to re-generate whenever a .edgeql file is updated.

Yes, we should also create a simple extension for VSCode eventually.

No complex params types. In the query builder we allow arbitrarily complex param types ine.params. The values are automatically serialized to JSON client side, passed as a JSON param, and re-cast in the generated query. This makes it possible to pass complex, strongly-typed objects directly into .run which is a big win for mutations.

Like passing tuples in?

Index file for convenience

Potentially .edgeql can be in different directories. Will we reflect dir structure in names? If not there can be potential for conflicts.

[colinhacks]

Like passing tuples in?

Exactly. Parameter types would be limited to what EdgeQL actually supports (go figure).

Potentially .edgeql can be in different directories. Will we reflect dir structure in names? If not there can be potential for conflicts.

Oops, meant to clarify this. I think we should flatten all the queries to one level and throw an informative error if there's a naming conflict. Reflecting the dir structure would make imports ugly and verbose. Also, this encourages more descriptive query names, which I think is desirable.

[1st1]

Oops, meant to clarify this. I think we should flatten all the queries to one level and throw an informative error if there's a naming conflict.

Hm, this would make my preferred project organization painful. I like using things like css modules and keeping related css/backend/frontend pieces close. I'd hate if the system forces me to have only globally unique query names. So i'm -1 on this.

[i0bs]

I'm earnestly for this, the proposed workflow here would make parsing and extrapolating the typings for schema infinitesimally easier. The only thing that concerns me, as @1st1 mentioned is flattening queries to one level