How do you export parser Typescript types along with the parser?

[AndrewRayCode]

I'm using peggy to generate a parser that I'm publishing to npm. The project the parser is in uses typescript.

I build the output file from peggy ./dist/parser.js. In my project, I also have ./parser.d.ts, where I give the parser generated by peggy its type:

export function parse(input: string): Program;

I compile typescript also the ./dist folder. My hope was that the parser.js from peggy and the parser.d.ts file from typescript would line up in the export.

However, after an npm publish of package, there is no parser.d.ts. And on import of package/dist/parser in my other project, it complains of no types for the parser.

I'm hoping this is a solved problem and I'm barking up the wrong tree.

[ghost]

I personally rolled back to [email protected], [email protected] Output required a little tweaking (escaped JS code like imports must be moved to the appropriate place)

[klaas1979]

thank you, did this as well.

[GideonMax]

Actually, this is related to typescript. When running tsc, the compiler removes all .d.ts files, as .d.ts files are meant for declarations which are only internally used, if you don't want that to happen you can either: a. use a regular .ts file (which compiles down to .d.ts) b. instead of running tsc, just copy the .d.ts file to the output directory

[AndrewRayCode]

the way I'm currently approaching this is by hand-creating my AST types, and basically hand enforcing they line up with the output of the parser https://github.com/ShaderFrog/glsl-parser/blob/main/src/ast/node.ts

And exporting a typescript definition file for the parser, which includes those nodes in the produced program https://github.com/ShaderFrog/glsl-parser/blob/main/src/parser/parser.d.ts

This sucks in the sense that the type safety could be off if I don't manually line up the parser code with the typescript defs.

Am I doing it wrong? I don't understand if there's a preferred method for doing this

[reverofevil]

The problem is, it can't be solved on peggy side. If you do something like

a = "a" b:b? { return ['a', b]; }
b = "b' a:a? { return ['b', a]; }

types for AST are

type A = ['a', B | undefined]
type B = ['b', A | undefined]

but right hand sides are only possible to extract from TS code with some typeof magic. Unfortunately, TS doesn't properly support recursive types. That typeof requires knowing the whole instantiation tree of its argument, but given the type is recursive all you get is "instantiation is possibly infinite" error message.

There's not many ways to fix it:

• use a programming language that supports proper sound type inference unlike TypeScript (there is no such languages that even resemble JS);
• instead of allowing arbitrary code in grammars (and generating AST) restrict parser language in a way to generate ST:

@a = "a" b:b? // here @ means "this rule has to create a node in ST"
@b = "b" a:a?

->

type A = {type: 'a', b: B | undefined}
type B = {type: 'b', a: A | undefined}

Neither thing is possible to do in peggy while keeping backwards compatibility. You're better off implementing own parser generator. The thing might looks something like this.

[siefkenj]

I work around part of this issue by importing and re-exporting my parser object in typescript. Basically something like

// @ts-nocheck
import _Parser from "grammar.peggy"
import ParserType from "types.ts"
export const Parser = _Parser as ParserType

I use esbuild to autogenerate the parser source and bundle it; Typescript happily creates the corresponding types.

[hildjj]

here are a few options (assuming the output of peggy is grammar.js):

1. You can use ts-pegjs, which you may or may not find worth the extra complexity.
2. You can add "allowJs": true to your tsconfig.json file, and put grammar.js in your typescript source directory. This will generate a rudimentary .d.ts file that liberally uses any.
3. Create a grammar.d.ts file next to grammar.js that pulls out the relevant bits from https://github.com/peggyjs/peggy/blob/main/lib/peg.d.ts

I just wrote a quick example of 3) here: https://gist.github.com/hildjj/0643e6b75d1ef6a83cf27531cdc3040f

It is completely reasonable for you to want this to be easier, so I'm going to leave this open as a feature request. Will think about it after the upcoming release.

[hildjj]

One possible approach if someone takes this on:

• Make sure that the types in peg.d.ts are appropriate for generated parsers
• Emit TypJSDoc comments that use import('peggy')

[siefkenj]

I have been working on this issue, particularly the issues that @polkovnikov-ph brought up and I have a working prototype that automatically generates typescript types from a Peggy grammar.

Source is here: https://github.com/siefkenj/peggy-to-ts

Playground is here: https://siefkenj.github.io/peggy-to-ts

I would be happy to modify this code as needed if the Peggy maintainers were interested in integrating it.

[somebody1234]

@reverofevil this is a tangent, but: note that flow is (mostly) sound and heavily resembles JavaScript. hegel is fully, 100% sound, and also heavily resembles javascript.

(this is ignoring the swath of languages that compile to JS but don't resemble it)