How to convert AST to an expression string?

[chetbox]

We have a situation where we're changing the context object in our application to expose more useful data. This means that expressions that our users have written now have to change so we would like to make this change for them.

e.g.

channel.foo

is now

channels.foo.value

To do this I plan to compile the expression, traverse its AST and update the {type: 'Identifier'} objects. How do I then convert this new AST back to an expression string?

[TomFrost]

Hi Chetan! That's a great strategy, but unfortunately Jexl doesn't include a way to de-compile the AST.

[chetbox]

Thanks @TomFrost. This is something we really need so I wrote my own in Typescript.

export type JexlAst =
  | { type: 'UnaryExpression'; operator: string; right: JexlAst }
  | { type: 'BinaryExpression'; operator: string; left: JexlAst; right: JexlAst }
  | { type: 'ConditionalExpression'; test: JexlAst; consequent: JexlAst; alternate: JexlAst }
  | { type: 'FilterExpression'; relative: boolean; expr: JexlAst; subject: JexlAst }
  | { type: 'Literal'; value: string | number | boolean }
  | { type: 'ArrayLiteral'; value: JexlAst[] }
  | { type: 'ObjectLiteral'; value: { [key: string]: JexlAst } }
  | { type: 'Identifier'; value: string; from?: JexlAst; relative?: boolean }
  | { type: 'FunctionCall'; name: string; pool: 'functions' | 'transforms'; args: JexlAst[] };

export function escapeKeyOfExpressionIdentifier(identifier: string, ...keys: string[]): string {
  if (keys.length === 0) {
    return identifier;
  }
  const key = keys[0];
  return escapeKeyOfExpressionIdentifier(
    key.match(/^[A-Za-z_]\w*$/)
      ? `${identifier}.${key}`
      : `${identifier}["${key.replace(/"/g, '\\"')}"]`,
    ...keys.slice(1)
  );
}

function getIdentifier(ast: Extract<JexlAst, { type: 'Identifier' }>): [string, ...string[]];
function getIdentifier(ast: Extract<JexlAst, { type: 'FilterExpression' }>): string[];
function getIdentifier(ast: Extract<JexlAst, { type: 'Identifier' | 'FilterExpression' }>): string[];
function getIdentifier(ast: Extract<JexlAst, { type: 'Identifier' | 'FilterExpression' }>): string[] {
  switch (ast.type) {
    case 'Identifier':
      return [
        ...(ast.from?.type === 'Identifier' || ast.from?.type === 'FilterExpression'
          ? getIdentifier(ast.from)
          : []),
        ast.value
      ];
    case 'FilterExpression':
      if (
        !ast.relative &&
        ast.expr.type === 'Literal' &&
        typeof ast.expr.value == 'string' &&
        ast.subject.type === 'Identifier'
      ) {
        // We are indexing into an object with a string so let's treat `foo["bar"]` just like `foo.bar`
        return [...getIdentifier(ast.subject), ast.expr.value];
      } else {
        return [];
      }
  }
}

export function expressionStringFromAst(ast: JexlAst | null): string {
  if (!ast) {
    return '';
  }

  switch (ast.type) {
    case 'Literal':
      return JSON.stringify(ast.value);
    case 'Identifier':
      return escapeKeyOfExpressionIdentifier(...getIdentifier(ast));
    case 'UnaryExpression':
      return `${ast.operator}${expressionStringFromAst(ast.right)}`;
    case 'BinaryExpression':
      return `${expressionStringFromAst(ast.left)} ${ast.operator} ${expressionStringFromAst(
        ast.right
      )}`;
    case 'ConditionalExpression':
      return `${expressionStringFromAst(ast.test)} ? ${expressionStringFromAst(
        ast.consequent
      )} : ${expressionStringFromAst(ast.alternate)}`;
    case 'ArrayLiteral':
      return `[${ast.value.map(expressionStringFromAst).join(', ')}]`;
    case 'ObjectLiteral':
      return `{ ${Object.entries(ast.value)
        .map(([key, value]) => `${JSON.stringify(key)}: ${expressionStringFromAst(value)}`)
        .join(', ')} }`;
    case 'FilterExpression':
      return `${expressionStringFromAst(ast.subject)}[${
        ast.relative ? '.' : ''
      }${expressionStringFromAst(ast.expr)}]`;

    case 'FunctionCall':
      switch (ast.pool) {
        case 'functions':
          return `${ast.name}(${ast.args.map(expressionStringFromAst).join(', ')})`;
        case 'transforms':
          // Note that transforms always have at least one argument
          // i.e. `a | b` is `b` with one argument of `a`
          return `${expressionStringFromAst(ast.args[0])} | ${ast.name}${
            ast.args.length > 1
              ? `(${ast.args
                  .slice(1)
                  .map(expressionStringFromAst)
                  .join(', ')})`
              : ''
          }`;
      }
  }
}

Would it be useful to make a PR here?

[chetbox]

On further inspection, while this works for simple cases, it doesn't work for more complex expressions because of operator precedence.

For example expressionStringFromAst will give "1 + 2 * 3" for the AST for "(1 + 2) * 3" which evaluate to different expressions. I think the way to do this successfully is using precedence rules in the grammar to surround sub-expressions with brackets where necessary.

Edit: I have a new version that's tested and works nicely if anyone is interested. Comment here and I'll post the latest version or publish it as a library.

[sreuter]

@chetbox Definitely interested!

[chetbox]

The implementation above is not far off but misses some important corner cases which could change the logic of the expression.

I've created a library with a much more robust implementation here: https://www.npmjs.com/package/jexl-to-string

Example:

import { jexlExpressionStringFromAst } from "jexl-to-string";
import { Jexl } from "jexl";

const jexl = new Jexl();
const compiledExpression = jexl.compile(input);
let ast = compiledExpression._getAst();
// Modify `ast` here
const newExpression = jexlExpressionStringFromAst(jexl._grammar, ast);

@sreuter