feat: add parser to build

[mathe42]

trafficstars

For a lot of tooling we need a parser. In https://github.com/surrealdb/surrealdb/blob/main/lib/src/sql/parser.rs#L9 we have a parse function (not 100% sure about what it returns). I think we should expose it in this package.

(replacement for https://github.com/surrealdb/surrealdb/issues/1203)

I'm happy to help but 0 experience with rust...

Tooling that depends on a parser

• Language-Server
• IDE-Extensions
• ...

[tobiemh]

Thanks @mathe42 would be good to have an understanding of what kind of things it would need to return...

1. What would be passed in (presuming SQL query)?
2. What would an example response look like (i.e. what could an ideal response look like)?
3. What are examples of how the response would be used (would help me to think about what kind of functionality can be exposed)?

[mathe42]

1. Yes basicly any string (can be potentaly "fksdjflkjdf").
2. Honestly I don't care about the format that much.
3. Main use case is a LanguageServer.

Main goal for this would be (we would also need something like https://github.com/surrealdb/surrealdb/issues/248) to have something like in the picture:

Where you could dynamicly select the namespace you want.

For long and complex queries that would be great!

---

The thing is: I think I can help to write a LanguageServer written in TypeScript. The alternative is to implement it in rust.

Here is a list of LangueServers: https://microsoft.github.io/language-server-protocol/implementors/servers/ And the LSP-Spec https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#languageFeatures

[mathe42]

I got it!

fn parse2(sql: String) {
    let parsedQueryResult = surrealdb::sql::parse(&sql);
    let parsedQuery = parsedQueryResult.unwrap();
    let jsonResult = serde_json::to_string(&parsedQuery);
    let json = jsonResult.unwrap();

    return json;
}

That was the result I wanted. It is not 100% the best output as the way from the AST to the SQL is not defined but I think that is a good starting point!

Example

For

INSERT IGNORE INTO company (name, founded) VALUES ('SurrealDB', '2021-09-10') ON DUPLICATE KEY UPDATE tags += 'developer tools'

you get

[
  {
    "Insert": {
      "into": "company",
      "data": {
        "ValuesExpression": [
          [
            [
              [
                {
                  "Field": "name"
                }
              ],
              "SurrealDB"
            ],
            [
              [
                {
                  "Field": "founded"
                }
              ],
              "2021-09-10T00:00:00Z"
            ]
          ]
        ]
      },
      "ignore": true,
      "update": {
        "UpdateExpression": [
          [
            [
              {
                "Field": "tags"
              }
            ],
            "Inc",
            "developer tools"
          ]
        ]
      },
      "output": null,
      "timeout": null,
      "parallel": false
    }
  }
]

[iDevelopThings]

I got it!

fn parse2(sql: String) {
    let parsedQueryResult = surrealdb::sql::parse(&sql);
    let parsedQuery = parsedQueryResult.unwrap();
    let jsonResult = serde_json::to_string(&parsedQuery);
    let json = jsonResult.unwrap();

    return json;
}

That was the result I wanted. It is not 100% the best output as the way from the AST to the SQL is not defined but I think that is a good starting point!

Example

For

INSERT IGNORE INTO company (name, founded) VALUES ('SurrealDB', '2021-09-10') ON DUPLICATE KEY UPDATE tags += 'developer tools'

you get

[
  {
    "Insert": {
      "into": "company",
      "data": {
        "ValuesExpression": [
          [
            [
              [
                {
                  "Field": "name"
                }
              ],
              "SurrealDB"
            ],
            [
              [
                {
                  "Field": "founded"
                }
              ],
              "2021-09-10T00:00:00Z"
            ]
          ]
        ]
      },
      "ignore": true,
      "update": {
        "UpdateExpression": [
          [
            [
              {
                "Field": "tags"
              }
            ],
            "Inc",
            "developer tools"
          ]
        ]
      },
      "output": null,
      "timeout": null,
      "parallel": false
    }
  }
]

Nice find :D

I was playing/digging a little more just; and I managed to figure how to add a new rpc command, it's so simple but because i know 0 rust I'm so proud 😂

You can dump the AST with your logic and this

src/net/rpc.rs, around line 130, the methods are matched, add this one below

"ast" => match params.take_two() {
	(Value::Strand(s), o) if o.is_none() => rpc.read().await.query_ast(s).await,
	(Value::Strand(s), Value::Object(o)) => rpc.read().await.query_with_ast(s, o).await,
	_ => return Response::failure(id, Failure::INVALID_PARAMS).send(chn).await,
},

then futher down, line 350 ish:

	async fn query_ast(&self, sql: Strand) -> Result<Value, Error> {
		let parsed_query_result = surrealdb::sql::parse(&sql);
		let parsed_query = parsed_query_result.unwrap();
		let json_result = serde_json::to_string(&parsed_query);
		let json = json_result.unwrap();
		debug!(target: LOG, "Executing AST: {}", json);
		Ok(Value::from(json))
	}

	async fn query_with_ast(&self, sql: Strand, mut vars: Object) -> Result<Value, Error> {
		let parsed_query_result = surrealdb::sql::parse(&sql);
		let parsed_query = parsed_query_result.unwrap();
		let json_result = serde_json::to_string(&parsed_query);
		let json = json_result.unwrap();
		debug!(target: LOG, "Executing AST: {}", json);
		Ok(Value::from(json))
	}

Now you can use rpc query :D

Simple additional for local testing, maybe @tobiemh can do a proper safe integration that's written properly in the future 👍

[mathe42]

@iDevelopThings cool. For my use case that is to little information so I will rewrite the parser in typescript...

[tobiemh]

What is the optimum output format @iDevelopThings ?

[iDevelopThings]

Honestly I'm totally okay with that format, it's better than trying to parse segments of the queries with regex 😂 From the few queries i tried too, i think it gave all the information that I'd need

My use case was to parse info from info for db; info for table x; so I could set up a migration system and know about other db/table configurations