tree-sitter-sql icon indicating copy to clipboard operation
tree-sitter-sql copied to clipboard

Errors when embedding Presto inside of Python strings

Open tomspeak opened this issue 1 year ago • 4 comments

Almost all of the SQL I write is inside of python data pipelines, using Spark and Presto flavoured syntax.

So my queries are written like

SqlQuery(
  select="""
    WITH users_in_threads AS (
          SELECT
              id,
          FROM {THREAD_TABLE}
          CROSS JOIN UNNEST(userid_array) AS t (userid)
      )
    SELECT
        userid,
        MAP_AGG(id, name) AS names,
        COUNT() AS cnt
    FROM {USERS_TABLE}
    GROUP BY
        1
  """
)

When using tree-sitter-sql via neovim, I extended it to also capture the content inside of these strings via an injection.

; extends

(string 
  (string_content) @injection.content
    (#vim-match? @injection.content "^\w*SELECT|FROM|INNER|JOIN|UNION|WHERE|CREATE|DROP|INSERT|UPDATE|ALTER|WITH.*$")
    (#set! injection.language "sql"))

This "works", using :InspectTree I can see it correctly captures SELECT/WITH statement, but once it gets to the python string interpolation FROM {THREAD_TABLE}, the parser errors.

This is quite a crazy use-case, but wondered if there is anything that can be done to make the parser looser in this context so it at least does not error out?

tomspeak avatar Oct 11 '24 11:10 tomspeak