pg_query.rs icon indicating copy to clipboard operation
pg_query.rs copied to clipboard

Support parsing complex queries with deeply nested ASTs

Open seanlinsley opened this issue 2 years ago • 5 comments
trafficstars

This matches the same behavior in the Ruby gem: https://github.com/pganalyze/pg_query/pull/238

Upstream changes were needed in the prost crate that haven't been merged yet: https://github.com/tokio-rs/prost/pull/785

Without further changes, the test suite would run into a stack overflow. There are some ways around this:

  • Create a thread with a larger stack size. This is what the PR currently does b/c it didn't require adding a dependency. But its performance is limited by it creating a new thread for every call. There is also safety concerns with passing unsafe data structures to a different thread.
  • Use the heap to dynamically grow the stack size https://crates.io/crates/stacker. This is a fairly popular crate, but I'm wary of the maybe_grow performance. If someone hasn't put together benchmarks we may want to use grow instead.
  • https://crates.io/crates/decurse
  • https://crates.io/crates/tailcall

seanlinsley avatar Jan 11 '23 15:01 seanlinsley

Since any solution to the stack overflow issue would involve pulling in dependencies that most users won't need, I decided to document this issue in the README so downstream applications can increase the stack size as needed for their application.

seanlinsley avatar Jan 13 '23 20:01 seanlinsley