Dain Sundstrom
Dain Sundstrom
BTW should the PR be titled: Migrate *Delta and Iceberg* to Glue v2
I think the root cause here is the query it attempting to load an entire table into a single array: ``` SELECT 'value' AS attribute_type, ARRAY_AGG(DISTINCT value) AS distinct_attributes FROM...
> [@dain](https://github.com/dain) if we switch from byte arrays to Slices (FFM based) we can allocate arrays bigger than 2 GB right? @wendigo that will introduce a different set of issues,...
I know it isn't the same, but Presto does support case insensitive regular expression matching with `(?i)` flag. See https://prestosql.io/docs/current/functions/regexp.html for more information.
The existing `LIKE` is implemented with a regular expression, so if `ILIKE` is added I would expect the same.
We also could just add a function.
> @dain, can you comment on the original intent and abstraction provided by QueryTracker from when you refactored that code a few years ago (if you recall)? Quite simply to...
@wendigo I don't see how this changes "memory copies/materialization"
I assume you are talking about: ``` private void enforceMemoryLimits() { List runningQueries = queryTracker.getAllQueries().stream() .filter(query -> query.getState() == RUNNING) .collect(toImmutableList()); memoryManager.process(runningQueries, this::getQueries); } ``` Which has been updated to:...
> @dain agreed but then on every usage site I'm doing ::stream so what's the point? There are lots of cases where I have collections an only interact with them...