blaze-persistence
blaze-persistence copied to clipboard
Blazebit
Try to get rid of null select items for unfetched attributes
When fetching only a subset of attributes, the select items which select null seem to cause potential problems at a certain point e.g. when we hit ~1000 select items, as that is what Oracle sets as maximum amount of select items for a result set.
A simple workaround is to use any of the non-JOIN fetch strategies, though that limits what can be filtered. I think though, that such models have to be rethought a bit, since the limit could be hit for valid GraphQL requests.
Hi @beikov , Contributing, for Postgres there are 1664 columns (ERROR: target lists can have at most 1664 entries). I have EntityView with Graphql that arrive at these values. I took a look at the ViewTypeObjectBuilder in applySelects
if (attributePath != null && this.fetches.contains(attributePath)) {
mapper.applyMapping(queryBuilder, this.parameterHolder, this.optionalParameters, this.viewJpqlMacro, this.embeddingViewJpqlMacro, false);
} else {
queryBuilder.select("NULL");
}
It seems to me that a possible start would be to remove queryBuilder.select("NULL"); and work with something internal in ObjectBuilder.
Do you already have an idea how to solve it? Maybe I can help with something.
Since the ViewTypeObjectBuilder etc. are only constructed once because they rely on the position of the select items, it is hard to actually omit select items.
I think we can try to implement a mode where we actually create new ViewTypeObjectBuilderTemplate etc. based on the contents of EntityViewSetting#fetches, but not sure how well that would perform.
Another alternative would be to add an indirection layer in the top-level ObjectBuilder chain that provides access to values of a row through methods.
I guess that the indirection is the most viable solution as it still allows caching, but we'd have to change quite a few internal contracts to make this possible. My thought process is, that we create a special EntityViewObjectBuilder interface which doesn't pass a Object[], but rather something like a Tuple. Then we can have two implementations, ArrayTuple and IndirectionArrayTuple.
My approach is to simply omit the null values but remember there positions. I am using a top level ObjectBuilder which calls the bottom most ViewTypeObjectBuilder to inflate the object array from the result list und push the inflated array down to the subsequent builders.
@beikov : Do you see any issues/downsides with that approach?
I would rather move the nullMappings into the InflatingDelegatingObjectBuilder for better separation of concerns, but other than that, this is roughly the solution that I was describing.
On the other hand it's in the responsibility of ViewTypeObjectBuilder to decide how it fetches from data layer. If it thinks it can omit select clauses then it should work around it by himself. I mean this applySelects is almost everything this class is about 😆 . But I need someone to instruct it to pump up the array before accessing any other builder.
Hello, I found the solution very interesting. I understand perfectly. @nlippke nice!
@beikov Is there a possibility to merge? Or how can I test in my environment?
I'll look into this for 1.6.10. I know this is important, so don't worry, I still have this on my radar.
Nice, excellent! Thanks!! I'll wait for 1.6.10, to start using it 100% here in the project.
Hey thanks I'm fine, just returned from a Snowboard trip. Hope you're also doing well. I have no news or plans for this yet, sorry. Ideally, I would like to implement an approach that involves almost no object and array copying which is a slightly bigger undertaking.