Duo Zhang
Duo Zhang
@ndimiduk This is the PR for changing release target to 17. PTAL. Thanks.
This problem only affects branch-2.x?
Mind explaining a bit? Thanks.
Ping @virajjasani
The PR is really big... Is it possible to contribute it with several smaller pieces so it is easier for us to review it? Thanks.
OK, so the problem is that, we do not store limit in the serialized scan object, so when deserializing, we can not get the limit. But the design of the...
> Thanks for the comment. > > > OK, so the problem is that, we do not store limit in the serialized scan object, so when deserializing, we can not...
Oh, there are some formatting problem with my reply so maybe you missed part of it... As I explained above, scan limit is a global limit, under parallel scenario a...
> Thanks, that is one option. But can we update the documentation of `Scan#setLimit` to say: > > > ... When used with `TableInputFormat` or `TableSnapshotInputFormat`, this limit is applied...
But we already have a rowsLimitPerSplit for TableSnapshotInputFormat right? So aligning the logic with TableInputFormat and TableSnapshotInputFormat is natural, and we also do not need to care about the serialization...