tispark icon indicating copy to clipboard operation
tispark copied to clipboard

[BUG] TiDAGRequest contains wrongResidual Filter

Open shiyuhang0 opened this issue 2 years ago • 1 comments

This pr try to fix it https://github.com/pingcap/tispark/pull/1308. But it still has some problems

Bug execute spark.sql(select * from tidb_catalog.test.t where id > 1).explan:

== Physical Plan ==
*(1) ColumnarToRow
+- TiKV CoprocessorRDD{[table: t] TableScan, Columns: id@LONG, Residual Filter: [id@LONG GREATER_THAN 1], Not(IsNull(id@LONG)), PushDown Filter: [id@LONG GREATER_THAN 1], Not(IsNull(id@LONG)), KeyRange: [([t\200\000\000\000\000\000\000\226_r\000\000\000\000\000\000\000\000], [t\200\000\000\000\000\000\000\226_s\000\000\000\000\000\000\000\000])], Aggregates: , startTs: 433153046377922561}

Expected answer

  • Residual Filter is the filter that pushdown to the coprocessor but can't pushdown to TiKV
  • PushDown Filter is the filter that can pushdown to TiKV The info is wrong for the same value of Residual Filter and PushDown Filter, we need to show the correct info, for this example, Residual Filter should be null

shiyuhang0 avatar Apr 24 '22 09:04 shiyuhang0

/lifecycle frozen

shiyuhang0 avatar Apr 27 '22 10:04 shiyuhang0