incubator-gluten icon indicating copy to clipboard operation
incubator-gluten copied to clipboard
verified publisher icon apache

[VL][1.2] Type mismatch error in VectorHasher

Open wForget opened this issue 1 year ago • 4 comments

Backend

VL (Velox)

Bug description

Type mismatch error in VectorHasher.

Spark version

Spark-3.5.x

Spark configurations

No response

System information

No response

Relevant logs

org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Type mismatch: VARCHAR vs. VARBINARY
Retriable: False
Expression: type_->kindEquals(vector.type())
Context: Operator: PartialAggregation[1] 1
Function: decode
File: /data/workspace/gluten-deploy-dist/ep/build-velox/build/velox_ep/./velox/exec/VectorHasher.h
Line: 168
Stack trace:
# 0  _ZN8facebook5velox7process10StackTraceC1Ei
# 1  _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_
# 3  _ZN8facebook5velox4exec13BaseHashTable20prepareForGroupProbeERNS1_10HashLookupERKSt10shared_ptrINS0_9RowVectorEERNS0_17SelectivityVectorEba
# 4  _ZN8facebook5velox4exec11GroupingSet21addInputForActiveRowsERKSt10shared_ptrINS0_9RowVectorEEb
# 5  _ZN8facebook5velox4exec11GroupingSet8addInputERKSt10shared_ptrINS0_9RowVectorEEb
# 6  _ZN8facebook5velox4exec15HashAggregation8addInputESt10shared_ptrINS0_9RowVectorEE
# 7  _ZN8facebook5velox4exec6Driver11runInternalERSt10shared_ptrIS2_ERS3_INS1_13BlockingStateEERS3_INS0_9RowVectorEE
# 8  _ZN8facebook5velox4exec6Driver4nextERSt10shared_ptrINS1_13BlockingStateEE
# 9  _ZN8facebook5velox4exec4Task4nextEPN5folly10SemiFutureINS3_4UnitEEE
# 10 _ZN6gluten24WholeStageResultIterator4nextEv
# 11 Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeHasNext
# 12 0x00007ff82ebb76c7

	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
	at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
	... 33 more

	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
	at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
	... 21 more

wForget avatar Oct 23 '24 06:10 wForget

Reproduce:

create table t1 (a string) stored as parquet;

-- insert a binary value to t1 location
INSERT OVERWRITE DIRECTORY 'hdfs://XXXXX/t1' USING parquet select cast('test' as binary) as a;

-- got type mismatch error
select count(1) from t1 group by a;

wForget avatar Oct 23 '24 06:10 wForget

hello,i met task error,but not print Stack trace as you like, had any config i miss? it print like

Retriable: False
Function: runInternal
File: /data/app/gluten/ep/build-velox/build/velox_ep/velox/exec/Driver.cpp
Line: 742
Stack trace:
# 0  
# 1  
# 2  
# 3  
# 4  
# 5  
# 6  
# 7  
# 8  

	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method)
	at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNextInternal(ColumnarBatchOutIterator.java:61)
	at org.apache.gluten.vectorized.GeneralOutIterator.hasNext(GeneralOutIterator.java:37)
	... 46 more

thx

LoseYSelf avatar Oct 24 '24 09:10 LoseYSelf

@LoseYSelf Are you using gluten 1.2.0? Maybe you need port #7038

wForget avatar Oct 24 '24 09:10 wForget

@LoseYSelf Are you using gluten 1.2.0? Maybe you need port #7038

thx

LoseYSelf avatar Oct 24 '24 09:10 LoseYSelf

This issue may be related to https://github.com/facebookincubator/velox/pull/10399

wForget avatar Apr 08 '25 06:04 wForget

Similar error:

Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Operator::getOutput failed for [operator: ValueStream, plan node ID: 0]: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: (VARBINARY vs. VARCHAR)
Retriable: False
Expression: BaseVector::typeKind() == source->typeKind()
Context: 0:VARCHAR
Additional Context: Top-level Expression: not(equalto(try_cast((coalesce(vvauto, 0:VARCHAR)) as DOUBLE), 4:DOUBLE)) Operator: TableScan[0] 0 Operator: ValueStream[0] 0
Function: copyValuesAndNulls
File: /data/workspace/gluten-deploy-dist/ep/build-velox/build/velox_ep/./velox/vector/FlatVector-inl.h
Line: 143
Stack trace:
# 0  _ZN8facebook5velox7process10StackTraceC1Ei
# 1  _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 2  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_
# 3  _ZN8facebook5velox10FlatVectorINS0_10StringViewEE18copyValuesAndNullsEPKNS0_10BaseVectorERKNS0_17SelectivityVectorEPKi
# 4  _ZN8facebook5velox10FlatVectorINS0_10StringViewEE4copyEPKNS0_10BaseVectorERKNS0_17SelectivityVectorEPKi
# 5  _ZN8facebook5velox4exec12ConstantExpr15evalSpecialFormERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 6  _ZN8facebook5velox4exec4Expr24evalSpecialFormWithStatsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 7  _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 8  _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 9  _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 10 _ZN8facebook5velox4exec12CoalesceExpr15evalSpecialFormERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 11 _ZN8facebook5velox4exec4Expr24evalSpecialFormWithStatsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 12 _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 13 _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 14 _ZN8facebook5velox4exec4Expr13evalWithNullsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 15 _ZN8facebook5velox4exec4Expr13evalEncodingsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 16 _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 17 _ZN8facebook5velox4exec8CastExpr15evalSpecialFormERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 18 _ZN8facebook5velox4exec4Expr24evalSpecialFormWithStatsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 19 _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 20 _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 21 _ZN8facebook5velox4exec4Expr13evalWithNullsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 22 _ZN8facebook5velox4exec4Expr13evalEncodingsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 23 _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 24 _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 25 _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 26 _ZN8facebook5velox4exec4Expr13evalWithNullsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 27 _ZN8facebook5velox4exec4Expr13evalEncodingsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 28 _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 29 _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 30 _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 31 _ZN8facebook5velox4exec4Expr13evalWithNullsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 32 _ZN8facebook5velox4exec4Expr13evalEncodingsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 33 _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 34 _ZN8facebook5velox4exec7ExprSet4evalEiibRKNS0_17SelectivityVectorERNS1_7EvalCtxERSt6vectorISt10shared_ptrINS0_10BaseVectorEESaISB_EE
# 35 _ZN8facebook5velox4exec25SimpleExpressionEvaluator8evaluateEPNS1_7ExprSetERKNS0_17SelectivityVectorERKNS0_9RowVectorERSt10shared_ptrINS0_10BaseVectorEE
# 36 _ZN8facebook5velox9connector4hive14HiveDataSource23evaluateRemainingFilterERSt10shared_ptrINS0_9RowVectorEE
# 37 _ZN8facebook5velox9connector4hive14HiveDataSource4nextEmRN5folly10SemiFutureINS4_4UnitEEE
# 38 _ZN8facebook5velox4exec9TableScan9getOutputEv
# 39 _ZN8facebook5velox4exec6Driver11runInternalERSt10shared_ptrIS2_ERS3_INS1_13BlockingStateEERS3_INS0_9RowVectorEE
# 40 _ZN8facebook5velox4exec6Driver4nextEPN5folly10SemiFutureINS3_4UnitEEE
# 41 _ZN8facebook5velox4exec4Task4nextEPN5folly10SemiFutureINS3_4UnitEEE
# 42 _ZN6gluten24WholeStageResultIterator4nextEv
# 43 Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeHasNext
# 44 0x00007ff0477b76c7

wForget avatar Apr 15 '25 05:04 wForget