Jay Narale comments

Results 78 comments of


                                            Jay Narale

[Velox Prestissimo] NULLs corrupted in VARCHAR and BIGINT columns at scale

https://github.com/facebookincubator/velox/pull/7411/files#r1383775458 Even when the original author added using cached nulls there was some uncertainty.

build(parquet): Use FBThrift instead of Apache Thrift

Parquet reader and writer use some arrow functionality in velox, would that part of arrow also be compatible?

build(parquet): Use FBThrift instead of Apache Thrift

I mean arrow internally uses apache thrift ( atleast for parquet )so will that flow be compatible with fbthrift in the current arrow version? Also what about future arrow upgrades...

[WIP] Feat: Add scan tracing events when run with datafusion ffi

cc: @westonpace When I run with DataFusion now, I see logs showing metrics like bytes read and I/O. Just wanted to check if this is the right approach. (By wrapping...

[WIP] Feat: Add scan tracing events when run with datafusion ffi

Updated my PR, now the ouput i get is below for eg: With traces i am unable to get this (via datafusion) ``` === FilteredReadExec Analysis === FilteredReadExec Analysis: I/O...

[WIP] Feat: Add scan tracing events when run with datafusion ffi

It would be great if we could somehow populate this to datafusion explain analyze/ new tracer

[WIP] Feat: Add scan tracing events when run with datafusion ffi

Nope unfortunately ( i tested after cherry picking). The problem is that datafusion is bypassing that exec() call. > let plan = scan.create_plan().await.map_err(DataFusionError::from)?; > > // Wrap the plan with...

[WIP] Feat: Add scan tracing events when run with datafusion ffi

Thanks a lot! Makes sense i will clean it up and add a tracer.