Ken, Wang

icon indicating an email [email protected]

icon company EBAY icon location SHANGHAI

Results 66 comments of Ken, Wang

[Ballista] Streaming style push-based shuffle and All-at-once stage scheduling in Ballista

> Hi, @mingmwang , nice work! > > You can open an RFC ticket if you are willing to, then we can discuss based on github. After we get the...

[Ballista] Streaming style push-based shuffle and All-at-once stage scheduling in Ballista

@heroWang I will start work on this maybe next month. Recently I'm busy with other stuff and do not get any bandwidth on DataFusion and Ballista

Improve benchmark performance

👍

Improve benchmark performance

I'm working on partition reasoning now which could help to avoid unnecessary shuffle. https://github.com/apache/arrow-ballista/issues/284

Improve benchmark performance

If any issues/gaps are identified in the physical planning phase, you can assign those issue to me and I can take care.

Offload listing files and inferring stats workload to Executors for large tables(>10K+ files)

@andygrove @yahoNanJing @Dandandan Please share your thoughts on this.