usql icon indicating copy to clipboard operation
usql copied to clipboard

Why no support for range partitioning in USQL?

Open ghost opened this issue 8 years ago • 1 comments

As a result of this limitation, I have to create distinct partitions for all the dates plus it does not support high precision datetime datatypes. I ran into a similar limitation with HIVEQL on HDINSIGHT clusters. If you use the same column for partitioning, it overwrites the column data.

I have a 700 million row table which results in 3423 distinct partitions. Trying to create these partitions @pdate1.......@pdate3423 results in a gigantic script.

The USQL job is stuck on "preparing" phase for 3 hrs and then quits without doing anything. SEVERITY Info DESCRIPTION Job was cancelled by Yarn. MESSAGE Job was cancelled. result:CompilationFailed

ghost avatar Jan 02 '18 16:01 ghost

Hi Arvind

Sorry for the late reply here. Can you please file a request for RANGE partition at http://aka.ms/adlfeedback? We have it on our backlog but more customer votes will help in prioritizing the work :).

Thanks Michael

PS: We are working on the other aspects with you now. Please note that the Yarn cancellations had to do with some system issues that we experienced in early January.

MikeRys avatar Feb 09 '18 23:02 MikeRys