azure-kusto-spark icon indicating copy to clipboard operation
azure-kusto-spark copied to clipboard

Fallback to export mode only if count query fails

Open ag-ramachandran opened this issue 2 years ago • 0 comments

The following is the behavior in case readMode is specified

a) If the user does not specify readMode , a count estimate is done and based on the ADX query limits there is switch to ForceDistributedMode (which uses export) or ForceSingleMode (which uses query directly). In the case that this count estimate query fails (query : Table | count), the fallback is to use ForceDistributedMode

b) If the user specifies the readMode as ForceSingleMode the estimate of the query limits is not performed. If the query fails in single mode , it will still fallback to export mode (ForceDistributedMode)

Expected behavior

The fallback to export mode should not happen in case ForceSingleMode is specified and the query fails.

ag-ramachandran avatar Sep 23 '22 04:09 ag-ramachandran