David Beavon

Results 25 comments of David Beavon

@Niharikadutta Did you see that decimals were added to Apache Arrow? - https://issues.apache.org/jira/browse/ARROW-11422 - https://github.com/apache/arrow/pull/9356 This happened recently. Was that a prerequisite? Decimal types are pretty critical for us as...

Is this a short-term goal (ie. is it included in "**1:1 API compatibility for Dataframes**")

+1 here too. I'm ok with using keyvault for secrets but the problem is how to manage the bootstrapping secrets that help you connect to keyvault in the first place.

This has been submitted with Databricks. The Databricks issue tracking (in SF) case number is **00101202** We should get an update from them. I will update this issue once I...

This is an update from Databricks Team following the SF number: 00101202 “We can treat this as a feature request I have already created for this. Here is the request...

@imback82 Is there any reason this group apply would fail the same way on a local windows cluster? I'm running spark-3.0.0-bin-hadoop3.2. I see the same error described above when running...

I did a bit more testing.... as-of now I am not getting the error anymore after upgrading to Microsoft.Spark.Worker.1.1.1 and the new version of nuget (microsoft-spark-3-0_2.12-1.1.1.jar)

I believe that I found the list of "audience" values that are allowed from "GetAccessToken": https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/microsoft-spark-utilities?pivots=programming-language-csharp#get-token ![image](https://user-images.githubusercontent.com/21175830/189434261-ab443b09-d51a-4d7c-8557-77bad10c4611.png) Unfortunately it doesn't appear that this functionality is going to allow me to...

Just an update. I'm still having troubles with SQL connectivity from Spark pools in synapse. The errors I get happen intermittently while using the .Net SqlClient. I'm working on a...

Another update. I changed some of my spark code to use SQL logins "contained users with UID/PWD" and after the change I was **not seeing the same problems with the...