spark icon indicating copy to clipboard operation
spark copied to clipboard

Published 20 hours ago • verified publisher icon dotnet

Using custom connectors written in C#

Open sjahongir opened this issue 3 years ago • 1 comments

Hi. I have connectors written in C#. For example, MSSQL Connector, used 'System.Data.SqlClient'. How can I use this connector (instead of JDBC) to read data from the source with spark.net?

sjahongir avatar Mar 06 '21 15:03 sjahongir

Hi @sjahongir, the scenario you described (using C# connector) is not supported, since .NET for Apache Spark is more like a wrapper around Apache Spark. This means that the connector should be written in Java/Scala. Could you explain why the built-in jdbc wouldn't work? https://github.com/dotnet/spark/blob/fe03c7431e174811fa51c84055343bc6e6025b80/src/csharp/Microsoft.Spark/Sql/DataFrameReader.cs#L154-L162

imback82 avatar Mar 08 '21 03:03 imback82