Decimal data type not implemented

[dylangmiles]

My source data in SQL Server has a decimal data type but I am getting a not implemented exception when working with this data. I have put together a minimal example to demonstrate this:

var schema = new StructType(new List<StructField>()
{
    new StructField("id", new IntegerType()),
    new StructField("v", new DecimalType(3,2)),
});

var data = _sparkContext.Parallelize(new List<object[]>()
{
    new object[] {1, 1.43m},
    new object[] {2, 2.18m,},
    new object[] {3, 6.89m},
});

var df = _sqlContext.CreateDataFrame(data, schema);

df.Show(10);

The results I get are:

DataAggregatorService.Test.AssessmentAggregatorTest.Test_Decimal threw exception: 
System.NotImplementedException: The method or operation is not implemented.
    at Microsoft.Spark.CSharp.Sql.DecimalType.get_JsonValue()
   at Microsoft.Spark.CSharp.Sql.StructField.get_JsonValue()
   at Microsoft.Spark.CSharp.Sql.StructType.<>c.<get_JsonValue>b__10_0(StructField f)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Microsoft.Spark.CSharp.Sql.StructType.get_JsonValue()
   at Microsoft.Spark.CSharp.Sql.DataType.get_Json()
   at Microsoft.Spark.CSharp.Sql.StructType.get_StructTypeProxy()
   at Microsoft.Spark.CSharp.Sql.SqlContext.CreateDataFrame(RDD`1 rdd, StructType schema)
   at DataAggregatorService.Test.AssessmentAggregatorTest.Test_Decimal() in C:\_sourceCode2005\CCI\DigiTRACC\DigiTRACC 6.0\DataAggregatorService.Test\AggregatorTest.cs:line 127


[2017-07-24T12:59:10.6157034Z] [ATHENA-DESKTOP] [Info] [SparkContext] Parallelizing 3 items to form RDD in the cluster with 1 partitions
[2017-07-24T12:59:10.7957062Z] [ATHENA-DESKTOP] [Info] [RDD`1] Executing Map operation on RDD (preservesPartitioning=False)

[skaarthik]

Thanks for reporting this issue. We will look into this.