usql

usql copied to clipboard

Hi, In my application, I am trying to read a file from data lake store. I read a file using

DataLakeStoreFileSystemManagementClient _adlsFileSystemClient;
Stream str = _adlsFileSystemClient.FileSystem.Open("adl1", "//TestETL//TestData.tsv");

I observed that, for DataLakeStoreFileSystemManagementClient to read a file, it requires Newtonsoft.Json version 6.0.0.0. Where as to login to Azure UserTokenProvider.LoginSilentAsync depends on Newtonsoft.Json version 9.0.0.0 or greater. In my application, I am using Newtonsoft.Json version 9.0.0.0.

When I create an assembly of my application and try to use it in USQL job, the job fails with error

This shows that USQL needs Newtonsoft.Json version 6.0.0.0 to read a file from DLS whereas Microsoft.Rest.Azure.Authentication depends on Newtonsoft.Json version 9.0.0.0 or greater.

If I use Newtonsoft.Json version 6.0.0.0 then my authentication fails.

I am stuck in this. Any solution / work around appreciated.

Thanks, Lokesh.

The same problem happens with the included DataFormats example found here: https://github.com/Azure/usql/tree/master/Examples/DataFormats

The project references Newtonsoft.Json 8.0.0.0, but Microsoft.Hadoop.Avro which is included references Newtonsoft.Json 6.0.0.0 so when I try to run it I get the same error as Lokesh.

This is what AsmSpy shows:

Reference: Newtonsoft.Json
  Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed
  Source: NotFound
    6.0.0.0 by Microsoft.Hadoop.Avro, Version=1.5.6.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
  Newtonsoft.Json, Version=8.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed
  Source: Local, Location: C:\Users\blomqvist\Downloads\AsmSpy\Newtonsoft.Json.dll
    8.0.0.0 by Microsoft.Analytics.Samples.Formats, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null

You need to downgrade to JSON 6.0, if you have nuget, change the version to older one, 2015-ish...

Yes, I should have mentioned that is exactly the workaround I used. Now my packages.config in the DataFormats example looks like this:

<packages>
  <package id="Microsoft.Hadoop.Avro" version="1.5.6" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="6.0.8" targetFramework="net45" />
</packages>

I had to downgrade the Newtonsoft.Json package to 6.0.7, because 6.0.8 was not making the assembly to recognize the 6.0.0.0.0 version 👎