otp icon indicating copy to clipboard operation
otp copied to clipboard

`odbc` does not play nice with `sqlite`'s `binary` blobs.

Open fmv1992 opened this issue 3 years ago • 5 comments

Describe the bug odbc does not play nice with binary blobs.

To Reproduce

  1. Create a table in sqlite:

    CREATE TABLE blob_data(
        md5sum text NOT NULL,
        chunk_number int NOT NULL,
        data blob NOT NULL
    );
    
  2. Store a moderately sized blob in it: anything greater than 400 bytes (I suspect 255 is the threshold).

  3. Try to retrieve it with odbc. The first 255 bytes come ok, the rest is just random garbage.

    Curiously, inserting big blobs (≈ 900 megabytes) works ok.

Expected behavior I expected that inserting a big blob and retrieving it would restore the same byte sequence.

Affected versions Erlang/OTP 24 [erts-12.3.1]

Additional context I played with other data types and by using text or varchar I can get that 400 limit to increase, but I'm never able to get past the 10 megabytes mark with any current data types.

fmv1992 avatar Jun 05 '22 02:06 fmv1992

Could this be related to #3321?

garazdawi avatar Jun 07 '22 06:06 garazdawi

The odbc application has not been actively developed for a very long time, and hence is not a high priority for Ericsson. So a PR is more than welcome to address any issues.

IngelaAndin avatar Jun 09 '22 14:06 IngelaAndin

I hope I am not opening a can of worms with this question but would it make sense in this case to move the application out of Erlang/OTP? Perhaps some external team would then pick it up and develop it? Or is the application in a "it just works for our intended use cases" scenario and that's why it has not been developed?

josevalim avatar Jun 09 '22 22:06 josevalim

Yes it is worth considering.

KennethL avatar Jun 10 '22 06:06 KennethL

@josevalim I think we actually have considered it a candidate for that on several occasions, but as it has not caused a lot of work in the past it has tagged along and other changes were prioritized. I think that it is still used for the use case it was developed for (which was not ours, but this is not a story to tell here) and I guess many others use more specialized SQL-server apps as ODBC does not make the backend as transparent as you might have hoped. But if there is someone interested in taking it on please let us know.

IngelaAndin avatar Jun 10 '22 10:06 IngelaAndin