flatbuffers icon indicating copy to clipboard operation
flatbuffers copied to clipboard

Int64 not work on web (flutter 2.5.3 sdk) [macos, FlatBuffers 2.0.0-dev.1]

Open bajizhh opened this issue 3 years ago • 10 comments

int64/uint64 unimplemented in dart2js, UnsupportedError threws in typed_data_dart2js.dart

void setInt64(int byteOffset, int value, [Endianness endian=Endianness.BIG_ENDIAN]) {
    throw new UnsupportedError("Int64 accessor not supported by dart2js.");
}

but protobuf supports int64.

any plan to fix this?

bajizhh avatar Dec 07 '21 10:12 bajizhh

@vaind @dnfield

aardappel avatar Dec 09 '21 18:12 aardappel

This is a fundamental limitation on JS. I don't think we should try to fix this or work around it in flatbuffers. I'm a bit curious about what the JS implemenation of flatbuffers does though.

dnfield avatar Dec 09 '21 18:12 dnfield

The issue is, as you can see from where the error is thrown, that JavaScript doesn't support 64-bit integers, or rather no higher than 2^53 - 1

From a quick search, protobuf may be using Int64 class.

The options I see how 64-bits could be supported for dart2js:

  • use some kind of wrapper class, e.g. the above-mentioned fixnum Int64
  • limit the read and written numbers (in JS) to the range that is representable in JS, throwing if it would overflow

vaind avatar Dec 09 '21 18:12 vaind

I'm a bit curious about what the JS implemenation of flatbuffers does though.

wrapper class

https://github.com/google/flatbuffers/blob/d0cede9c90c5257537c293517a21376408b549fa/ts/builder.ts#L580_L582 https://github.com/google/flatbuffers/blob/master/ts/long.ts

    createLong(low: number, high: number): Long {
      return Long.create(low, high);
    }

vaind avatar Dec 09 '21 18:12 vaind

Besides, I wonder why dart SDK throws on setInt64 - that should be safe to implement, right? We obviously have an integer that is in the limited range that JS can represent so why not just write it 🤔

vaind avatar Dec 09 '21 19:12 vaind

Not sure what the right time is to start leaning on BigInts.. but they've been in JS for a while now.

aardappel avatar Dec 09 '21 19:12 aardappel

Not sure what the right time is to start leaning on BigInts.. but they've been in JS for a while now.

Hmm, dart already does have those in the core... https://api.dart.dev/stable/2.6.1/dart-core/BigInt-class.html - that would be a good option (probably FB generation-time switchable, we don't want to reduce performance for non-web users)

vaind avatar Dec 09 '21 19:12 vaind

Dart used to transparently convert numbers to BigInts but stopped doing that in Dart 2.

I'm not really a dart2js expert. I don't know if that is unimplemented on typed data because no one has asked for it, or because it's been deemed harmful for some reason. At any rate, it would be better to have it fixed upstream in Dart and make it work here, or to just not have it work here for the same reasons it doesn't work in Dart.

dnfield avatar Dec 09 '21 21:12 dnfield

Any known workarounds? I have tons of flatbuffer files with Int64 written values inside them and would like to read it in flutter web. I would like to avoid turning it into json files server side.

srmncnk avatar Aug 18 '22 12:08 srmncnk

fixnum? I think this is how protobuf achieves identical int64 behaviors on VM and Web.

The only concern I have: at the end of the page https://dart.dev/guides/language/numbers, there's a small note that says: "Use these types with care, though: they often result in significantly bigger and slower code." I'm not sure if it's referring to the BigInt class from dart:core or Int64 class from the fixnum package.

GZGavinZhao avatar Aug 25 '22 00:08 GZGavinZhao

This issue is stale because it has been open 6 months with no activity. Please comment or label not-stale, or this will be closed in 14 days.

github-actions[bot] avatar Mar 04 '23 01:03 github-actions[bot]

This issue was automatically closed due to no activity for 6 months plus the 14 day notice period.

github-actions[bot] avatar Mar 18 '23 20:03 github-actions[bot]

Hello, any news on this?

In my flutter app I'm trying to serialize a Dart object that has a Int64 field:

  Uint8List _serializeObject(Object object) {
    var fbb = new fb.Builder();
    fbb.startTable(2);
    fbb.addInt64(0, object.myField, null);  // <- fails
    // ...
    fbb.finish(fbb.endTable());
    return fbb.buffer;
  }

When running it, I get the runtime error:

Unsupported operation: Int64 accessor not supported by dart2js.

dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 296:3       throw_
dart-sdk/lib/_internal/js_dev_runtime/private/native_typed_data.dart 608:5        setInt64]
packages/flat_buffers/flat_buffers.dart 822:12                                    [_setInt64AtTail]
packages/flat_buffers/flat_buffers.dart 298:7                                     addInt64
...

My goal is to have object.myField to be a 64-bit int also in Javascript, and to be serialized as such. Now it's an int and afaiu int are 64-bit floating points in the browser, leading to a (2^53-1) valid integer range.

For this reason, let's say it's okay for me to have object.myField be capped to 32-bit. How can I use flatbuffers API to serialize 2 uint32? One for padding, and the other one with a valid value? This to mock up a fake uint64 value.

loryruta avatar Jul 05 '24 11:07 loryruta