google
[Dart] Deserializing an image requires copying the bytes - fb 23.5.26 / dart 3.2.3 / macos 13.6.1
- flatc 23.5.26
- Flatbuffers for Dart (flat_buffers) 23.5.26
- Dart SDK 3.2.3 / Flutter 3.16.3
- macOS Ventura 13.6.1
Context: Hi I'm making a Flutter app and need downloadable resource bundles. Flutter's Asset Bundles are supposedly platform-specific (i.e. requires building different bundles for Android vs iOS), so I'm trying to use Flatbuffers as a cross-platform bundle format. Could use Protobufs, but zero-copy deserialization was attractive, so I'm looking at Flatbuffers and Cap'n Proto. Each asset bundles will contain not just structs of basetypes but also several images.
Problem:
I can't seem to be able to access the image bytes as Uint8List. The only available accessor is a List<int> which I can't cast to a Uint8List, so it requires copying using Uint8List.fromList().
Here is all the code for the whole project:
fbuf_image.fbs - To keep it simple, let's say for now it's just a string and an image
table FbufImage {
name: string;
image1: [ubyte];
}
root_type FbufImage;
img2buf.py - This makes a flatbuffer file called bundle1.fbuf using a test.png in the current folder
import flatbuffers
import FbufImage # generated by 'flatc --python fbuf_image.fbs'
def build_fbuf(name, image_path):
builder = flatbuffers.Builder(0)
with open(image_path, "rb") as file1:
data1 = file1.read()
image_offset1 = builder.CreateByteVector(data1)
name_offset = builder.CreateString(name)
# Make the ImageBundle after all other vectors/strings are built
FbufImage.Start(builder)
FbufImage.AddName(builder, name_offset)
FbufImage.AddImage1(builder, image_offset1)
ImageBundle = FbufImage.End(builder)
builder.Finish(ImageBundle)
return builder.Output()
def main():
buf = build_fbuf("bundle1", "./test.png")
with open("bundle1.fbuf", "wb") as file:
file.write(buf)
if __name__ == '__main__':
main()
setup.sh - this generates the Flutter project with all the files in the proper places
#! /bin/zsh
# Generate Python (FbufImage.py) and Dart code (fbuf_image_generated.dart)
SCHEMA="fbuf_image.fbs"
flatc --dart --python $SCHEMA
python3 img2buf.py
# Generate the Flutter project
PROJ=fbuf_image # must be snake_case (https://dart.dev/tools/pub/pubspec#name)
PROJDIR=$PROJ
if [ -d $PROJDIR ] ; then
cd $PROJDIR
flutter clean
else
flutter create --platform=web --org com.tompark --project-name $PROJ $PROJDIR
cd $PROJDIR
fi
flutter pub add flat_buffers
# Copy a flatbuf asset to project, and add it to pubspec.yaml
mkdir -p ./assets/fbuf
cp -vf "../bundle1.fbuf" "assets/fbuf"
cat << EOF > assets.txt
- assets/fbuf/
EOF
PUBSPEC=pubspec.yaml
sed '/^ # assets:/s/# //' $PUBSPEC >$PUBSPEC.tmp # uncomment the "assets:" section, it has 2 space indentation
sed '/^ assets:/r assets.txt' $PUBSPEC.tmp >$PUBSPEC
rm $PUBSPEC.tmp
cp ../main.dart ./lib/main.dart
cp ../fbuf_image_generated.dart ./lib
main.dart - This is the Flutter app that loads the flatbuffer image and displays it
import 'package:flutter/material.dart';
import 'dart:io';
import 'dart:typed_data'; // ByteBuffer, Uint8List
import 'package:flutter/services.dart'; // rootBundle
import 'package:flat_buffers/flat_buffers.dart' as fbuf;
import 'fbuf_image_generated.dart'; // FbufImage
void main() {
WidgetsFlutterBinding.ensureInitialized();
runApp(const MainApp());
}
class MainApp extends StatelessWidget {
const MainApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flatbuffers Image Demo',
theme: ThemeData(
colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
useMaterial3: true,
),
home: const TestPage(),
);
}
}
class TestPage extends StatelessWidget {
const TestPage({Key? key}) : super(key: key);
static Future<FbufImage> load(String name) async {
// Get original file from assets in rootBundle
final assetBytes = await rootBundle.load("fbuf/${name}.fbuf");
final assetBuffer = assetBytes.buffer;
final Uint8List buffer = assetBytes.buffer.asUint8List();
return FbufImage(buffer);
}
@override
Widget build(BuildContext context) {
return Scaffold(
body: FutureBuilder<FbufImage>(
future: TestPage.load("bundle1"),
builder: (context, snapshot) {
if (snapshot.hasData) {
final bundle = snapshot.data as FbufImage ?? FbufImage([]);
final name = bundle.name ?? "";
if (bundle.image1 != null) {
final bytes = bundle.image1 ?? [];
if (bytes is Uint8List) {
print("${name}.image1 is already a Uint8List, len=${bytes.length}");
return Image.memory(bytes as Uint8List);
}
print("${name}.image1 is being copied, len=${bytes.length}");
return Image.memory(Uint8List.fromList(bytes));
}
}
return Center(child: CircularProgressIndicator());
}
)
);
}
}
I'm running the Flutter program using flutter run -d chrome
The terminal output always says: bundle1.image1 is being copied, len=226933
In other words, in the generated Dart file (fbuf_image_generated.dart), the generated class looks like this:
class FbufImage {
FbufImage._(this._bc, this._bcOffset);
factory FbufImage(List<int> bytes) {
final rootRef = fb.BufferContext.fromBytes(bytes);
return reader.read(rootRef, 0);
}
static const fb.Reader<FbufImage> reader = _FbufImageReader();
final fb.BufferContext _bc;
final int _bcOffset;
String? get name => const fb.StringReader().vTableGetNullable(_bc, _bcOffset, 4);
List<int>? get image1 => const fb.Uint8ListReader().vTableGetNullable(_bc, _bcOffset, 6);
@override
String toString() {
return 'FbufImage{name: ${name}, image1: ${image1}}';
}
}
The root problem
The catch is that the image1 property is returning a List<int>? and it cannot be cast to a Uint8List.
This means that the only way to create an image from the bytes is to copy it to a Uint8List by doing this:
Image.memory(Uint8List.fromList(bytes));
Is there a way to create the image without copying the bytes?
The whole purpose of trying to use Flatbuffers instead of Protobuf was to avoid copying the byte array. Thank you very much!
I just worded this question more succinctly and clearly here: https://stackoverflow.com/questions/77623267/how-to-avoid-copying-the-bytes-in-dart-when-deserializing-an-image-from-a-flatbu
Perhaps Uint8ListReader.read and Int8ListReader.read could be changed to stop returning a _FbUint8List and to instead return Uint8List.view(bc.buffer.buffer, listOffset + 4)?
Thanks James. I was hoping I could just modify something in the generated Dart code, but you're right. It does appear that this needs to be addressed in the Flatbuffers Dart package.
I am a Dart newbie, having just started this Flutter project a few months ago. I read somewhere that Uint8List is relatively new to Dart, and List<int> was used everywhere before that, and that's still used for backward compatibility. I'm guessing the Flatbuffers Dart package was written before the existence of Uint8List, and maybe for compatibility reasons it's not so easy to just change the API.
I didn't know much about ByteData or ByteBuffer, and didn't feel confident about digging around in the Flatbuffers source code, but spent some time today seeing if I could make sense of the Flatbuffers structure.
The following code is totally heinous, but as a test I wanted to see if it's possible to create the Image without the copy, and it works although I'd never use this code in production. It's a replacement for the TestPage class in the Flutter app given above:
class TestPage extends StatelessWidget {
const TestPage({Key? key}) : super(key: key);
static Future<ByteData> load(String name) async {
// Get original file from assets in rootBundle
final byteData = await rootBundle.load("fbuf/${name}.fbuf");
return byteData;
}
@override
Widget build(BuildContext context) {
return Scaffold(
body: FutureBuilder<ByteData>(
future: TestPage.load("bundle1"),
builder: (context, snapshot) {
if (snapshot.hasData) {
final ByteData data = snapshot.data!;
// The first uint32 is the offset to the root vtable
// See 'abstract class Reader<T>' in https://github.com/google/flatbuffers/blob/master/dart/lib/flat_buffers.dart#L1014
final root_offset = data.getUint32(0, Endian.little);
final vt_offset = data.getInt32(root_offset, Endian.little); // vTableSOffset, vtable starts at (root_offset - vt_offset)
final vt_size = data.getUint16(root_offset - vt_offset, Endian.little); // vtable size in bytes
final obj_size = data.getUint16(root_offset - vt_offset + 2, Endian.little); // object size in bytes, incl the vt_offset == 4 + (2 fields * 2 byte offset each)
final field1_offset = data.getUint16(root_offset - vt_offset + 2 + 2, Endian.little);
final field2_offset = data.getUint16(root_offset - vt_offset + 2 + 2 + 2, Endian.little);
print("\nroot_offset=${root_offset}, vt_offset=${vt_offset}, vt_size=${vt_size}, obj_size=${obj_size}, field1_offset=${field1_offset}, field2_offset=${field2_offset}");
final obj_offset = root_offset;
final name_offset = data.getInt32(obj_offset + field1_offset, Endian.little);
final img_offset = data.getInt32(obj_offset + field2_offset, Endian.little);
print("\nname_offset=${name_offset}, img_offset=${img_offset}");
final int name_data_offset = obj_offset + field1_offset + name_offset;
final name_len = data.getUint32(name_data_offset, Endian.little);
final name_bytes = data.buffer.asUint8List(name_data_offset+4, name_len);
final name_str = utf8.decode(name_bytes);
print("\nname_data_offset=${name_data_offset}, name_len=${name_len}, name='${name_str}'");
final int image_data_offset = obj_offset + field2_offset + img_offset;
final image_len = data.getUint32(image_data_offset, Endian.little);
print("\nimage_data_offset=${image_data_offset}, image_len=${image_len}");
final Uint8List image_bytes = data.buffer.asUint8List(image_data_offset+4, image_len);
if (image_bytes.length == image_len) {
print("using image bytes hacked from ByteData, len=${image_bytes.length}");
return Image.memory(image_bytes);
}
final bundle = FbufImage(data.buffer.asUint8List());
final name = bundle.name ?? "";
if (bundle.image1 != null) {
final bytes = bundle.image1 ?? [];
if (bytes is Uint8List) {
print("${name}.image1 is already a Uint8List, len=${bytes.length}");
return Image.memory(bytes as Uint8List);
}
print("${name}.image1 is being copied, len=${bytes.length}");
return Image.memory(Uint8List.fromList(bytes));
}
}
return Center(child: CircularProgressIndicator());
}
)
);
}
}
I guess this exercise wasn't helpful for any practical use, except for my own education. Since there are alternatives I could use, maybe I won't use Flatbuffers for my project after all. Should I just close this issue? If any of the project contributors see this message, feel free to close this issue. Thanks anyway!
I personally think that this would be worth doing, especially if it's as simple as my suggestion from https://github.com/google/flatbuffers/issues/8183#issuecomment-1846282614.
Oh geez, I just realized that it was you who wrote that backward compatibility thing about Uint8List vs List<int>.
There are a number of functions that take or return sequences of bytes as
List<int>, but usually that's for historical reasons, and changing those APIs to useUint8Listnow would break existing code. (There was an attempt to change the APIs, but it broke more code than expected and changes to some of the APIs had to be reverted.)
Thanks @tompark for the suggestions. Indeed, the use case looks very reasonable. Maybe it would be enough to implement Uint8List on _FbUint8List?
If you'd like to experiment and contribute a PR, I'd be more than willing to review and help landing it.
I am extremely interested in seeing this land and would be willing to help on the review/testing. My use case isn't quite an image, but rather packed signal data that is parsed as a Uint8List. I leaned towards protobufs in design initially because I was under the impression it was impossible to avoid the copy during deserialization.
Should other reader classes drop their wrapped type for the dart:typed_data equivalent (eg Uint16ListReader drops _FbUint16List for Uint16List)?
https://github.com/google/flatbuffers/blob/7106d86685e3395efc6df5d2ee5914fe622ed6b9/dart/lib/flat_buffers.dart#L1207-L1213
Would it not be enough to just take this path instead of the lazy path? I don't see where it's possible to configure whether the lazy path is taken though. Seems like the only way is to change the generated code to pass in a false parameter to Uint8ListReader. Sorry for the spam on the thread.
Hello. Sorry @vaind and @NotTsunami, for my lack of an earlier reply.
In Dec I did take a quick look at trying to change the Uint32ListReader.read return type as suggested by @jamesderlin, but:
- You'd have to change not only the dart library code, but also the
flatccode generator. - It's not backward-compatible. You'd be changing the return type of that class, so any existing app code that calls this API will break or at least cause warnings.
Why you'd have to change the flatc code generator:
For the fbuf_image.fbs example I gave above, here's what flatc --dart fbuf_image.fbs generated:
class FbufImage {
FbufImage._(this._bc, this._bcOffset);
factory FbufImage(List<int> bytes) {
final rootRef = fb.BufferContext.fromBytes(bytes);
return reader.read(rootRef, 0);
}
static const fb.Reader<FbufImage> reader = _FbufImageReader();
final fb.BufferContext _bc;
final int _bcOffset;
String? get name => const fb.StringReader().vTableGetNullable(_bc, _bcOffset, 4);
List<int>? get image1 => const fb.Uint8ListReader().vTableGetNullable(_bc, _bcOffset, 6);
@override
String toString() {
return 'FbufImage{name: ${name}, image1: ${image1}}';
}
}
As you can see, the get image1 computed property returns a List<int>?.
If you want it to return a Uint8List? instead, you have to modify the flatc app.
Backward-compatibility vs adding new APIs
The Uint8ListReader.read function is a required implementation of the Reader<T> protocol so you could just copy-paste a duplicate of each reader with the desired type:
So, instead of: class Uint8ListReader extends Reader<List<int>>
Smtg like this: class Uint8ListReader2 extends Reader<Uint8List>
But you'd have to do this for every type of list that you wanted, i.e. UInt16List, UInt32List, Float64List, etc.
I myself didn't want to clutter up the official library with copy of all of these accessor classes.
If someone else wants to do this, it seems straightforward.
What I did instead It would have been nice to contribute something upstream, but what I ended up doing wasn't conductive to that.
As simple as dart/lib/flat_buffers.dart may be, honestly I was very uncomfortable touching it in any way (esp. that data accessor Reader<T> protocol with its vTableGetNullable function and that private BufferContext class). The generated code seems surprisingly unfriendly and highly coupled with internals of the library code.
Since I'd already bothered to figure out the file format, I encoded the binary files using Python and just wrote my own flatbuffer decoder class in Dart.
I was thinking some time I'd write this up as a blog post but I haven't gotten around to it yet, and since others seem to need code like this now, I'll just post it here.
I'm not suggesting that this is a good idea, but if anyone wants to use my code, here it is. (Not to mention that the test code below can serve as half-decent examples of encoding more complex kinds of flatbuffers.)
flatbuffer.dart
import 'dart:typed_data'; // ByteData, Uint8List
import 'dart:convert' show utf8;
const bool dbg = true;
const int _sizeofUint16 = 2;
const int _sizeofUint32 = 4;
class Flatbuffer {
late final ByteData data;
late final int root; // data offset in bytes
late final int vTable; // data offset in bytes
late final int vCount; // number of fields in vtable
Flatbuffer(ByteData this.data, [int rootOffset = -1]) {
final int len = data.lengthInBytes;
if (rootOffset < 0) {
rootOffset = (len < 4) ? 0 : data.getUint32(0, Endian.little);
}
root = rootOffset;
final int vTableSOffset = (root <= 0 || root >= len-4) ? 0 : data.getInt32(root, Endian.little);
final int vTableOffset = root - vTableSOffset;
final int vTableSize = (vTableOffset < 0) ? 0 : data.getUint16(vTableOffset, Endian.little);
final int vHeaderSize = (vTableOffset < 0) ? 0 : data.getUint16(vTableOffset+2, Endian.little);
if (dbg) print("[FB] datalen=${len}, rootOffset=${rootOffset}, vtOffset=${vTableOffset}, vtSize=${vTableSize}, hdrSize=${vHeaderSize}");
if (vTableSize <= 0) {
vTable = 0;
vCount = 0;
} else {
vTable = vTableOffset + (_sizeofUint16 * 2); // add 4 bytes to step past vTableSize,vObjSize
vCount = (vTableSize - (_sizeofUint16 * 2)) ~/ 2;
}
if (dbg) print("[FB] table entries offset=${vTable}, numFields=${vCount}");
}
// derefField
// If field is scalar/struct, return the offset to the scalar value or the struct's in-line data block
// If field is vector/string, return the offset to the pointer to the data (use 'derefPointer' to get offset of the data)
@pragma('vm:prefer-inline')
int derefField(int idx) => (idx < 0 || idx >= vCount) ? 0 : data.getUint16((vTable + idx*2), Endian.little);
@pragma('vm:prefer-inline')
int derefPointer(int offset) => offset + data.getUint32(offset, Endian.little);
// vecFieldData
// If field is a single blob, then the blob is prefixed with uint32 length
// If field is an array of struct, then the array is prefixed with uint32 count (the array length or element size is not stored anywhere)
(int,int) vecFieldData(int fieldIdx) {
final fieldOffset = derefField(fieldIdx);
//print("[FB.bytesField] field[${fieldIdx}].offset=${fieldOffset} ${ fieldOffset==0?'RETURNING NULL':'' }");
if (fieldOffset == 0) return (0,0);
final dataOffset = derefPointer(root + fieldOffset);
//print("[FB.bytesField] data offset=${dataOffset}");
if (dataOffset > data.lengthInBytes) return (0,0);
final length = data.getUint32(dataOffset, Endian.little);
if (dbg) print("FB.bytes: field[${fieldIdx}].delta=${fieldOffset}, data[${root+fieldOffset}] --> data[${dataOffset}], len=${length}");
if (dataOffset + length > data.lengthInBytes) return (0,0);
return (dataOffset + _sizeofUint32, length);
}
Uint8List? bytesField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asUint8List(offset, length);
}
String? stringField(int fieldIdx) {
final bytes = bytesField(fieldIdx);
return (bytes == null) ? null : utf8.decode(bytes);
}
Float32List? float32ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asFloat32List(offset, length);
}
Float64List? float64ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asFloat64List(offset, length);
}
Int16List? int16ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asInt16List(offset, length);
}
Int32List? int32ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asInt32List(offset, length);
}
Int64List? int64ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asInt64List(offset, length);
}
Int8List? int8ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asInt8List(offset, length);
}
Uint16List? uint16ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asUint16List(offset, length);
}
Uint32List? uint32ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asUint32List(offset, length);
}
Uint64List? uint64ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asUint64List(offset, length);
}
Uint8List? uint8ListField(int fieldIdx) {
final (offset, length) = vecFieldData(fieldIdx);
return (length == 0) ? null : data.buffer.asUint8List(offset, length);
}
// SCALARS -- the value is inline so DO NOT USE 'derefPointer'
int? structOffset(int fieldIdx) {
final valueOffset = derefField(fieldIdx);
if (dbg) print("FB.struct: field[${fieldIdx}].delta=${valueOffset} --> ${ valueOffset==0? 'NULL': 'data[${root+valueOffset}]' }");
return (valueOffset == 0) ? null : root + valueOffset;
}
int? uint16Field(int fieldIdx) {
final valueOffset = derefField(fieldIdx);
if (dbg) print("FB.uint16: field[${fieldIdx}].delta=${valueOffset} --> ${ valueOffset==0? 'NULL': 'data[${root+valueOffset}]' }");
return (valueOffset == 0) ? null : data.getUint16(root + valueOffset, Endian.little);
}
double? float32Field(int fieldIdx) {
final valueOffset = derefField(fieldIdx);
if (dbg) print("FB.float: field[${fieldIdx}].delta=${valueOffset} --> ${ valueOffset==0? 'NULL': 'data[${root+valueOffset}]' }");
return (valueOffset == 0) ? null : data.getFloat32(root + valueOffset, Endian.little);
}
}
That doesn't work with the flatc generated code, but the data accessors in the above decoder are trivial to use.
Here are examples for deserializing tables containing a Uint8List, a table containing an inlined struct, and a table containing an array of struct with nested structs:
test_types.dart
import 'dart:typed_data'; // ByteData, Uint8List
import 'flatbuffer.dart';
// table Test01 {
// name: string;
// image1: [ubyte];
// }
class Test01 extends Flatbuffer {
Test01(ByteData bd) : super(bd);
// Fields
String? get name => stringField(0);
Uint8List? get image1 => bytesField(1);
@override
String toString() => 'Test01{name: ${name}, image1.len: ${image1?.lengthInBytes}}';
}
// table Test02 {
// x: uint16;
// y: uint16;
// dx: float;
// dy: float;
// name: string;
// image1: [ubyte];
// }
class Test02 extends Flatbuffer {
Test02(ByteData bd) : super(bd);
// Fields
int? get x => uint16Field(0);
int? get y => uint16Field(1);
double? get dx => float32Field(2);
double? get dy => float32Field(3);
String? get name => stringField(4);
Uint8List? get image1 => bytesField(5);
@override
String toString() => 'Test02{x=${x}, y=${y}, dx=${dx}, dy=${dy}, name: ${name}, image1.len: ${image1?.lengthInBytes}}';
}
// struct Rect {
// left: float;
// top: float;
// width: float;
// height: float;
// }
class Rect {
static const size = 16;
final ByteData bd;
final int bdOffset;
const Rect(this.bd, this.bdOffset);
double get left => bd.getFloat32(bdOffset + 0, Endian.little);
double get top => bd.getFloat32(bdOffset + 4, Endian.little);
double get width => bd.getFloat32(bdOffset + 8, Endian.little);
double get height => bd.getFloat32(bdOffset + 12, Endian.little);
}
// table Test03 {
// id: uint16;
// rect: Rect;
// f4: [float];
// name: string;
// image1: [ubyte];
// }
class Test03 extends Flatbuffer {
Test03(ByteData bd) : super(bd);
// Fields
int? get id => uint16Field(0);
Rect? get rect {
final offset = structOffset(1);
return (offset == null) ? null : Rect(data, offset);
}
Float32List? get f4 => float32ListField(2);
String? get name => stringField(3);
Uint8List? get image1 => bytesField(4);
// @override
// String toString() => 'Test02{id: ${id}, rect:[${left},${top},${width},${height}], f4:[${f4[0]},${f4[1]},${f4[2]},${f4[3]}], name: ${name}, image1.len: ${image1?.lengthInBytes}}';
}
// struct Item {
// final uint16 type;
// final Rect btnRect;
// final Rect imgRect;
// }
class Item {
static const size = 36;
final ByteData bd;
final int bdOffset;
const Item(this.bd, this.bdOffset);
int? get type => bd.getUint16(bdOffset + 0, Endian.little);
//padding 2 bytes
Rect get btnRect => Rect(bd, bdOffset + 4);
Rect get imgRect => Rect(bd, bdOffset + 4 + Rect.size);
// Rect? get btnRect {
// final offset = structOffset(1);
// return (offset == null) ? null : Rect(data, offset);
// }
// Rect? get imgRect {
// final offset = structOffset(2);
// return (offset == null) ? null : Rect(data, offset);
// }
}
// table Test04 {
// id: uint16;
// numItems: uint16;
// items: [Item];
// }
class Test04 extends Flatbuffer {
Test04(ByteData bd) : super(bd);
// Fields
int? get id => uint16Field(0);
int? get numItems => uint16Field(1);
List<Item> get items {
final (offset, count) = vecFieldData(2);
// This should be the itemCount
const size = Item.size; // The element size is hard-coded in the generated dart code
// final int size = length ~/ count;
// final rem = length % count;
// if (rem != 0) print('WARNING: vector length is not a multiple of item count: len=${length}, count=${count}, elemSize=${size}, remainder=${rem}');
if (dbg) print('[List<Item>] offset=${offset}, numItems=${numItems} == itemCount=${count}, itemSize=${size} (bytes)');
List<Item> items = [];
for (var idx=0; idx<count; idx++) {
items.add(Item(data, (idx*size)+offset));
}
return items;
}
// @override
// String toString() => 'Test02{id: ${id}, rect:[${left},${top},${width},${height}], f4:[${f4[0]},${f4[1]},${f4[2]},${f4[3]}], name: ${name}, image1.len: ${image1?.lengthInBytes}}';
}
This is an example main.dart that uses the above tables:
main.dart
import 'package:flutter/material.dart';
import 'dart:io';
import 'dart:typed_data'; // ByteBuffer, Uint8List
import 'dart:convert' show utf8;
import 'package:flutter/services.dart'; // rootBundle
import 'flatbuffer.dart' as fbuf;
import 'test_types.dart';
void main() {
WidgetsFlutterBinding.ensureInitialized();
runApp(const MainApp());
}
class MainApp extends StatelessWidget {
const MainApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flatbuffers Demo',
theme: ThemeData(
colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
useMaterial3: true,
),
home: const TestPage(),
);
}
}
class TestPage extends StatelessWidget {
const TestPage({Key? key}) : super(key: key);
static Future<ByteData> load(String name) async {
final fname = "fbuf/fbuf_${name}.bin";
// Get original file from assets in rootBundle
final byteData = await rootBundle.load(fname);
print("[MAIN] loaded file: '$fname', len: ${byteData.lengthInBytes}");
//final assetBuffer = assetBytes.buffer; // ByteBuffer
//final Uint8List buffer = assetBytes.buffer.asUint8List();
return byteData;
}
static String? getNameField(Object obj) {
return (obj is Test01) ? obj.name
: (obj is Test02) ? obj.name
: (obj is Test03) ? obj.name
: null;
}
static Uint8List? getImageField(Object obj) {
return (obj is Test01) ? obj.image1
: (obj is Test02) ? obj.image1
: (obj is Test03) ? obj.image1
: null;
}
@override
Widget build(BuildContext context) {
const String testName = "test04";
return Scaffold(
body: FutureBuilder<ByteData>(
future: TestPage.load(testName),
builder: (context, snapshot) {
if (snapshot.hasData) {
final ByteData data = snapshot.data!;
final bundle = switch(testName) {
"test01" => Test01(data),
"test02" => Test02(data),
"test03" => Test03(data),
"test04" => Test04(data),
_ => Test01(data),
};
if (bundle is Test02) {
final int? x = bundle.x;
final int? y = bundle.y;
final double? dx = bundle.dx;
final double? dy = bundle.dy;
print("[MAIN] Scalars: x=${x}, y=${y}, dx=${dx?.toStringAsFixed(4)}, dy=${dy?.toStringAsFixed(4)}\n");
}
else if (bundle is Test03) {
final int? id = bundle.id;
final Rect? rect = bundle.rect;
final double? left = rect?.left;
final double? top = rect?.top;
final double? width = rect?.width;
final double? height = rect?.height;
print("[MAIN] Scalars: id=${id}, rect=[${left?.toStringAsFixed(4)},${top?.toStringAsFixed(4)},${width?.toStringAsFixed(4)},${height?.toStringAsFixed(4)}] \n");
final Float32List f4 = bundle.f4 ?? Float32List.fromList([-1,-1,-1,-1]);
print("[MAIN] Vec: f4=[${f4[0].toStringAsFixed(4)},${f4[1].toStringAsFixed(4)},${f4[2].toStringAsFixed(4)},${f4[3].toStringAsFixed(4)}] \n");
}
else if (bundle is Test04) {
final int? id = bundle.id;
final int? numItems = bundle.numItems;
print("[MAIN] Scalars: id=${id}, count=${numItems}\n");
final List<Item> items = bundle.items;
final int count = items.length;
for (int idx=0; idx<count; idx++) {
final int type = items[idx]?.type ?? 0;
final Rect? btnRect = items[idx]?.btnRect;
final Rect? imgRect = items[idx]?.imgRect;
print(" [${idx}]: type=${type}");
print(" [${idx}]: btnRect=[${btnRect?.left?.toStringAsFixed(4)},${btnRect?.top?.toStringAsFixed(4)},${btnRect?.width?.toStringAsFixed(4)},${btnRect?.height?.toStringAsFixed(4)}]");
print(" [${idx}]: imgRect=[${imgRect?.left?.toStringAsFixed(4)},${imgRect?.top?.toStringAsFixed(4)},${imgRect?.width?.toStringAsFixed(4)},${imgRect?.height?.toStringAsFixed(4)}]");
}
}
else {
}
final name = getNameField(bundle) ?? ""; //bundle.name ?? "";
final Uint8List bytes = getImageField(bundle) ?? Uint8List(0); //bundle.image1 ?? Uint8List(0);
if (name != null) print("[MAIN] Vec: name='${name}'");
if (bytes.length > 0) {
if (bytes is Uint8List) {
print("[MAIN] image1 is already a Uint8List, len=${bytes.length}");
return Image.memory(bytes as Uint8List);
}
print("[MAIN] image1 is being copied, len=${bytes.length}");
return Image.memory(Uint8List.fromList(bytes));
//return Image.memory((bytes as Uint8List).bc.buffer.buffer.asUint8List(6, bytes.length));
}
}
return Center(child: CircularProgressIndicator());
}
)
);
}
}
And for completeness, here are the .fbs files and the Python scripts to create the flatbuffer binaries.
test01.fbs
table Test01 {
name: string;
image1: [ubyte];
}
root_type Test01;
fbuf01.py
import flatbuffers
import Test01 # generated by 'flatc --python test01.fbs'
def build_fbuf(name, image_path):
builder = flatbuffers.Builder(0)
with open(image_path, "rb") as file1:
data1 = file1.read()
image_offset1 = builder.CreateByteVector(data1)
name_offset = builder.CreateString(name)
# Make the ImageBundle after all other vectors/strings are built
Test01.Start(builder)
Test01.AddName(builder, name_offset)
Test01.AddImage1(builder, image_offset1)
ImageBundle = Test01.End(builder)
builder.Finish(ImageBundle)
return builder.Output()
def main():
buf = build_fbuf("bundle1", "./test.png")
with open("fbuf_test01.bin", "wb") as file:
file.write(buf)
if __name__ == '__main__':
main()
test02.fbs
table Test02 {
x: uint16;
y: uint16;
dx: float;
dy: float;
name: string;
image1: [ubyte];
}
root_type Test02;
fbuf02.py
import flatbuffers
import Test02 # generated by 'flatc --python test02.fbs'
def build_fbuf(name, image_path):
builder = flatbuffers.Builder(0)
with open(image_path, "rb") as file1:
data1 = file1.read()
image_offset1 = builder.CreateByteVector(data1)
name_offset = builder.CreateString(name)
# Make the ImageBundle after all other vectors/strings are built
# Add the fields in reverse order (bottom up)
# image1, name, dy, dx, y, x
Test02.Start(builder)
Test02.AddImage1(builder, image_offset1)
Test02.AddName(builder, name_offset)
Test02.AddDy(builder, 0.2)
Test02.AddDx(builder, 0.1)
Test02.AddY(builder, 688)
Test02.AddX(builder, 42)
ImageBundle = Test02.End(builder)
builder.Finish(ImageBundle)
return builder.Output()
def main():
buf = build_fbuf("bundle2", "./test.png")
with open("fbuf_test02.bin", "wb") as file:
file.write(buf)
if __name__ == '__main__':
main()
test03.fbs
struct Rect {
left: float;
top: float;
width: float;
height: float;
}
table Test03 {
id: uint16;
rect: Rect;
f4: [float];
name: string;
image1: [ubyte];
}
root_type Test03;
fbuf03.py
import flatbuffers
import Test03 # generated by 'flatc --python test03.fbs'
# import Rect
def build_fbuf(name, image_path):
builder = flatbuffers.Builder(0)
# table Test03 {
# id: uint16;
# rect: Rect;
# f4: [float];
# name: string;
# image1: [ubyte];
# }
id_value = 42
left_value = 11.1
top_value = 22.2
width_value = 33.3
height_value = 44.4
f4_values = [1.01, 2.02, 3.03, 4.04]
# name = 'bundle3'
# image_path = './test.png'
# Build in bottom-up order: image1, name, f4, rect, id
# Build 'image1'
with open(image_path, "rb") as file1:
data1 = file1.read()
image_offset1 = builder.CreateByteVector(data1)
# Build 'name'
name_offset = builder.CreateString(name)
# Build 'f4' (array of float)
f4_offsets = []
builder.StartVector(4, 4, 4) # elemSize:4, numElems:4, alignment:4
for val in reversed(f4_values):
offset = builder.PrependFloat32(val)
f4_offsets.append(offset)
f4_offset = builder.EndVector()
# Build rect?
# NO: a Flatbuffer struct is stored in-line
# If it's built ahead of time here, then later when calling 'Test03.AddRect()', Flatbuffers throws an exception:
# "flatbuffers.builder.StructIsNotInlineError: flatbuffers: Tried to write a Struct at an Offset that is different from the current Offset of the Builder."
# builder.Prep(4, 16) # size: 4, dataBytes: 16
# builder.PrependFloat32(height_value)
# builder.PrependFloat32(width_value)
# builder.PrependFloat32(top_value)
# builder.PrependFloat32(left_value)
# rect_offset = builder.Offset()
# Make the ImageBundle after all other vectors/strings are built
# Add the fields in reverse order (bottom up)
# image1, name, dy, dx, y, x
Test03.Start(builder)
Test03.AddImage1(builder, image_offset1)
Test03.AddName(builder, name_offset)
Test03.AddF4(builder, f4_offset)
# Rect
# A Flatbuffer struct is stored in-line, so build it here instead of ahead of time
# rect_offset = Rect.CreateRect(builder, left, top, width, height) # see the generated 'Rect.py'
builder.Prep(4, 16) # size: 4, dataBytes: 16
builder.PrependFloat32(height_value)
builder.PrependFloat32(width_value)
builder.PrependFloat32(top_value)
builder.PrependFloat32(left_value)
rect_offset = builder.Offset()
Test03.AddRect(builder, rect_offset)
Test03.AddId(builder, id_value)
Bundle = Test03.End(builder)
builder.Finish(Bundle)
return builder.Output()
def main():
buf = build_fbuf("bundle3", "./test.png")
with open("fbuf_test03.bin", "wb") as file:
file.write(buf)
if __name__ == '__main__':
main()
test04.fbs
struct Rect {
left: float;
top: float;
width: float;
height: float;
}
struct Item {
type: uint16;
btn_rect: Rect;
img_rect: Rect;
}
table Test04 {
id: uint16;
num_items: uint16;
items: [Item];
}
root_type Test04;
fbuf04.py
import flatbuffers
import json
from types import SimpleNamespace
import Test04 # generated by 'flatc --python test04.fbs'
# import Rect
def build_item(builder, item):
# struct Item {
# final uint16 type;
# final Rect btn_rect;
# final Rect img_rect;
# }
type = item.type
btnRect = item.btn_rect
imgRect = item.img_rect
# see the generated 'Item.py'
# item_offset = Item.CreateItem(builder, type, btnRect_left, btnRect_top, btnRect_width, btnRect_height, imgRect_left, imgRect_top, imgRect_width, imgRect_height):
builder.Prep(4, 36)
builder.Prep(4, 16)
builder.PrependFloat32(imgRect.height)
builder.PrependFloat32(imgRect.width)
builder.PrependFloat32(imgRect.top)
builder.PrependFloat32(imgRect.left)
builder.Prep(4, 16)
builder.PrependFloat32(btnRect.height)
builder.PrependFloat32(btnRect.width)
builder.PrependFloat32(btnRect.top)
builder.PrependFloat32(btnRect.left)
builder.Pad(2)
builder.PrependUint16(type)
return builder.Offset()
def build_fbuf(name, obj):
# table Test04 {
# id: uint16;
# num_items: uint16;
# items: [Item];
# }
id_value = obj.id
count_value = obj.num_items
items = obj.items
# Build in bottom-up order: items, numItems, id
builder = flatbuffers.Builder(0)
# Build 'items' (array of Item)
# vec_offsets = []
builder.StartVector(36, len(items), 4) # elemSize:36 (4+16+16), numElems:x, alignment:4
for item in reversed(items):
offset = build_item(builder, item)
# vec_offsets.append(offset)
print(f" item offset: {offset}")
items_offset = builder.EndVector()
print(f"vector of items offset: {items_offset}")
# Make the bundle after all other vectors/strings are built
# Add the fields in reverse order (bottom up)
# items, num_items,
Test04.Start(builder)
Test04.AddItems(builder, items_offset)
Test04.AddNumItems(builder, len(items))
Test04.AddId(builder, id_value)
Bundle = Test04.End(builder)
builder.Finish(Bundle)
return builder.Output()
def main():
obj = json.loads(
'''{
"id": 42, "num_items": 4,
"items": [
{"type":1, "btn_rect":{"left":0.21, "top":0.26, "width":0.17, "height":0.17}, "img_rect":{"left":0.0, "top":0.05, "width":0.66, "height":0.64}},
{"type":2, "btn_rect":{"left":0.65, "top":0.18, "width":0.17, "height":0.17}, "img_rect":{"left":0.19, "top":0.125, "width":1.24, "height":0.92}},
{"type":3, "btn_rect":{"left":0.43, "top":0.41, "width":0.17, "height":0.17}, "img_rect":{"left":0.2, "top":0.22, "width":0.8, "height":0.8}},
{"type":4, "btn_rect":{"left":0.60, "top":0.72, "width":0.17, "height":0.17}, "img_rect":{"left":0.505, "top":0.615, "width":0.82, "height":0.965}}
]
}''',
object_hook=lambda d: SimpleNamespace(**d)
# See https://stackoverflow.com/questions/6578986/how-to-convert-json-data-into-a-python-object
)
buf = build_fbuf("bundle4", obj)
with open("fbuf_test04.bin", "wb") as file:
file.write(buf)
if __name__ == '__main__':
main()
- It's not backward-compatible. You'd be changing the return type of that class, so any existing app code that calls this API will break or at least cause warnings.
No. Uint8ListReader.read and Int8ListReader.read both return List<int>. If they returned Uint8List.view(...) as I suggested, the return type would not need to change; Uint8List is a subtype of List<int>. Of course, List<int> is a wider type than necessary and isn't ideal, but it still allow callers to cast to the narrower Uint8List type without creating a copy.
No.
Uint8ListReader.readandInt8ListReader.readboth returnList<int>. If they returnedUint8List.view(...)as I suggested, the return type would not need to change;Uint8Listis a subtype ofList<int>. Of course,List<int>is a wider type than necessary and isn't ideal, but it still allow callers to cast to the narrowerUint8Listtype without creating a copy.
OK, I'm wrong and obviously I don't know about enough about Uint8List. I said earlier that I do not feel comfortable with submitting a pull request, but since that seems to be the way I'm supposed to resolve this issue other than closing it, then I'll just close the issue. If someone else wants to submit the pull request then they can open a new issue or reopen this one.
To reiterate, what I'm proposing is to change Uint8ListReader.read and Int8ListReader.read to return a Uint8List/Int8List along all code paths instead of just along some of them. Currently the lazy paths returns a _FbUint8List, so that prevents callers from casting the result to a Uint8List/Int8List. All of this happens dart/lib/flat_buffers.dart, so I don't think it requires any changes for code generation. In the generated code, a line such as:
List<int>? get image1 => const fb.Uint8ListReader().vTableGetNullable(_bc, _bcOffset, 6);
calls into Uint8ListReader().vTableGetNullable which I think returns whatever Uint8ListReader.read returns.
To be precise, I think the following should work?
diff --git a/dart/lib/flat_buffers.dart b/dart/lib/flat_buffers.dart
index 6f307872..113364e1 100644
--- a/dart/lib/flat_buffers.dart
+++ b/dart/lib/flat_buffers.dart
@@ -729,7 +729,7 @@ class Builder {
@pragma('vm:prefer-inline')
void _writeUTFString(String value) {
- final bytes = utf8.encode(value) as Uint8List;
+ final bytes = utf8.encode(value);
final length = bytes.length;
_prepare(4, 1, additionalBytes: length + 1);
_setUint32AtTail(_tail, length);
@@ -1202,7 +1202,7 @@ class Uint8ListReader extends Reader<List<int>> {
@pragma('vm:prefer-inline')
List<int> read(BufferContext bc, int offset) {
final listOffset = bc.derefObject(offset);
- if (lazy) return _FbUint8List(bc, listOffset);
+ if (lazy) return Uint8List.view(bc.buffer.buffer, listOffset + 4);
final length = bc._getUint32(listOffset);
final result = Uint8List(length);
@@ -1247,7 +1247,7 @@ class Int8ListReader extends Reader<List<int>> {
@pragma('vm:prefer-inline')
List<int> read(BufferContext bc, int offset) {
final listOffset = bc.derefObject(offset);
- if (lazy) return _FbUint8List(bc, listOffset);
+ if (lazy) return Int8List.view(bc.buffer.buffer, listOffset + 4);
final length = bc._getUint32(listOffset);
final result = Int8List(length);
@@ -1337,24 +1337,6 @@ class _FbUint16List extends _FbList<int> {
int operator [](int i) => bc._getUint16(offset + 4 + 2 * i);
}
-/// List backed by 8-bit unsigned integers.
-class _FbUint8List extends _FbList<int> {
- _FbUint8List(BufferContext bc, int offset) : super(bc, offset);
-
- @override
- @pragma('vm:prefer-inline')
- int operator [](int i) => bc._getUint8(offset + 4 + i);
-}
-
-/// List backed by 8-bit signed integers.
-class _FbInt8List extends _FbList<int> {
- _FbInt8List(BufferContext bc, int offset) : super(bc, offset);
-
- @override
- @pragma('vm:prefer-inline')
- int operator [](int i) => bc._getInt8(offset + 4 + i);
-}
-
/// List backed by 8-bit unsigned integers.
class _FbBoolList extends _FbList<bool> {
_FbBoolList(BufferContext bc, int offset) : super(bc, offset);
(I dislike the magic 4 offsets and would prefer replacing them with symbolic constants if I understood what it represented. At any rate, the code currently already uses them, so it's not any worse. Additionally, the old Int8ListReader.read used _FbUint8List instead of _FbInt8List, and _FbInt8List was unused, which was probably a bug.)
I would be fine creating a pull request, but I am unfamiliar with flatbuffers and have no idea how to test the change. Otherwise, although I hate having to keep asking him for favors, perhaps @dnfield could help or provide additional insight; it appears that he originally wrote this file.
To reiterate, what I'm proposing is to change
Uint8ListReader.readandInt8ListReader.readto return aUint8List/Int8Listalong all code paths instead of just along some of them. Currently thelazypaths returns a_FbUint8List, so that prevents callers from casting the result to aUint8List/Int8List. All of this happensdart/lib/flat_buffers.dart, so I don't think it requires any changes for code generation. In the generated code, a line such as:List<int>? get image1 => const fb.Uint8ListReader().vTableGetNullable(_bc, _bcOffset, 6);calls into
Uint8ListReader().vTableGetNullablewhich I think returns whateverUint8ListReader.readreturns.To be precise, I think the following should work?
diff --git a/dart/lib/flat_buffers.dart b/dart/lib/flat_buffers.dart index 6f307872..113364e1 100644 --- a/dart/lib/flat_buffers.dart +++ b/dart/lib/flat_buffers.dart @@ -729,7 +729,7 @@ class Builder { @pragma('vm:prefer-inline') void _writeUTFString(String value) { - final bytes = utf8.encode(value) as Uint8List; + final bytes = utf8.encode(value); final length = bytes.length; _prepare(4, 1, additionalBytes: length + 1); _setUint32AtTail(_tail, length); @@ -1202,7 +1202,7 @@ class Uint8ListReader extends Reader<List<int>> { @pragma('vm:prefer-inline') List<int> read(BufferContext bc, int offset) { final listOffset = bc.derefObject(offset); - if (lazy) return _FbUint8List(bc, listOffset); + if (lazy) return Uint8List.view(bc.buffer.buffer, listOffset + 4); final length = bc._getUint32(listOffset); final result = Uint8List(length); @@ -1247,7 +1247,7 @@ class Int8ListReader extends Reader<List<int>> { @pragma('vm:prefer-inline') List<int> read(BufferContext bc, int offset) { final listOffset = bc.derefObject(offset); - if (lazy) return _FbUint8List(bc, listOffset); + if (lazy) return Int8List.view(bc.buffer.buffer, listOffset + 4); final length = bc._getUint32(listOffset); final result = Int8List(length); @@ -1337,24 +1337,6 @@ class _FbUint16List extends _FbList<int> { int operator [](int i) => bc._getUint16(offset + 4 + 2 * i); } -/// List backed by 8-bit unsigned integers. -class _FbUint8List extends _FbList<int> { - _FbUint8List(BufferContext bc, int offset) : super(bc, offset); - - @override - @pragma('vm:prefer-inline') - int operator [](int i) => bc._getUint8(offset + 4 + i); -} - -/// List backed by 8-bit signed integers. -class _FbInt8List extends _FbList<int> { - _FbInt8List(BufferContext bc, int offset) : super(bc, offset); - - @override - @pragma('vm:prefer-inline') - int operator [](int i) => bc._getInt8(offset + 4 + i); -} - /// List backed by 8-bit unsigned integers. class _FbBoolList extends _FbList<bool> { _FbBoolList(BufferContext bc, int offset) : super(bc, offset);(I dislike the magic
4offsets and would prefer replacing them with symbolic constants if I understood what it represented. At any rate, the code currently already uses them, so it's not any worse. Additionally, the oldInt8ListReader.readused_FbUint8Listinstead of_FbInt8List, and_FbInt8Listwas unused, which was probably a bug.)I would be fine creating a pull request, but I am unfamiliar with flatbuffers and have no idea how to test the change. Otherwise, although I hate having to keep asking him for favors, perhaps @dnfield could help or provide additional insight; it appears that he originally wrote this file.
That patch isn't quite right, as the ints are written to the Uint8List as Int32 instead of Int8, but I opened up a PR that writes them correctly. Still unsure of any side effects, and it's functionally untested. Also, instead of removing _FbInt8List I just made Int8ListReader use the currently unused type. Hopefully it can get some traction and land!