paimon-rust icon indicating copy to clipboard operation
paimon-rust copied to clipboard

spec: Introduce unittests and integration tests

Open Xuanwo opened this issue 1 year ago • 7 comments

Our community works hard on implementation specifications. It's time for us to introduce unit tests and integration tests. We need to ensure that we can serialize/deserialize specifications correctly.

Xuanwo avatar Aug 01 '24 05:08 Xuanwo

I will add the Snapshot test for soon.

QuakeWang avatar Aug 01 '24 08:08 QuakeWang

The serialize/deserialize in Snapshot has completd by #39. I will add ut for the rest files.

QuakeWang avatar Aug 06 '24 06:08 QuakeWang

The serialize/deserialize in Snapshot has completd by #39. I will add ut for the rest files.

That would be greatly appreciated.

Xuanwo avatar Aug 06 '24 06:08 Xuanwo

The following JSON is output by ManifestFileData with Java.

{manifest-eddc1df1-3bfd-4918-bff3-a41560a8d280, 10000, 65, 35, org.apache.paimon.stats.BinaryTableStats@c2774299, 0}

The following JSON is output by ManifestFileData with Rust. In the Rust version has using #[serde(rename = "_FILE_NAME")] for rename the definition.

{"_FILE_NAME":"manifest_file_meta.json","_FILE_SIZE":1024,"_NUM_ADDED_FILES":10,"_NUM_DELETED_FILES":5,"_PARTITION_STATS":{"_MIN_VALUES":[1,2,3],"_MAX_VALUES":[4,5,6],"_NULL_COUNTS":[]},"_SCHEMA_ID":0}

Which one should be used when adding the serialize/deserialize test?

cc @Xuanwo

QuakeWang avatar Aug 13 '24 02:08 QuakeWang

Does it mean ManifestFileData in java has it's own output format? It doesn't look like a valid JSON to me.

cc @devillove084, would you like to generate some fixture for ManifestFileData too?

Xuanwo avatar Aug 13 '24 03:08 Xuanwo

Does it mean ManifestFileData in java has it's own output format? It doesn't look like a valid JSON to me.

cc @devillove084, would you like to generate some fixture for ManifestFileData too?

@Xuanwo No problem, I will mention another ISSUE and related PR in some time later.

devillove084 avatar Aug 13 '24 04:08 devillove084

Does it mean ManifestFileData in java has it's own output format? It doesn't look like a valid JSON to me. cc @devillove084, would you like to generate some fixture for ManifestFileData too?

@Xuanwo No problem, I will mention another ISSUE and related PR in some time later.

link #54

devillove084 avatar Aug 13 '24 15:08 devillove084