alcor icon indicating copy to clipboard operation
alcor copied to clipboard

[VPC] The /segments/createDefaultTable API creating default segment table for network takes too long

Open cj-chung opened this issue 4 years ago • 3 comments

Issue The /segments/createDefaultTable is the first API need to be call before creating any network resources. It will create default segment partition table for network resource. The implementation right now using two for-loops, each for-loop takes 4096 iteration to update database.

Suggestion fix Make the database update in one call after finishing the datatable creation in the memory.

cj-chung avatar Jun 04 '21 17:06 cj-chung

No permission assign

DavidLiu506 avatar Jun 04 '21 20:06 DavidLiu506

Discussed with @DavidLiu506, we will create a new service method to support bulk creation of segment ranges, and use iCache.putAll to consolidate 4K DB write.

xieus avatar Jun 07 '21 19:06 xieus

@pkommoju also suggested one possible optimization as follows if putAll takes too long for large db write.

Putall is also considered not good enough for large data, they suggest the following way:

// Get the data streamer reference and stream data.

try (IgniteDataStreamer<Integer, String> stmr = ignite.dataStreamer(The_name_f_theCache_goes_here)) {   

    // Stream entries.

    for (int i = 0; i < 100000; i++)

        stmr.addData(i, Integer.toString(i));
}

Basically, I think we will create another method in ICache interface to handle this method.

xieus avatar Jun 07 '21 19:06 xieus