gravitino icon indicating copy to clipboard operation
gravitino copied to clipboard

Optimize the configuration iterator in FileSystem provider

Open yuqi1129 opened this issue 10 months ago • 1 comments

The following code is very time-consuming in FileSystemProvider like HDFSFileSystemProvider and needs to be optimized according to performance test:

    Configuration configuration = new Configuration();
    config.forEach(
        (k, v) -> {
          configuration.set(k.replace(GRAVITINO_BYPASS, ""), v);
        });
Image

The operation configuration.set() will take much time.

We can convert the map to an XML stream and store the XML stream in the configuration only once.

yuqi1129 avatar Feb 27 '25 02:02 yuqi1129

@yuqi1129 please assign it to me.

sunxiaojian avatar Mar 01 '25 14:03 sunxiaojian