pinot icon indicating copy to clipboard operation
pinot copied to clipboard

InputFileSegmentNameGenerator raised NullPointerException when pattern contains optional group

Open jhyao opened this issue 1 year ago • 0 comments

When using segment name generator type inputFile, I found a issue that when my file.path.pattern contains an optional part, it will cause a NullPointerException. For example, pattern is '.+/(\w+)(.parquet)?', and template is '${filePathPattern:\1}'. Because not all my files have parquet extension, so I added an optional part in the pattern, even I didn't use the second group in template, it still get NullPointerException. The reason is current code loop all groups and do string replacement, but the group may be null for optional group, then the replace method will raise NullPointerException.

      for (int i = 1; i <= m.groupCount(); i++) {
        segmentName = segmentName.replace(String.format(PARAMETER_TEMPLATE, i), m.group(i));
      }

To fix it, need to check null on groups, if null, skip replacement or replace with empty string.

jhyao avatar Jul 03 '24 15:07 jhyao