pinot
pinot copied to clipboard
InputFileSegmentNameGenerator raised NullPointerException when pattern contains optional group
When using segment name generator type inputFile, I found a issue that when my file.path.pattern contains an optional part, it will cause a NullPointerException. For example, pattern is '.+/(\w+)(.parquet)?', and template is '${filePathPattern:\1}'. Because not all my files have parquet extension, so I added an optional part in the pattern, even I didn't use the second group in template, it still get NullPointerException. The reason is current code loop all groups and do string replacement, but the group may be null for optional group, then the replace method will raise NullPointerException.
for (int i = 1; i <= m.groupCount(); i++) {
segmentName = segmentName.replace(String.format(PARAMETER_TEMPLATE, i), m.group(i));
}
To fix it, need to check null on groups, if null, skip replacement or replace with empty string.