flink
flink copied to clipboard
apache
[FLINK-28548][Connectors / FileSystem] Fix the exception FileNotFoundException when the commit partition base path is not created
What is the purpose of the change
The commit partition base path is not created when no data is sent which may cause FileNotFoundException. Just fix it.
Brief change log
- Check whether the base path exists before listStatus for the path.
Verifying this change
This change added tests and can be verified as follows:
- Added test testPartitionPathNotExist in FileSystemCommitterTest
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving): (no) - The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
Documentation
- Does this pull request introduce a new feature? (no)
CI report:
- adf83ff7afed8571281c8fe0ec03c8e72a7e0906 Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build
