seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Fix][connnector-hive] readStrategy which has field FileSystem should close when MultipleTableHiveSourceReader close.

Open sohurdc opened this issue 6 months ago • 2 comments

Purpose of this pull request

Solve the problem of hive memory leak. HiveSourceConfig holds a ReadStrategy object, and ReadStrategy holds a FileSystem object. The FileSystem object needs to be closed. However, the HiveSourceConfig class does not have a close method. HiveSourceConfig is referenced in MultipleTableHiveSourceReader and MultipleTableHiveSourceSplitEnumerator; however, the ReadStrategy attribute in HiveSourceConfig is only accessible in the HiveSourceReader class. The ReadStrategy attribute is not used in the MultipleTableHiveSourceSplitEnumerator class. Therefore, when MultipleTableHiveSourceReader is closed, just close the ReadStrategy. image

Does this PR introduce any user-facing change?

no

How was this patch tested?

no,I ran my job and found no problems, but I did not test the failure recovery.

Check list

  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update plugin-mapping.properties and add new connector information in it
    2. Update the pom file of seatunnel-dist
    3. Add ci label in label-scope-conf
    4. Add e2e testcase in seatunnel-e2e
    5. Update connector plugin_config

sohurdc avatar Jun 17 '25 15:06 sohurdc

link #8637

Hisoka-X avatar Jun 18 '25 02:06 Hisoka-X

LGTM. Wating ci passes.

Hisoka-X avatar Jun 18 '25 03:06 Hisoka-X

@sohurdc please check ci

Carl-Zhou-CN avatar Jun 24 '25 13:06 Carl-Zhou-CN

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Oct 23 '25 00:10 github-actions[bot]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

github-actions[bot] avatar Oct 30 '25 00:10 github-actions[bot]