Browser4 icon indicating copy to clipboard operation
Browser4 copied to clipboard

浏览器资源不会释放

Open suntsao opened this issue 1 year ago • 1 comments

环境

  • pulsar-all版本:1.13.2-SNAPSHOT
  • 浏览器:google-chrome-stable-127.0.6533.99-1.x86_64.rpm

关键业务代码

PulsarSession session = PulsarContexts.createSession();
LoadOptions options = session.options("-parse -refresh");
options.getEvent().getBrowseEventHandlers().getOnDocumentActuallyReady().addLast((page, driver, other) -> {
    FeaturedDocument loadDocument = session.parse(page);
    CheckStatus checkStatus = CheckStatus.SUCCESS;
    for (SiteStatusCheckHandle handler: handlers) {
        checkStatus = handler.process(page, loadDocument);
        if(CheckStatus.SUCCESS != checkStatus){
            break;
        }
    }

    System.out.printf(page.getUrl() + " -> " + checkStatus + "\n");
    siteCheckService.handleCheckResultForUrl(page.getUrl(), checkStatus);
    return loadDocument;
});

List<String> urls = this.siteMapper.getAllSite().stream().map(siteModel -> siteModel.getUrl()).toList();
session.loadAllAsync(urls, options);
session.getContext().await();
session.close();

问题描述

服务器内存12G,采集检查的 URL 总共才 1600 条,基本上到1000 条左右,就因为内存不够无法执行; 查了一下应该是浏览器的资源没有释放;

不知道我这种写法是不是有什么问题?没有正确释放资源?

suntsao avatar Sep 06 '24 12:09 suntsao

Use session.submit() instead of session.loadAllAsync().

platonai avatar Sep 13 '24 10:09 platonai