webmagic icon indicating copy to clipboard operation
webmagic copied to clipboard

下个迭代能不能把SubPageProcessor的match参数从request改成page

Open yuweiming2016 opened this issue 4 years ago • 0 comments

下个迭代能不能把SubPageProcessor的match参数从request改成page,有些时候要用到。 例如match返回false时,同时要把page.setSkip设置为false,让它不要触发Pipeline

    @Override
    public void process(Page page) {
        for (SubPageProcessor subPageProcessor : subPageProcessors) {
            if (subPageProcessor.match(page.getRequest())) {
                SubPageProcessor.MatchOther matchOtherProcessorProcessor = subPageProcessor.processPage(page);
                if (matchOtherProcessorProcessor == null || matchOtherProcessorProcessor != SubPageProcessor.MatchOther.YES) {
                    return;
                }
            }
        }
    }

yuweiming2016 avatar Feb 08 '21 02:02 yuweiming2016