frontera icon indicating copy to clipboard operation
frontera copied to clipboard
verified publisher icon scrapinghub

While using sqlalchemy backend 301 redirect remained as QUEUED

Open RajatGoyal opened this issue 10 years ago • 1 comments

Only redirected url is marked marked as CRAWLED but the redirected remains as QUEUED.

RajatGoyal avatar Sep 13 '15 12:09 RajatGoyal

Yes, that's confusing. At the same time it depends on canonical solver behavior. So I think, the right way is to mark as CRAWLED url considered as canonical, and leave other urls canonical solver to decide.

sibiryakov avatar Sep 17 '15 03:09 sibiryakov