';s' download fails but 'right click download linked file' succeds.

[hidroto]

Commit: 3.0-alpha-65-gee984e6 WebKit compile: 2.16.6 WebKit run: 2.18.3 GTK compile: 3.22.17 GTK run: 3.22.17 libsoup compile: 2.56.1 libsoup run: 2.56.1 Extension dir: /usr/local/lib/vimb

Steps to reproduce

Process one: goto https://arxiv.org/abs/1712.04741 type ';s' to save the pdf link displays error 'download of 1712.04741 failed (forbidden)'

Process two: goto https://arxiv.org/abs/1712.04741 right click on pdf link click 'download linked file' file is downloaded as expected

Expected behaviour

process one should have downloaded the file just like process two.

Actual behaviour

process one fails

Possable causes

i believe that arxiv.org has in place methods to stop non-browsers from downloading the content as i also
can not download the pdf with wget. perhaps process one uses a diffrent user-agent to process two.

[fanglingsu]

@hidroto Thank you for reporting this bug. I can confirm this behaviour. I'll do some more testing and investigation soon.

[rti]

Could it be also related to the referer header?

[fanglingsu]

@rti I do not think it is realted to the Referer. If I set :set header=Referer to remove the referer header from the request and click to the pdf link the download starts as expected.

[fanglingsu]

It seems that the issue is caused by the redirect. The link to the pdf points to 'arxiv.org/pdf/1712.04741' which redirects with HTTP 302 to 'arxiv.org/pdf/1712.04741.pdf'. It seems to be an issue in webkit_web_view_download_uri but also calling webkit_web_context_download_uri direct does not honor the redirects.

[fanglingsu]

To fix this issue we have to find a way to

1. tell webkit to download a given URI to an also given destination path
2. allow webkit to solve or follow redirects for us.

The webkit_web_vie_download_uri seems to assume that the URI is the final URI so that the redirect is the same like file not available.

Maybe we can use webkit_web_view_save_to_file, but this does generate a self-contained format which includes other external sources too which mit not be what the user expects.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity within the last 60 days. It will be closed after 7 days if no further activity occurs. Thank you for your contributions.