Tar download from HEASARC hanging?

[matteobachetti]

When I use Heasarc.download_data with host=heasarc, I see an empty tar file being created and then the query hanging for a long time. Is it the expected behavior?

[keflavich]

This could be related to other HEASARC services turning off http access. Could you check if changing http to https solves the issue?

[bsipocz]

Cc @zoghbi-a as I don't expect any such issues with heasarc itself, the module just got a full rewrite recently.

[zoghbi-a]

This is because of the recent server overload issues. I think the server response has been better in the last two days. Do you still see them @matteobachetti?

Do you have an estimate of big the file is? If it is large (> 5-10 GB), you can try breaking the call into smaller calls, or switch to using host='aws'. You don't have to be running on aws to do that.

[matteobachetti]

@keflavich @zoghbi-a I was using the link provided by Heasarc.locate_data, for a single NuSTAR obsid which should be around 500 MB. I understand the HEASARC overload issue, I will use S3 whenever possible.

[bsipocz]

Do we have that workaround mentioned in the docs? If not, maybe we should.

[matteobachetti]

BTW, would it be possible to have a mechanism to restart an interrupted download? Maybe not for the tar file, but for the S3 download (that works on single files) it should be pretty straightforward