aospy
aospy copied to clipboard
spencerahill
Add section on parallelized calculations to docs
C.f. https://github.com/spencerahill/aospy/pull/172#issuecomment-292574427 and subsequent comments in that thread.
I think it's important to provide some guidance within our docs on how to e.g. use dask's status page and/or progressbar. This could be pretty brief, ultimately referring folks to distributed's own documentation on this stuff.
<continuing convo from https://github.com/spencerahill/aospy/pull/172#issuecomment-295032793>
What about adding an option e.g. open_status_page that automatically opens the status page in their browser (e.g. using the builtin webbrowser)?
this would open a browser on the remote machine (which one would need to use X11 or VNC to view, both of which can be slow).
Good point. Does your recent ssh wizardry we've discussed offline come to bear on this at all?
One other thing to think about is that it would also require a little extra logic to make sure it directed the user to the proper URL (one does not need to always use port 8787 for the diagnostics server, so for an external client the URL wouldn't always be http://localhost:8787).
Could we do something like have open_status_page accept either False, True, or a port number to enable directly specifying the port?
Good point. Does your recent ssh wizardry we've discussed offline come to bear on this at all?
Unfortunately no -- I don't have an idea for how to programmatically open a user's local browser from a remote machine.
Could we do something like have open_status_page accept either False, True, or a port number to enable directly specifying the port?
Yes, I think that would be a good solution.
Yes, I think that would be a good solution.
Ok, let's plan on implementing this then.
Unfortunately no -- I don't have an idea for how to programmatically open a user's local browser from a remote machine.
Ok, good to know. I don't think this is a deal breaker, but we should address it. What comes to mind for me is a prominent warning in the docs about this. We could potentially warn in the terminal also, but my instinct is that that's overkill. Although don't feel too strongly either way.
I would also have this off by default -- the user should have to specify they want it.
@spencerkclark I think your summary here https://github.com/spencerahill/aospy/issues/236#issuecomment-345487779 of how to use dask-scheduler to initiate multiple dask workers and your own Distributed scheduler would be a great addition to the docs.