community
community copied to clipboard
dask
Regular Online Tutorials
As a community service we could offer online tutorials hosted by one of the Dask maintainers.
The tutorial at https://tutorial.dask.org was designed to run in a three-to-four hour period (for SciPy). We could offer this, or some stripped down version of it as an online web video call. Hopefully it's relatively straightforward to deliver, and also a good learning experience for those involved in giving it.
My hope is that as more of us deliver this tutorial we then are able to refine and improve the tutorial over time. Preparing, delivering, and then tuning the tutorial afterwards would, I think, be a high value maintenance activity. I think that we may have the bandwidth to start delivering the tutorial somewhat regularly.
We could try using Twitch and/or YouTube streaming for this, that way they can be archived afterwards and watched back later. Many conferences that have moved online are using these services and it seems to work quite well.
I've been experimenting this week with streaming bits of my open source maintenance work on Twitch and it was pretty easy to get started.
This sounds fun.
One question around infrastructure & format though: do you envision attendees following along live and running the examples, asking questions, etc.?
In the past we've temporarily set up a cluster with jupyterhub & Dask so that attendees could play with a real cluster. Would we attempt the same thing here, or would we just have attendees run things locally? My only concern is around who would pay for the cluster computing resources. I'm not too worried about the technical side of things. It's pretty easy to set up a Jupyterhub & Dask Gateway on a kubernetes cluster these days.
Tom
On Fri, May 22, 2020 at 4:18 AM Jacob Tomlinson [email protected] wrote:
We could try using Twitch and/or YouTube streaming for this, that way they can be archived afterwards and watched back later. Many conferences that have moved online are using these services and it seems to work quite well.
I've been experimenting this week with streaming bits of my open source maintenance work on Twitch and it was pretty easy to get started.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-632589937, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIUNTKJ62G3V3XOG4FTRSY7MTANCNFSM4NG6DARQ .
It seems like there are two approaches to video tutorials
- Broadcast with twitch/youtube
- Video conference
I think that they're both useful. It seems like twitch/youtube support more viewers, but don't allow for as much engagement. My guess is that they're better if we're doing new content every time. Video conferencing might be better if we have smaller groups (say 10-20) where the students are going through exercises on their own and being coached by community members. This seems better to me if we do the standard tutorial on a regular basis.
My only concern is around who would pay for the cluster computing resources
I think that the Dask/NumFOCUS account could be used if we're looking at something in the few hundred dollars range.
In the past we've temporarily set up a cluster with jupyterhub & Dask so that attendees could play with a real cluster. Would we attempt the same thing here, or would we just have attendees run things locally?
I recently gave the tutorial using the binder setup at tutorial.dask.org, and found it good enough to teach people how dask works and how to think about it. Yes, they didn't get a real "distributed" cluster, but they still got to see how dask's api works and learned how to reason about the various concepts.
@mrocklin yeah I agree with that assessment. I think Twitch/YouTube would be good for doing something like a talk. Especially if the recording would be useful afterwards. But then video conference would be good for a more workshop style session.
If we have human resources, I'm inclined to try both and see what engages people.
If I were to organize this I would suggest the following:
- We schedule two online tutorials, one for two weeks from now and one for two weeks following. We get ~suckers~ volunteers to sign up to deliver these.
- On alternating weeks we also get a couple of people to sign up to give a Twitch-style talks about some topic like "all the ways to deploy Dask on Kubernetes" or "How to use Dask with Xarray"
I'll volunteer for the second round :)
On Fri, May 22, 2020 at 11:32 AM Matthew Rocklin [email protected] wrote:
If we have human resources, I'm inclined to try both and see what engages people.
If I were to organize this I would suggest the following:
- We schedule two online tutorials, one for two weeks from now and one for two weeks following. We get suckers volunteers to sign up to deliver these.
- On alternating weeks we also get a couple of people to sign up to give a Twitch-style talks about some topic like "all the ways to deploy Dask on Kubernetes" or "How to use Dask with Xarray"
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-632786283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOITRC2VBAXE2B3HDGVLRS2SIBANCNFSM4NG6DARQ .
Happy to be a sucker, although I would like to rotate off Czar duties before that time so that I can spend that time preparing.
If your Czar duties are tracking dask/distributed then I'm happy to do extra work to allow you to go be a sucker on this one :) I'm excited to see if this is something that we can start doing regularly.
On Wed, May 27, 2020 at 5:42 AM Jacob Tomlinson [email protected] wrote:
Happy to be a sucker, although I would like to rotate off Czar duties before that time so that I can spend that time preparing.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-634633612, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTDZ5E2ZG4X53L5WVM3RTUDB3ANCNFSM4NG6DARQ .
Ok sounds good.
two weeks from now
From when you sent that message that would be Friday the 5th of June.
Do you have thoughts on a time? I wonder whether we should try and do it at a time like 2pm, which would be practical for folks in immediately neighbouring timezones. Perhaps trying to alternate the presenter between continents week-to-week to be as inclusive as possible.
I think that we'll want at least a weak of lead time to advertise and prep. +1 on switching around time zone availability to support a broad set of attendees.
Ok given that time is passing I suggest we go for Friday June 12th at 1pm UTC. That'll be 9am east coast US, 2pm for me and 3pm central Europe, so should have good coverage. I also suggest we pencil in 90 minutes for it.
Then maybe next time someone in North America could run it to get coverage for west coast?
I think we should also use Zoom for this first one. Do we have access to a Zoom pro account for avoiding the time limit and doing recordings?
@mrocklin are you able to coordinate promoting it?
I think we should also use Zoom for this first one. Do we have access to a Zoom pro account for avoiding the time limit and doing recordings?
Yes, through NumFOCUS. I've sent you and support@numfocus an e-mail
@mrocklin are you able to coordinate promoting it?
Definitely. The obvious things to do is tweet it. I can see if there are a couple of other channels.
I taught a 4-hour workshop on oreilly.com focused on Dask Dataframes and teaching it again at the end of July. It alternates between lecture/exercise. May be up for doing a volunteer teaching session or integrating these into the docs. Here's a link to the notebooks: https://github.com/dgerlanc/dask-scaling-dataframe.
Not currently. We have some work to do around evangelizing this. @jacobtomlinson what would you prefer? We could do something like Evite to rate limit attendees (probably not a problem) and then send out the address to those people a couple of days before. Alternatively we could just blast out a public address.
I'm ok with either. I might lean slightly towards Evite just because it's different and we might learn something. I'm totally happy to defer to you as the person running this though.
On Wed, Jun 10, 2020 at 12:52 PM Julia Signell [email protected] wrote:
Is there an invite that I can share?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642222304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTHUHCQPH76KZ6CGDPTRV7P6LANCNFSM4NG6DARQ .
Yeah I think having people register would be good. We could use something like Eventbrite?
Yes, that's what we used for the Dask Summit and it worked well
(I said Evite before, but I think I meant Eventbrite :/ )
On Wed, Jun 10, 2020 at 2:07 PM Jacob Tomlinson [email protected] wrote:
Yeah I think having people register would be good. We could use something like Eventbrite?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642264736, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTAO7CV7GPB2EBDUKF3RV7Y23ANCNFSM4NG6DARQ .
This is still planned for tomorrow right? I am going to tell my work people about it :)
We're planning this for next Friday I think
On Thu, Jun 11, 2020 at 7:08 AM Julia Signell [email protected] wrote:
This is still planned for tomorrow right? I am going to tell my work people about it :)
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642680072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTD3FXAPUVN4IG5ZKVTRWDQOVANCNFSM4NG6DARQ .
@jacobtomlinson no, I don't currently have an account. I can put this on my TODO list for the day though.
On Thu, Jun 11, 2020 at 7:20 AM Matthew Rocklin [email protected] wrote:
We're planning this for next Friday I think
On Thu, Jun 11, 2020 at 7:08 AM Julia Signell [email protected] wrote:
This is still planned for tomorrow right? I am going to tell my work people about it :)
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642680072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTD3FXAPUVN4IG5ZKVTRWDQOVANCNFSM4NG6DARQ .
I had a bit of a play around with using OBS and Blender to make my video setup for the tutorials a little more professional.
I made a couple of scene transitions, they use the Dask logo and add a nice touch. Although they should be used sparingly (way less then I do in this example).
I also made some little elements that pop in to show more information to the viewers. I can share the Blender files, they are quite easy to edit, but I'm also happy to just bulk render out a load that folks can reuse when we run the tutorial again.
I think this would add a nice professional touch to the tutorials, but keen to hear thoughts from others.
Example video on YouTube
Ha! This is great!
Personally I prefer the fade over the wipe. My personal style tends to be understated though. The titles/twitter thing looked great as well.
I have to say I'm reallly enjoying you taking on the whole video evangelism role. I'm looking forward to seeing this progress.
Just watched it. Here’s my immediate thoughts:
-
The horizontal wipes are almost ok. Just feel a little fast. (On a Macbook 13”, I’d probably like them best with doubled duration.) The diagonal wipes are much too fast and feel very busy / even startling. Even after a few times and knowing the Dask logo very well I could not recognise any details and I felt kind of alerted. (Maybe the fact that the movement starts from above adds to this startling effect?)
-
The fading between different layouts felt very good. I like these a lot!
-
The info-box with name / twitter handle looks nice. But the movement is too much for me. I’d probably like them even more with a fade-in rather than the lateral movement.
I like the name/twitter as is and they feel very polished. But I agree that the diagonal wipe is probably too much. The stripes looked like construction tape to me which might be where the feeling of alert comes from. I thin k the horizontal wipe could be nice for continuity between tutorials, but I think it needs to slow down a bit in the middle if that's possible.
Nice work @jacobtomlinson.
ccing @lauralorenz, who does all our livestreams at Prefect, in case she has any OBS general advice.
Hi @jacobtomlinson, as @jcrist mentions I'm managing the livestreams at Prefect and also use OBS. You seem to have a good handle on it based on your example vid and your animations look really sleek IMHO! But I'm happy to swap stories some time :) My general setup on OBS for coding livestreams is
- using dynamic bitrate and streaming at 15 FPS and sometimes only up to 720p to reduce my encoding load per OBS' performance tips (IMHO 15 fps / 720p has been fine for screencasts + my little video feed in the corner, 1080p is a smidge nicer though)
- I'm now using a hardware encoder / pass-through so I can stream from a different computer then where I am demoing from since my little Macbook couldn't take OBS + docker + node (I'm using a Happauge HD PVR Pro 60, so far so good; as you may already know it's hard to get the more popular brands like Elgato as they are widely out of stock)
My recent new experiments have been about
- having distributed cohosts -- tl;dr they call in on hangouts and I use a browser source (be warned - browser source has some CPU performance issues) with some software audio mixing (blackhole on macOS) to get it into OBS and into my own headphones properly
- I'm experimenting with chat overlay to increase interactivity (but it comes with its own troubles and really depends on the content you are running too)
Ironically I was never a fan of video tutorials myself, but we have observed there is a subset of users who prefer them, and it has been really useful to pass along video links to new users in particular.
