surveys icon indicating copy to clipboard operation
surveys copied to clipboard

More objective metric for resources selection

Open SachaG opened this issue 2 years ago • 11 comments

Problem: current resources lists are too arbitrary.

Solutions:

  • Use traffic as an independent metric to rank resources instead of/in addition to previous-years mentions in the survey.
  • Show more options (possibly with a "load more options…" button?)

Other possible metrics:

  • social media mentions?
  • mentions in places where people might list their education? resumes? LinkedIn?

SachaG avatar Oct 11 '22 02:10 SachaG

Great resource to start with: https://www.classcentral.com/providers

SachaG avatar Oct 11 '22 02:10 SachaG

Hi @SachaG !

I think the "official list" is a bit arbitrary yes. However, combined with the "free form answers", it does cover most of the relevant JS learning resources on the web today. The only two additions I would include from ClassCentral are Coursera and Skillshare, as both are big on JS content.

As for the ranking, I think it's important to use an objective traffic measure. The best one is probably SimilarWeb's Global Traffic Rank (GTR). In this sheet, I have listed rankes the sites using the GTR: https://docs.google.com/spreadsheets/d/13PNP_t8ElHb8hoYZbcwQjn-jYzUTnbxh4isOe1D8n7Y/edit?usp=sharing

Note: for some sites, their Global Traffic Rank is too affected by non-JS-related-traffic in order to be relevant. I have moved these down towards the bottom (if they get voted to the top, then you can consider moving them up next year). The exceptions are Udemy and YouTube, as both clearly should be included in the "main list" IMHO, given their popularity.

perborgen avatar Oct 11 '22 11:10 perborgen

Super helpful, thanks!

SachaG avatar Oct 11 '22 21:10 SachaG

I think maybe we should just have a separate question for YouTube/Twitch actually, and free up one spot in this list.

SachaG avatar Oct 12 '22 00:10 SachaG

So here is the revised list, using a mix of the GTR rankings and the previous years' State of JS data. I left off YouTube because I want to give it its own category, and Reddit because I feel it doesn't quite fit the "learning" category as much as the "news" one.

Main List

      - id: stack_overflow
      - id: udemy
      - id: w3schools
      - id: mdn
      - id: freecodecamp
      - id: codecademy
      - id: web_dev
      - id: javascript_info
      - id: scrimba
      - id: frontendmasters
      - id: wes_bos
      - id: platzi

"Show More" List

      - id: skillshare
      - id: coursera
      - id: egghead
      - id: fireship
      - id: kent_c_dodds
      - id: o_reilly
      - id: pluralsight
      - id: linkedin_learning
      - id: pluralsight
      - id: designcode
      - id: academind
      - id: ui_dev
      - id: vue_mastery
      - id: newline
      - id: epic_react
      - id: midu_dev
      - id: levelup
      - id: doka

SachaG avatar Oct 12 '22 00:10 SachaG

Totally agree that it makes sense to move YouTube into its own category @SachaG . The revised list looks great IMO!

perborgen avatar Oct 12 '22 08:10 perborgen

I can only comment on LinkedIn Learning since that's what I have numbers for. We have front-end courses including CSS, JavaScript, HTML, React.js, etc with +300,000 active learners, and our front-end web library sees tens of thousands of active learners per month.

Some examples:

I agree for large platforms like ours, GTR is irrelevant, but that doesn't warrant relegating the platform to a secondary list. Instead, it means a different metric is needed. Most major learning platforms have public stats available on request and these could be used for ranking.

If a large learning platform doesn't show up in the results from the "other - please explain" option, it is likely due to a combination of "out of sight, out of mind," and the surveys not reaching an entire section of the industry.

mor10 avatar Oct 12 '22 17:10 mor10

@mor10 thanks for the data! I personally don't have the time to get in touch with every platform on the list and ask them for their numbers, but if a third party like maybe https://www.classcentral.com/ could do that and establish some kind of objective ranking, I'd be happy to use that instead.

As for the surveys not reaching an entire section of the industry, that's definitely an issue but I'm afraid it doesn't have any easy solution…

SachaG avatar Oct 12 '22 22:10 SachaG

Just picking a random item on your primary list: what are the viewer and user numbers for Platzi? Are they more than 200,000 for a single course? If not, do you see how omitting a large platform because it also covers other focus areas is sub-optimal?

mor10 avatar Oct 12 '22 22:10 mor10

Solving this issue likely means reformatting how you're grouping the options. Maybe split them into platform documentation (men, etc), blogs (CSS tricks), video learning (Udemy), other and have several questions. As long as you have a limited list you won't get reliable data. Whomever is on the primary list will always get the bulk of the votes, and if the primary list excludes entire platforms, your results are invalid.

mor10 avatar Oct 12 '22 22:10 mor10

I think splitting up the items into more questions kinda ends up being the same as showing all items in fewer questions… I don't have a perfect solution to all this, but if we can already iterate towards something better than what we had before, I think that's a win.

SachaG avatar Oct 13 '22 04:10 SachaG