directory
directory copied to clipboard
Use Travis CI to download Rubygems data for listed gems
The following Rake task takes the directory listing (_data/themes.yml
and _data/plugins.yml
clubbed into a single _data/entries.yml
) and downloads each gem into their respective directory
# frozen_string_literal: true
require "rubygems"
require "rake"
require "open-uri"
require "safe_yaml"
task :default do
entries = SafeYAML.load_file("_data/entries.yml")
entries.each do |type, list|
FileUtils.mkdir_p("_data/#{type}")
list.each do |item|
begin
data = open("https://rubygems.org/api/v1/gems/#{item}.yaml").read
File.open("_data/#{type}/#{item}.yml", "wb") { |file| file.puts data }
rescue OpenURI::HTTPError
puts "#{item} does not seem to exist at rubygems.org"
next
end
end
end
end
The data files can then be used to define the front end.. We can then set up Travis to download these files at a set interval.. Thoughts?
How would _data/entries.yml
get created?
Also, in order for this to work, Travis would have to commit these back to GitHub, would it not? I'm not sure that's something they would support doing. I'm also not sure if I've understood your idea correctly. Let me know if I've got it wrong.
How would
_data/entries.yml
get created?
If this idea gets approved, I'll open a PR that clubs existing _data/themes.yml
and _data/plugins.yml
files into _data/entries.yml
and add the rake task to this repo as well. In future, any user desiring to get their plugin listed need to open a PR adding their gem / plugin-name / theme-name to _data/entries.yml
Travis would have to commit these back to GitHub, would it not?
Yes, Travis CI can be used to do this.. Its documented. My plan is therefore:
- set up Travis to commit back to this repo. (I've already run trials on my fork). Perhaps @parkr can lend a hand at adding a secure access token for this repo..)
- instruct Travis to:
- download gem data from Rubygems.org API if available
- use this data to build the production-ready site
- push the generated site back to
gh-pages
branch on this repo (default) - set up cron jobs to have this automated at set intervals
- update Readme document here to ease contribution and development
I'm onboard with this. :+1:
Out of curiousity, what drove the idea behind creating a single file vs. keeping the two files and just smashing them together as part of the rake task?
👍 Interesting, right now we have manually added documents, this would allow us to display more info like latest version, author, downloads, link to changelog, etc. 😄
Downloads could be use as a sort filter.
{
"authors": "Mark Otto",
"bug_tracker_uri": null,
"changelog_uri": null,
"runtime": [
{
"name": "jekyll",
"requirements": "~> 3.6"
},
{
"name": "jekyll-feed",
"requirements": "~> 0.9"
},
{
"name": "jekyll-gist",
"requirements": "~> 1.5"
},
{
"name": "jekyll-paginate",
"requirements": "~> 1.1"
},
{
"name": "jekyll-sitemap",
"requirements": "~> 1.1"
}
]
},
"documentation_uri": "http://www.rubydoc.info/gems/jekyll-theme-poole/2.1.0",
"downloads": 857,
"gem_uri": "https://rubygems.org/gems/jekyll-theme-poole-2.1.0.gem",
"homepage_uri": "https://github.com/poole/poole",
"info": "The Jekyll Butler, designed and developed by @mdo to provide a clear and concise foundational setup for any Jekyll site.",
"licenses": [
"MIT"
],
"metadata": {
"plugin_type": "theme"
},
"name": "jekyll-theme-poole",
"project_uri": "https://rubygems.org/gems/jekyll-theme-poole",
"version": "2.1.0",
}
what drove the idea behind creating a single file vs. keeping the two files and just smashing them together as part of the rake task?
Easier to code to load a single YAML file and proceed.., as against reading two files and merging data or clobbering the two files into one and then loading that.... :smile:
allow us to display more info
yes.. though we still need the manual data in sync with showcasing themes and plugins that have not been gemmified..
though we still need the manual data in sync with showcasing themes and plugins that have not been gemmified..
I think this directory should only include plugins and themes that are available as Gems
This issue has been automatically marked as stale because it has not been commented on for at least two months.
The resources of the Jekyll team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master
branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, please consider whether it can be accomplished in another way. If it cannot, please elaborate on why it is core to this project and why you feel more than 80% of users would find this beneficial.
This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.
No! This is not going to die due to inactivity!
What do we need to do to carry this across the finish line?
This issue has been automatically marked as stale because it has not been commented on for at least two months.
The resources of the Jekyll team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master
branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, please consider whether it can be accomplished in another way. If it cannot, please elaborate on why it is core to this project and why you feel more than 80% of users would find this beneficial.
This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.
What do we need to do to carry this across the finish line?
This issue has been automatically marked as stale because it has not been commented on for at least two months.
The resources of the Jekyll team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master
branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, please consider whether it can be accomplished in another way. If it cannot, please elaborate on why it is core to this project and why you feel more than 80% of users would find this beneficial.
This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.
Is there a PR for this?
Nope. I had one somewhere around in Feb, but lost it to a corrupted Git index. Now I don't remember what exactly I planned on doing at the time..
This should be fairly straightforward of a process: list all gems, hit the rubygems metadata endpoint for each gem (costly, do once per month?), write yaml, commit. I’d can get a deploy key for this repo from @jekyllbot, but want to make sure it can only be used by Travis’s scheduler instead of any person running a Travis build.