traveling-ruby
traveling-ruby copied to clipboard
phusion
Which native extensions should we package?
In issue #3 I have described how I envision support for native extensions to work. Since requiring developers to build native extensions themselves is too much hassle, I proposed providing precompiled versions of the most popular native extensions, that developers can just easily drop into their package.
But this raises the question: which native extensions? We can't automatically package them all. For reasons explained in #3, packaging native extensions requires a lot of manual work. So we have to choose.
I plan on packaging these gems:
Because they're popular:
- nokogori
- json
Because Rails apps likely need these:
- sqlite3
- mysql2
- pg
- bcrypt
I've opened a poll for more native extensions. Please add your input here: https://docs.google.com/spreadsheets/d/1oNgrwHyV9UaMGFTttsfA9QPvL461NWLfHeeAap3Az_A/edit?usp=sharing
This gist seems interesting to list all your gems with native extensions :
require 'rubygems'
specs_with_extensions = Gem::Specification.each.select { |spec| spec.extensions.any? }
specs_with_extensions.each do |spec|
puts "#{spec.name} (extensions: #{spec.extensions.inspect})"
end
(source https://gist.github.com/aelesbao/1414b169a79162b1d795)
P.S: I don't know if this is wanted, but your Google Doc is restricted.
The doc is supposed to be public. I've fixed that and updated the link.
I know I can lookup all gems with native extensions, but I cannot package all gems because it's too much work. The best thing I can do is package the most popular ones, hence the poll.
@FooBarWidget Can't packaging be automated for any gem? Do you need to choose?
No it cannot be automated. See https://github.com/phusion/traveling-ruby/issues/3#issuecomment-66348237 where I said:
I think it is impossible to generalize and to automate this. The current Traveling Ruby binaries are hand-tweaked to ensure that they are portable and have no dependencies on non-system libraries. I think it is impossible to guarantee this for gems: to make the resulting binary portable would probably require a lot of manual work.
Packaging nokogiri and sqlite already took me a lot of manual work. I had to install libxml, libxslt and libsqlite3 from source, compile them with very specific settings, edit Makefiles everywhere to ensure correct linking flags, and do this twice (once for Linux, once for OS X).
So yes, I have to choose.
The set above includes my requirements for BOSH CLI https://github.com/cloudfoundry/bosh/blob/master/Gemfile.lock thanks!
@drnic Thanks, but can you help me by filtering out the gem names that contain native extensions, and modifying the spreadsheet? :)
@sheerun Ruby 2.1 has Fiddle: http://ruby-doc.org/stdlib-2.1.0/libdoc/fiddle/rdoc/Fiddle.html Doesn't that make ffi obsolete?
I was linking for completeness. I think nokogiri, mysql2, sqlite, pg are the native gems
On Wed, Dec 10, 2014 at 8:48 AM, Hongli Lai [email protected] wrote:
@drnic Thanks, but can you help me by filtering out the gem names that contain native extensions, and modifying the spreadsheet? :)
Reply to this email directly or view it on GitHub: https://github.com/phusion/traveling-ruby/issues/6#issuecomment-66482464
Also, gems that want to be compatible with Ruby versions before 2.1 still have to use FFI for awhile. I also have no idea where JRuby is on implementing Fiddle, which may matter for compatibility as well.
@Nowaker Glad you like it. :) I'm compiling a list of Traveling Ruby users. Could you write a short ~50 words introduction about Virtkick, and could you write another ~50 words paragraph in which you describe how Traveling Ruby has helped you?
@FooBarWidget Sure, no problem!
VirtKick in 50 words: VirtKick is a self-hosted cloud panel, similar to DigitalOcean. With privacy in mind we simplify creating, managing, hosting and providing virtual servers. Forget about installing packages or editing configuration files - all is set up automatically and ready to use. 100% open source.
VirtKick + Traveling Ruby: @Rush will provide a nice text, he did the hacking. :)
@FooBarWidget https://github.com/cloudfoundry-community/traveling-bosh is using it too
BOSH is a Cloud Foundry project for release engineering, deployment, and lifecycle management of large-scale cloud software. BOSH can provision and deploy software over hundreds of VMs and performs monitoring, failure recovery and software updates with zero-to-minimal downtime.
@FooBarWidget: Here is how traveling-ruby helps VirtKick:
VirtKick's goal is a one click installation via standalone installer and/or a distro package. We use rails and other dependencies, many of which we have at bleeding-edge versions from git. traveling-ruby bundles it all. Without it, the installation process was long, required a network connection and lots of hard dependencies.
@Rush @drnic Thanks. :) Could you also provide photos, icons or pictures that I can use? Should be at least 256x256 pixels.
@FooBarWidget Some logos are here: https://press.virtkick.io/presskit. Let me know if it's enough, or you need something more.
@Nowaker Looks good. Should I quote you or @Rush? What are your positions within your organization?
I've added your testimonials to the website, but I've edited them a little bit: http://phusion.github.io/traveling-ruby/ Is this ok?
Is it possible to include https://github.com/brianmario/yajl-ruby?
Sorry I didn't spot it in https://github.com/cloudfoundry/bosh/blob/master/Gemfile.lock#L31 before as a native extension.
Specifically yajl-ruby (~> 1.1.0) (perhaps in addition to 1.2.1)
See other issues - I'm struggling to build this myself on both osx & linux - sorry.
I can only package a single version. Maintaining two versions is too much of a burden.
Sent from my Android phone.
@FooBarWidget ok, the latest yajl-ruby is ok if that's all that's possible. I'll try to work it in upstream. TIA.