wkhtmltopdf_binary_gem
wkhtmltopdf_binary_gem copied to clipboard
zakird
Gem size 0.12.5
new version of gem much bigger than previous one
0.12.5 - November 07, 2019 (238 MB) 0.12.4 - June 11, 2018 (45.1 MB)
0.12.5 totally unreal use it on heroku
Looking at the merge commit, the cause seems to be that we lost the generic builds (like bin/wkhtmltopdf_linux_amd64) for distribution-with-version-specific ones (like bin/wkhtmltopdf_ubuntu_14.04_amd64.gz).
There are 6 builds for ubuntu only…
If this specificity is really necessary, I guess that the install process should wget the correct file only, and it should not be part of the gem directly. Otherwise this is just going to get worse with new distributions and versions, and we'll have a >1Go gem.
It would also be nice to leave a generic file for unofficially supported distributions. I mean, I'm on ubuntu 19.04, I had to link to the binary for ubuntu 18.04 and it works fine.
Sorry for not proposing a solution as a pull request, I don't know how to do this exactly and can't put this as a high priority. I'll downgrade.
@vitalinfo Did 0.12.4 work for you on Heroku? Which stack? Does the size of 0.12.5 prevent it from being installed on Heroku? Is there a reason you aren't using https://github.com/rposborne/wkhtmltopdf-heroku for a targeted build?
@fonji I'm not excatly sure why they decided to release multiple versions instead of a generic x86 and x64 fat binary as they have until this version, but I'm sure they have their reasons. I like your suggestion to wget the proper file, but worry that might be difficult to handle errors or assume every system has wget. Why is the size difference enough of a reason to downgrade for you?
Heroku allows to deploy applications whose weight is maximum 500 MB Having one single gem add 200MB prevents us from deploying (which is enough of a reason to downgrade ;) )
@unixmonkey yes, 0.12.4 works fine on Heroku for us
our current slug size 300+Mb, but with 0.12.5 it grows more than 500+Mb
and basically it pretty obvious why according to these numbers
+200MB on a patch version dooms the version unusable
@vitalinfo Did 0.12.4 work for you on Heroku? Which stack? Does the size of 0.12.5 prevent it from being installed on Heroku? Is there a reason you aren't using https://github.com/rposborne/wkhtmltopdf-heroku for a targeted build?
@fonji I'm not excatly sure why they decided to release multiple versions instead of a generic x86 and x64 fat binary as they have until this version, but I'm sure they have their reasons. I like your suggestion to wget the proper file, but worry that might be difficult to handle errors or assume every system has
wget. Why is the size difference enough of a reason to downgrade for you?
wkhtmltopdf-heroku won't work on macOS development enviroment and we understand it wouldn't be a good practice to have different gems for development and production enviroment. This is, so far, the only gem we have'd this problem
@vitormd I disagree that it "dooms the version unusable". I know that it excludes some people due to size and system limitations, but many have been waiting a long time for this update and are happy to have it.
I also disagree that you shouldn't configure your setup to use a different gem for development and production environments, as long as they are the same version of wkhtmltopdf, it shouldn't matter. You are already using different binaries because of the differing platforms, even if they are packaged into the same gem.
This gem merely wraps the wkhtmltopdf released packages, which changed from having a few "universal"-style linux binaries in 0.12.4, to having multiple distribution-specific ones in 0.12.5.
This gem preserves the behavior of prior versions by bundling everything together, whether you need it or not, for convenience, and executing the one that works on your system.
I certainly wish it were smaller, but which binaries should we exclude to make it so?
What's stopping you from pulling out the 2 binaries you need and dropping them into your own project?
I think it might be possible to build our own statically-compiled fat binary for the linux systems, but it would be difficult for me to know for sure it works across all distributions.
I'd happily consider at a PR or in-depth explanation of how to improve the situation.
I'm using AWS Elastic Beanstalk. Application use "amzn_2018.03_amd64" SO. When I use 0.12.5 gem version, log said:
Invalid platform, must be running on Ubuntu 14.04/16.04/18.04 CentOS 6/7, Debian 8/9, or intel-based macOS (missing binary: /opt/rubies/ruby-2.3.8/lib/ruby/gems/2.3.0/gems/wkhtmltopdf-binary-0.12.5/bin/wkhtmltopdf_amzn_2018.03_amd64). (RuntimeError)
So, I downgrade to 0.12.4 gem version
I get this error on 0.12.5:
expected no Exception, got #<RuntimeError: Failed to execute:
9228 ["/usr/local/bundle/bin/wkhtmltopdf", "--margin-top", "0", "--marg...r/local/bundle/bin/wkhtmltopdf:23:in `load'
9229 from /usr/local/bundle/bin/wkhtmltopdf:23:in `<main>'
I rolled back to 0.12.4 and it works.
I certainly wish it were smaller, but which binaries should we exclude to make it so?
All of them. The gem should download the single one actually necessary for the system during the installation, and leave the others out. This would also shrink the gem size to a few kb only, although a note would be welcomed in the Readme.md about the real size, possibly an output message at the end of the bundler run.
Btw, for people having problem with size on heroku: https://github.com/rposborne/wkhtmltopdf-heroku Just went down from 450MB to 180MB by moving to this gem, which installs only the one necessary for heroku.
@unixmonkey @vitalinfo I think that this gem should behave like the webdrivers gem. Currently is unusable for us due to space limitation.
@unixmonkey I think we can close this issue, because we don't have immediate plans to make this kind of change in gem structure for now 🤝
Anyone using packaged/vendored gems with Bundler also hits the issue that this gem's size now requires the use of Git/GitHub LFS (large file storage).
Hey! This article helped me with the same problem https://www.eyupatis.com/how-to-decrease-heroku-slug-size-wkhtmltopdf-edition
Hi. I always make use of this gem. I thank the maintainer.
Now, I'm also facing this problem and would like to discuss a mitigating solution.
The mitigation I propose is to cut support.
This gem currently support many distributions, but some of which are already EOL.
- debian 10 went EOL on 2022-09-10
- ref. https://wiki.debian.org/DebianReleases
- ubuntu 16 went EOL in April 2021 and ubuntu 18 will go EOL in April 2023
- ref. https://wiki.ubuntu.com/Releases
- centos 6 went EOL in May 2017, 8 went EOL in December 2021, and 7 will go EOL in June 2024
- ref. https://endoflife.date/centos
Now, if we exclude the EOL ones (but keep the latest ones for debian and centos), we can reduce the size by about 110MB
# before
❯ du -h -d 1 | grep bin
263M . /bin
# after
❯ du -h -d 1 | grep bin
159M . /bin
Also, we are aware that i386 usage is relatively low at this moment, and if we exclude them, we can reduce it by another 50 MB or so
❯ du -h -d 1 | grep bin
110M . /bin
I know this is not a perfect solution, but I believe it is a mitigation for many cases!
Is it possible to include this mitigation?
Also, the suggestions so far have been mitigations by cutting support, but we may as well consider mitigations by providing a per-platform gem like the nokogiri gem.
FWIW, you could do...
# Install the gem
bundle
# Execute the binary once so that it decompresses the correct OS-based binary
bundle exec wkhtmltopdf --version
# Remove all the compressed binaries afterwards
rm -rf $(bundle show wkhtmltopdf-binary)/bin/*.gz
You can put the above in your Dockerfile so that your final image is smaller. At the end of the day, I think this gem could leverage arch-based distributions that some other gems use:
https://rubygems.org/gems/nokogiri or https://rubygems.org/gems/sqlite3/versions
Bundle will install the right gem variant/release based on your system arch. Each one would house just 1 binary.
