Compiled extension has exploded in size in recent versions

[mvz]

trafficstars

It seems the compiled extension has become a lot bigger recently. This is becoming a problem on my VPS. Here's the size on my development machine for some recent versions:

12804	sassc-2.0.1/ext
74344	sassc-2.2.1/ext
155156	sassc-2.3.0/ext

What could be going on and how can I reduce this size?

[mvz]

I think the relevant change was 82caf8e6e457796a767e59c773db7b301c2210a4

[mvz]

I think what happened is that the commit I mentioned above made sassc-ruby use mkmf.rb, which picks up $CXXFLAGS form rbconfig.rb, which sets it to -g -O2 on my machines. It's the -g flag that makes the .o files blow up.

[mvz]

I should note that I have not consciously set -g. I just use ruby-install.

[jaredbeck]

I also noticed this recently. The sassc gem is 101 MB now. The majority of our other gems are < 1 MB.

du -sh vendor/bundle/ruby/2.6.0/gems/* | sort -hr
101M	vendor/bundle/ruby/2.6.0/gems/sassc-2.4.0
16M	vendor/bundle/ruby/2.6.0/gems/sorbet-static-0.5.5848-x86_64-linux
8.3M	vendor/bundle/ruby/2.6.0/gems/ffi-1.13.1
...
40K	vendor/bundle/ruby/2.6.0/gems/dotenv-rails-2.7.6
12K	vendor/bundle/ruby/2.6.0/gems/rails-6.0.3.2

This is a problem for us on Heroku, where "slug" size is important.

[xinwelcome]

This is a real problem on the slug size. Any chance it will be fixed?

[timon]

This seems to be related to a bunch of .o files lying around after compiling the extension. (the issue above says the same)

[timon]

@xinwelcome @jaredbeck I have ended up with the following workaround:

# lib/tasks/heroku.rake
# This file was heavily influenced by https://blog.saeloun.com/2020/05/04/how-to-reduce-heroku-slug-size.html
namespace :heroku do
  desc 'Removes extra .o files from native extension builds'
  task :clean_gem_artifacts do
    Bundler.bundle_path.
      glob('**/ext/**/*.o').
      each(&:delete)
  end
end

if Rake::Task.task_defined?('assets:clean')
  Rake::Task['assets:clean'].enhance ['heroku:clean_gem_artifacts']
else
  Rake::Task.define_task('assets:clean' => 'heroku:clean_gem_artifacts')
end

[nevinera]

This is also contributing to a problem in distributed CI workflows, where disk workspaces need to be decompressed on fairly small containers - the total size of our installed gems directory (the bulk of that workspace) is up to 660MB, and this is one of the largest contributors to that.

I'm not following the MakeMakefile logic, but it should be straightforward to clean up the .o files after building the .so files, right? It looks like there's some provision for it under MakeMakefile::CLEANINGS, but I can't find any documentation on how to use that..

[anthonyamar]

Hello, I'm facing the same problem with sassc-2.4.0 at 161M. It's the biggest gem I have in my bundle and cause Heroku stop allowing me to deploy my app because of the slug size.

The workaround didn't make the job for me, it doesn't run during the build. Even deleting everything manually didn't work to reduce de slug size during build.

Any chance you'll put the lib on a diet soon? It's a very big problem, it causes at least 30% of the slug size and Heroku is very strict with those 500M max.

[timon]

@anthonyamar I've noted a typo in the snippet I posted before and updated it.Perhaps this helps?

[anthonyamar]

@timon Thanks! Yes, what's the typo? How do you launch the script before Heroku slug compression?

[x-yuri]

@anthonyamar Click on the word "edited" on the post, and you'll see a list of revisions. Then click on the last revision.

[chrishough]

@timon that hack worked for me, but from what I can tell only shaved off about 20ish MB

[josephecombs]

For old versions of ruby without access to Pathname#glob, you can use instead:

Dir.glob("#{Bundler.bundle_path}/**/ext/**/*.o").map{|path| File.delete(path) }