django-pipeline
django-pipeline copied to clipboard
jazzband
Can we improve upon the sass compiler performance?
First off, thanks a lot for django-pipeline because I cannot imagine developing with django without it.
Here's the situation: Today I decided to switch a project to using sass and in development mode my requests went from taking ~30-40ms to 1.5 seconds. It's also no longer possible to use live reload tricks which let me adjust the css without reloading the page.
You might think 1.5 seconds is not a lot but this is a trivial setup at the moment with nothing but bootstrap 3.1 being pulled in but I don't think it will gain much more time. Even with a nearly empty scss file and no bootstrap it takes 1.3 seconds to load a page.
Just simply clicking around my app going page to page feels really slow and it's slowly killing me inside.
With debug false and after running collectstatic then things are fast like usual because the sass files have been pre-compiled to css (right?), so there's no problem here.
With rails on the same machine using sass and coffee script the same requests happen in 100ms or so in development mode and it's handling the scss/coffeescript conversion to css/js behind the scenes in between page views. It feels very seamless to develop against.
Are there any strategies we can implement to improve the sass compiling time in development mode? Technically it appears possible because it's not delayed in rails at all.
We need to change how pipeline compile files, both for development and production, something like #299, but with better use of cache, and how compilers works, maybe inspired by django-staticbuilder.
Sadly, I have very little time to do so :expressionless:
Are you saying pipeline is adding overhead in production per request, or are you only talking about the time it takes to run collectstatic?
No, in production pipeline is just using staticfiles to point to file in STATIC_ROOT, there is no overhead.
But for both development and collectstatic, we don't really make a good job at playing well with compilers.
Is the general improved strategy for development to somehow store the mtime of each sass file and only rebuild the css per request if the css has changed?
In the case of sass and others, it's more complicated, because of @import.
I'll take a look at rail's source tomorrow and see how they are doing it. Hopefully it's something I/we/community can apply to pipeline.
@AntelopeSalad Outside of rails, look at django-staticbuild and issue #299 (and the PR that goes with it).
I don't really know what to make from 299's PR, it's only a 1 line change?
The staticbuild's middleware uses a time comparison, but you said that won't work because of imports.
+1 and ideas:
- Is it possible to add a setting to django-pipeline for a directory to watch and, if all files in the directory are unchanged, don't recompile? I have all my scss files in my project's assets directory. Sure, editing one file would cause them all to be recompiled, but unless you're doing heavy styling work, this should solve the problem in all other cases. In any case, this would need to be done before starting Sass, as it seems loading Sass + all of its dependencies seems to be the key limiting factor
- Is it possible to keep Sass running as a daemon with the --watch option while the development server is running? That way, Sass would take care of itself automatically in this regard.
- Can we abuse the .sass-cache directories to get a list of files to "track"?
Another idea:
- If this is so much faster in Rails, how about a Rails app that runs alongside the Django one in development mode? Just proxy requests for stylesheet files over to it.
@AntelopeSalad They are incomplete but interesting solutions, #299 would make sass like compiler work almost like intended, but wouldn't work for development. django-staticbuild try do don't put that much intelligence in the process, leaving the compiler trying to be smarter.
@AntelopeSalad @charredUtensil Could you try your setup against #326?
P.S: The templatetags names have change.
@cyberdelia Tried running that. I see no performance benefit whatsoever from the 1.4.x branch. It's still forking tons of sass processes, four at a time, for each .scss file on every page load. On my main development box with RVM and lots of other Ruby stuff installed, sass takes a little longer to start each time. I also have a VM with a more plain Ruby installation that can start sass quicker, but this is less than ideal. I just dropped some other old Python packages with bizarre, obsolete dependencies, and am trying to get rid of the VM in favor of developing this app locally again, but I'm still seeing 10-20s page loads with django-pipeline.
@charredUtensil With this new version, @import should world, so you should be able to point at only at couple of files, avoiding starting too many multiple processes.
@cyberdelia Better, but it's still calling sass once on every request, even though I haven't modified any files.
Joining the conversation pretty late here but wanted to check in and see what you all have been doing for this.
@AntelopeSalad have you come up with your own solution or are you still dealing with super long requests in your dev environment?
I like @charredUtensil's idea number 2 regarding taking advantage of Sass's --watch option. Ideally: you save a change to a scss file, Sass automatically recompiles the CSS, and by the time you refresh your browser it's already sitting there ready to be loaded, right?
@cyberdelia Couldn't this be as simple as an optional setting that tells SASSCompiler not to run compile_file() when DEBUG is False? Only when this setting is enabled, it can optimistically assume the developer/user is running their own sass --watch and thus that the compilation is already done.
Thoughts?
@cyberdelia curious to hear your thoughts on this 6c95ac3e8157e2052d64722b7512ba0ebe63830a
Pretty simple/harmless, and it's allowing me to use sass with pipeline in my dev environment with 10x faster requests. In my dev settings only, I add PIPELINE_DISABLED_COMPILERS = ('pipeline.compilers.sass.SASSCompiler',). Pipeline still renders the output file source to my page, but it never actually runs the Sass compiler and assumes the file has already been compiled.
I have sass --watch running in another terminal so that whenever I change/save a .scss file it's automatically compiled.
And if I'm not working with CSS at the moment, I don't even have to worry about running that command. After all, why do I need pipeline to recompile my Sass files on every request if I'm not working on CSS?
Unless you know of a better solution, I will definitely be using this going forward!
Any progress on this? I've been using the "patch" of @danxshap and it's been working great.
I had the same problem till a colleague told me about sassc and libsass. This sass compiler is made in C which makes it ten times faster. I followed these instructions to install it on my Mac (brew has some older versions) https://www.snip2code.com/Snippet/189303/Install-SassC---LibSass-for-Mac-OS-X-10- .
# Install SassC Interpreter
$ cd /usr/local/src
$ curl -kL https://github.com/hcatlin/libsass/archive/master.zip > libsass.zip
$ unzip libsass.zip
# Install LibSass Library
$ cd /usr/local/src
$ curl -kL https://github.com/sass/sassc/archive/master.zip > sassc.zip
$ unzip sassc.zip
# Compile SassC
$ export SASS_LIBSASS_PATH="/usr/local/src/libsass-master"
$ cd sassc-master
$ make
$ sudo ln -s $(pwd)/bin/sassc /usr/local/bin/sassc
$ chmod +x /usr/local/bin/sassc
# Example Usage
$ /usr/local/bin/sassc ~/Desktop/input.scss ~/Desktop/output.css
Next, you just add these settings and it works a lot faster
PIPELINE_COMPILERS = (
'pipeline.compilers.sass.SASSCompiler',
)
PIPELINE_SASS_BINARY = '/usr/bin/env sassc'
PIPELINE_COMPILERS = ( 'pipeline.compilers.sass.SASSCompiler', )
PIPELINE_SASS_BINARY = '/usr/bin/env sassc'``
