Improve devops

[connorskees]

This implementation is at a point where it makes sense to

• [x] Setup CI (fixed in #10)
• [ ] Focus on increasing code coverage (see grcov) to at least 85%
  • Ideally we would only increase coverage through the integration tests using SCSS as input
• [ ] Implement better benchmarking with criterion and compare to other implementations

[Keats]

Can you use dart-sass for the spec rather than Ruby like https://github.com/kaj/rsass does? Ruby is a pain

[connorskees]

Sure, I assume you mean using the --impl dart-sass flag to the Ruby spec runner? I'm not aware of any way to run the spec suite without Ruby. In my personal dev I don't use this flag because it adds too much noise -- dart-sass also checks for error spans, which causes a lot of spurious failures.

I did some analysis of failing tests this morning actually. Seems most failures are coming from core_functions. I think many of the color tests are from color functions level 4 -- i.e. the color function syntax using a slash, rgba(0, 0, 0 / 0). Hoping to address these failures very soon.

Results

grass -- failures / total

2079 / 6256 

spec/arguments -- 4 / 4
spec/core_functions -- 941 / 3569
 - color -- 383 / 1424
 - list -- 86 / 228
 - map -- 41 / 121
 - math -- 203 / 469
 - meta -- 135 / 356
 - modules -- 23 / 102
 - selector -- 66 / 740
 - string -- 4 / 129
spec/css -- 164  / 282
spec/directives -- 333 / 490
spec/expressions -- 0 / 1
spec/libsass/ -- 79 / 174
spec/libsass-closed-issues/ -- 227 / 607
spec/libsass-todo-issues/ -- 25 / 29
spec/libsass-todo-tests/ -- 3 / 4
spec/non_conformant -- 238 / 961
spec/operators -- 1 / 1
spec/parser -- 1 / 3 (indented syntax)
spec/values -- 64 / 130


rsass -- failures / total

2311 / 6256

spec/arguments -- 4 / 4
spec/core_functions -- 818 / 3569
 - color -- 34 / 1424
 - list -- 0 / 228
 - map -- 0 / 121
 - math -- 16 / 469
 - meta -- 86 / 356
 - modules -- 11 / 102
 - selector -- 669 / 740
 - string -- 2 / 129
spec/css -- 234  / 282
spec/directives -- 298 / 490
spec/expressions -- 1 / 1
spec/libsass/ -- 91 / 174
spec/libsass-closed-issues/ -- 286 / 607
spec/libsass-todo-issues/ -- hangs, come back to this
spec/libsass-todo-tests/ -- 4 / 4
spec/non_conformant -- 499 / 961
spec/operators -- 1 / 1
spec/parser -- 2 / 3
spec/values -- 49 / 130

[Keats]

It's on npm: https://github.com/kaj/rsass/blob/master/check_spec#L32-L37 I don't have bundle etc but I can run the test suite there

[connorskees]

Would you just like this added to the readme?

The full command is

# This script expects node >=v14.14.0. Check version with `node --version`
git clone https://github.com/connorskees/grass --recursive
cd grass && cargo b --release
cd sass-spec && npm install
npm run sass-spec -- --command '../target/release/grass'

I'll note this appears to execute significantly slower than the Ruby spec runner.

[Keats]

It would be great. It might be slower but since I never managed to get the ruby sass spec running it's still better for me.

[connorskees]

Added in https://github.com/connorskees/grass/commit/c86b5f82e9a07ffd2a1e62838cd84fbbaaa6157f

[connorskees]

Closing this issue:

• performance comparison and benchmarking has been implemented in https://github.com/connorskees/sass-perf (though imperfect)
• https://github.com/connorskees/grass/commit/23abe152bda3db20ec3896807a445f3a75ac230f increases unit test code coverage to 90%. this is on top of the sass-spec and integration tests through bootstrap, bulma, and other large libraries