Improve CSS extracts

[cdoublev]

These are requests for a future major release, I guess.

The following is a representation of the current shape of the extracted CSS data related to properties/at-rules/descriptors:

properties: {
  <PropertyName>: {
    name: <PropertyName>,
    ...,
  },
},
atrules: {
  <RuleName>: {
    descriptors: [
      {
        name: <DescriptorName>,
        for: <RuleName>,
        ...,
      },
  ],
}

A collection of properties/rules has a different shape than a collection of descriptors: a mapping from the property/rule names to their definition vs. a list of descriptor definitions. What is the reason for using a mapping for properties/rules? At least, the name field seems redundant in properties entries.

EDIT: I guess that collecting only property names is better done with Object.keys(properties) than with properties.map(({ name }) => name).

Now that descriptors are nested in its corresponding at-rule (instead of in a flat list of descriptors restricted to the rule corresponding to their for field), I think the for field is also no longer required.

It is great to have at-rules extracted, thanks for the hard work! I think it would be better to have a rules field instead of atrules. This would allow to get the definition of qualified rules (no name, no descriptors). For exemple, if a keyframe rule were defined with an appropriate markup, it could be extracted as follow:

rules: [
  {
    value: '<keyframe-selector># { <declaration-list> }',
  },
  {
    descriptors: [],
    value: '@keyframes <keyframes-name> { <rule-list> }',
  },
]

Note that for rules, I also used a list instead of a mapping. Afaict, there can be no duplicate between rule definitions, and I did not add the name for @keyframes because it can be resolved from value. It is good to have a default descriptors field for at-rules, but it would be inappropriate for a qualified rule such as a keyframe rule.

Finally, @layer has two alternative value definitions: @layer <layer-name>? { <stylesheet> } and @layer <layer-name>#;. Only the former is currently extracted. My best guess is that they could be joined with | but I did not implemented this rule in my CSS parser therefore I cannot tell if it is the right thing to do.

* EDIT: actually, the following would be a better solution for @layer.

rules: [
  {
    descriptors: [],
    value: '@layer <layer-name>? { <stylesheet> }',
  },
  {
    descriptors: [],
    value: '@layer <layer-name>#;',
  },
]

[tidoust]

What is the reason for using a mapping for properties/rules? At least, the name field seems redundant in properties entries.

Indexed lists give a direct way to find an item in the list and it's trivial to convert such lists to an array with Object.values, which the repetition of the key inside the object also makes easier. That said, we should be consistent throughout, and I note the key isn't currently repeated for atrules.

I think we may need to switch to arrays for valuespaces in any case when we start extracting the data-dfn-for attribute and if we start extracting actual values. Perhaps we could use that opportunity to switch to arrays everywhere. Although perhaps it would be better to have contextual info attached to the property they apply to instead of being listed within the valuespaces namespace.

I think it would be better to have a rules field instead of atrules.

I'm not clear what regular rules mean here. Are we only talking about <rule-list>? That seems to be only used in @keyframes. I would rather try to find a mechanism to complete the at-rule definition somehow to explain what <rule-list> is supposed to mean for the at-rule. In this case, that would be a list of <keyframe-block> rules (noting that <keyframe-block> is currently extracted as a valuespace).

In your example, what is the point of extracting <keyframe-block> as a rule if it cannot be linked to the @keyframes at-rule?

Most at-rules are not fully described with formal grammar as far as I can tell, so I don't know how we can really improve extracts. The at-rules typically make use of block contents: <style-block>, <declaration-list> (usually linked with descriptors), <rule-list> and <stylesheet>.

Even with descriptors, while we now successfully extract them, where these descriptors may be used in the at-rule is usually not directly encoded in the grammar.

Finally, @layer has two alternative value definitions: @layer ? { } and @layer #;

That's a good point and we need to support that in the extract. There are two kinds of at-rules: statement at-rules which are simpler constructs that end in a semicolon, and block at-rules. I wonder whether there are at-rules other than @layer that exist in both kinds. Do you know?

[cdoublev]

Perhaps we could use that opportunity to switch to arrays everywhere. Although perhaps it would be better to have contextual info attached to the property they apply to instead of being listed within the valuespaces namespace.

+1 for arrays everywhere. Contextual info attached to the property/descriptor/value definition does not hurt, of course. I guess you mean one of the following:

{
  "properties": [
    {
      "name": "grid-template-columns",
      "valuespaces": [{ "value": "fit-content(<length-percentage>)" }]
    },
    {
      "name": "width",
      "valuespaces": [{ "value": "fit-content(<length-percentage [0,∞]>)" }],
    }
  ],
  "valuespaces": [
    { "value": "fit-content(<length-percentage>)", "for": ["grid-template-columns", "grid-template-rows"] },
    { "value": "fit-content(<length-percentage [0,∞]>)", "for": ["width", "height", "many-other-props"] }
  ]
}

I do not have an opinion on which option would be appropriate because both fit-content() values will be directly included in a parent value definition instead of being referenced via <fit-content()>: I do not need to resolve and care about the property a function whose name is fit-content applies to.

I would rather try to find a mechanism to complete the at-rule definition somehow to explain what is supposed to mean for the at-rule. [...]

In your example, what is the point of extracting <keyframe-block> as a rule if it cannot be linked to the @keyframes at-rule?

It would be great if they were linked! You can also consider a style sheet to be defined with <rule-list>. In this context (a style sheet), <rule-list> accepts top-level at-rules and style rules as qualified rules. In the context of @keyframes, <rule-list> only accepts keyframe rule.

The following is a simplified excerpt of the context definition I use for parsing.

const topLevelContext = {
  qualified: 'style', // (CSS Syntax) "Interpret all of the resulting top-level qualified rules as style rules"
  rules: [
    // ... other top-level rules
    { 
      name: 'keyframes',
      // type: 'keyframes', (implicit, default to `name`)
      prelude: '<keyframes-name>',
      qualified: 'keyframe',
      value: '<rule-list>',
      rules: [
        { 
          cascading: false,
          prelude: '<keyframe-selector>#',
          // Declaration property names allowed in a keyframe rule
          properties: require('../properties/animatable.js'),
          type: 'keyframe',
          value: '<declaration-list>',
        },
      ],
    },
    {
      prelude: '<selector-list>',
      // Declaration property names allowed in a style rule
      properties: require('../properties/names.js'),
      type: 'style',
      rules: [/* Definitions of nested style rules and nested conditional rules */],
      value: '<style-block>',
    }
    // ... other top-level rules
  ],
  value: '<rule-list>',
}

I hope it makes things clearer about how I would ideally consume the extracted data. Note that I define relations between rules with nesting but I could have defined them with a flat list and some additional fields.

Even with descriptors, while we now successfully extract them, where these descriptors may be used in the at-rule is usually not directly encoded in the grammar.

Yeh, I thought I posted a comment about this issue on the w3c/csswg-drafts repository but I cannot find it. Assuming prelude-only would be a property only allowed in the prelude of @media, I have no way to eg. accept @media (prelude-only: 'whatever-is-valid-for-this-property') {} but prevent @media all { prelude-only: 'whatever-is-valid-for-this-property' }.

I wonder whether there are at-rules other than @layer that exist in both kinds. Do you know?

Fortunately, there is no other case of rules with alternative syntaxes! :)

[tidoust]

Finally, @layer has two alternative value definitions: @layer <layer-name>? { <stylesheet> } and @layer <layer-name>#;. Only the former is currently extracted. My best guess is that they could be joined with | but I did not implemented this rule in my CSS parser therefore I cannot tell if it is the right thing to do.

Note both value definitions are now extracted and joined with | (easier to achieve in the crawler than keeping the values separated): https://github.com/w3c/webref/blob/f614a81be2c1e4b0fc6b90135c14152aa192cd4b/ed/css/css-cascade.json#L27-L30

[cdoublev]

I think this issue no longer needs to stay open! I took a look on the changes in v6 and it looks great, thanks. 🎉