Custom sort order not respected in layered plot

[kanitw]

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {"a": "A","b": 28}, {"a": "B","b": 55}
    ]
  },

  "layer": [{
    "mark": "rule",
    "encoding": {
      "x": {"field": "a", "type": "ordinal", "sort": ["B", "A"]},
      "y": {"field": "b", "type": "quantitative"}
    }
  },{
    "mark": "circle",
    "encoding": {
      "x": {"field": "a", "type": "ordinal"},
      "y": {"field": "b", "type": "quantitative"}
    }
  }]
}

The order should be "B, A", not "A, B".

A similar problem can be seen in https://github.com/vega/vega-lite/pull/5012.

[domoritz]

Is this a duplicate of https://github.com/vega/vega-lite/issues/4932?

[domoritz]

Or of https://github.com/vega/vega-lite/issues/3153?

[kanitw]

They are all similar but slightly different:

• #3153 uses domain in the spec, this issue uses sort.
• #4932 specifies sort in all layers, this issue specifies only one.

[kanitw]

I think the solution for this particular one is to consider sort as one of the component (or part of the scale component) that gets respected when we later derive the CalculateNode (for sort index).

[kanitw]

If we extend Vega domain sort to support either (a) sorting by a calculated field based on each unique value (b) custom sort order.

Then such domain sort could be merged and we can easily fix this issue.

[ben741]

I'm encountering something similar, where adding "color" to an encoding within a layered plot breaks sorting. I'm not sure if it has the same root cause as described here, or whether it should be treated as a separate issue.

Here's my code, where deleting "color": {"field": "highlight"}, causes the sort to work properly:

{
  "data": {
    "values": [
      {"name": "a", "val": 2, "highlight": 1, 
       "lower": 1.5,"upper": 2.5},
      {"name": "c", "val": 1, "highlight": 0,
      "lower": 0.5,"upper": 1.5},
      {"name": "b", "val": 3, "highlight": 0,
      "lower": 2.5,"upper": 3.5}
    ]
  },
  "layer": [
    {
      "mark": "bar",
      "encoding": {
        "color": {"field": "highlight"},
        "y": {
          "field": "name",
          "type": "ordinal",
          "sort": {"field": "val", "order": "descending"}
        },
        "x": {"field": "val", "type": "quantitative"}
      }
    },
    {
      "mark": "rule",
      "encoding": {
        "x": {
          "field": "upper",
          "type": "quantitative",
          "scale": {"zero": false}
        },
        "x2": {"field": "lower", "type": "quantitative"},
        "y": {
          "field": "name",
          "type": "ordinal",
          "sort": {"field": "val", "order": "descending"}
        }
      }
    }
  ]
}

[kanitw]

@ben741 - try adding "stack": false to x for now ("x": {"field": "val", "type": "quantitative", "stack": false}).

The underlying issue is that bar with color encoding is stacked by default (as Vega-Lite doesn't see the data and have to prepare for cases where there are multiple bars per y value.

So for fixing the bug, I'm actually not sure if it is the same example as the example above.

[ben741]

Using "stack": false works perfectly for my case. Thanks!

[john-guerra]

I think I can reproduce this bug, sorting by custom field doesn't seem to work with layered charts. See this example, a simple barchart layered with tick marks and sorting isn't working...

here a minimalistic one:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "data": { "values": [
    { "label": "a", "value": 10},
    { "label": "b", "value": 15}
  ]},
  "encoding": {
    "x": {
      "field": "value",
      "type": "quantitative"
    },
    "y": {
          "field": "label",
          "type": "nominal",
          "sort": {"field": "value", "order": "descending"}
        }
  },
  "layer": [
    {
      "mark": "bar"
    },
    {
      "mark": "text",
      "encoding" : {
        "text": {
          "field": "value",
          "type": "quantitative"
        }
      }
    }
  ]
}

If you have the two layers it doesn't sort by value, but if you remove one of the layers it does

Brought to my attention by @aaizemberg

I tried following the discussion among the many threads, but I couldn't tell if @domoritz already sees it as a bug. Are we missing something?

[domoritz]

Yes, I see this and related sorting issues as bugs.

[jonathanzong]

@MufaroMakiwa has introduced a PR addressing this issue here: https://github.com/vega/vega-lite/pull/8567

This issue is about merging behavior for conflicting sort domains, which I think is more or less undefined for some cases right now. There are two example specs in this issue, Ham's and John's that show two slightly different issues. Our PR addresses John's example, which is also a blocking issue for bar chart races in Animated Vega-Lite.

In Ham's example, a custom sort is defined on one layer and not defined on the other. I don't actually think this is a bug because if you lift the custom sort definition to the top level, it correctly propagates to the layers and does the expected behavior: demo in editor. When the sorts conflict across layers like in Ham's original example, I think it's reasonable to do what we do now (warn and set the default sort).

In John's example, there is a sort order defined on the top level, but it's not correctly/consistently applying to all child layers. This is because in normalizeSortField in compile/scale/domain.ts, the default op is set to sum for layers with a stack in their corresponding unit model (i.e. the bar layer), and min for the default case. Because the op is filled in differently for each layer, the compiler thinks we've specified two different sorts. So just to be clear, this is a case that (I think) only happens when you have a bar chart with additional layers that aren't bars. And the issue is that the compiler is checking the stack case only on the current layer, and the text layer doesn't know that it's in a layered bar chart.

To address this, we've added a case in the mergeDomains function that checks if all the sorts have an op defined, and if there is one of them that isn't a min (i.e. default) op, let that one win in the merge (but still emit a warning that there are conflicting sorts). Suggestions welcome -- we tried to stick to a minimal change that will fix the issue.

[chris-canipe]

I'm working around this by specifying "sort": false in the first layer and the desired sort in the last:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {"a": "A","b": 28}, {"a": "B","b": 55}
    ]
  },

  "layer": [{
    "mark": "rule",
    "encoding": {
      "x": {"field": "a", "type": "ordinal", "sort": false},
      "y": {"field": "b", "type": "quantitative"}
    }
  },{
    "mark": "circle",
    "encoding": {
      "x": {"field": "a", "type": "ordinal", "sort": ["B", "A"]},
      "y": {"field": "b", "type": "quantitative"}
    }
  }]
}