GeoChoropleth refactor

[mtraynham]

Kind of a large refactor, which adds:

• Auto zoom to geo features with the current projection
• Graticule
• Sphere
• Title accessor for features
• Simplifies render and redraw functions

TODO:

• ~~Fix tests (changes to SVG class names don't match)~~
• Add moar tests!
• ~~Document~~

Future:

• Projection change transitions:

var _projectionTween = function (projectionA, projectionB, width, height) {
    return function (d) {
        var t = 0,
            projection = d3.geo.projection(function (λ, φ) {
                λ *= 180 / Math.PI;
                φ *= 180 / Math.PI;
                var p0 = projectionA([λ, φ]),
                    p1 = projectionB([λ, φ]);
                return [(1 - t) * p0[0] + t * p1[0], (1 - t) * -p0[1] + t * -p1[1]];
            }).scale(1).translate([width / 2, height / 2]),
            path = d3.geo.path().projection(projection);

        return function (_) {
            t = _;
            return path(d);
        };
    };
},

• ~~d3.geo.zoom (Jason Davies license?)~~

Strikethroughs have been completed.

[gordonwoodhull]

Cool! Is this change interface-compatible? Not that it matters. I'll queue this up for the 2.1 release (and if we switch to gitflow I'll be able to merge it to a develop branch once you've written tests).

There should no license concerns with Jason Davies' code - it's BSD licensed like d3. I wonder if it's distributed anywhere. We pulled in his box plot implementation from d3-plugins but at a glance it doesn't look like he's published d3.geo.zoom there.

[mtraynham]

By interface compatible, do you mean with leaflet.js?

So two things about the projection transitions vs. projection zooms. I couldn't get them to work together (also with the automatic zoom to projection). There was just too much path-projection manipulation going on at the same time.

I was however able to get d3.geo.zoom to work with the implementation I have checked in. I was unsure about Davies license because it was from his personal blog and not posted anywhere. This is more or less the code for the zooming, which I was able to integrate fairly easily. http://www.jasondavies.com/maps/rotate/d3.geo.zoom.js

[mtraynham]

Also, I updated the web demo so you can try out the new features.

[gordonwoodhull]

His code links to a LICENSE.txt in the same directory which is BSD, so we're good there.

We should link to his source and expand the license link in our copy.

As for interface compatibility, I was asking whether existing uses of geoChoropleth will be broken by the changes. It sounds like at least CSS customizations might get lost.

It's fine to change the interface with a version bump (we won't try to cram this into 2.0) but I'm asking out of curiosity, and it would be nice to document any changes to help folks migrate.

[mtraynham]

Sounds good. Do you like the idea of a CHANGELOG.md file? Might be good for the 2.0 bump.

[seriousben]

Excited about this :) Do you have any examples you could share?

[gordonwoodhull]

Sure.. but maybe we should start the changelog after 2.0 - I don't even know half the things that went into 2.0.

[mtraynham]

@seriousben You can check out my fork and use the 'geo-refactor' branch. After you run npm install, run grunt server. Navigate to http://0.0.0.0:8888/web/vc/index.html for the demo USA map. To try out some of the features, change the usChart in web/vc/index.html with something like this:

            usChart.width(990)
                    .height(500)
                    .dimension(states)
                    .group(stateRaisedSum)
                    .colors(d3.scale.quantize().range(["#E2F2FF", "#C4E4FF", "#9ED2FF", "#81C5FF", "#6BBAFF", "#51AEFF", "#36A2FF", "#1E96FF", "#0089FF", "#0061B5"]))
                    .colorDomain([0, 200])
                    .colorCalculator(function (d) { return d ? usChart.colors()(d) : '#ccc'; })
                    .addLayer(statesJson.features, "state", function (d) {
                        return d.properties.name;
                    })
                    .showGraticule(true)
                    .showSphere(true)
                    .zoomable(true)
                    .title(function (d) {
                        return "State: " + d.key + "\nTotal Amount Raised: " + numberFormat(d.value ? d.value : 0) + "M";
                    });

After trying it, I noticed there is something strange with the auto zoom. I think it's related to the GeoJSON and bounding box calculation, rather than the code itself because it does fit the view, but not as you would expect.

[mtraynham]

Alrighty, I just fixed the web demo by using a fresh copy of the US states from Natural Earth (public domain license). Also required me to change the key accessor on the layer to return d.properties.postal instead of name.

[mtraynham]

Hey Gordon, It does not, a redraw is required when the layers have been updated. It's an interesting suggestion. A render (re)creates the paths it thinks is necessary (layers/sphere/graticule) and sets up the zoom. Then it calls redraw which handles the data from crossfilter, title updates, projection changes, color updates... When I hear redraw, I associate that to what needs to be updated when you see a filter happen.

My thoughts are:

• Should we be removing layers that Crossfilter has no data for?
  • It seems like that was the goal of removing layers in #698
  • I want to suggest we leave that to the implementation, but how does that work?
• What do you expect to be updated often?
  • How often do you remove/add layers?
  • This process could be handled in redraw by removing certain paths and adding new ones, BUT you are adding a step to a commonly called function.
• How much slower is it to recreate all the layers vs. removing a particular one?
  • d3 is built on the notion of data binding. So removal of certain features is definitely something that is supported as #698 has shown.
  • Currently, we loop the layers and add each as a path. Should we consider a secondary d3 data binding approach to the layers that binds data to paths?
  • This refactor simplified a lot, so is there more overhead here than when previously calling redraw?

If there is an optimum case here, I'd rather support that than handling both cases (adding/removing layers during redraw & render).

I don't know if I have ever brought up the case of benchmarking certain features, but let's do what's more performant or more common since there seems to be no right or wrong solution.

[npmiller]

@ThomasRobertFR maybe you could comment on that.

[gordonwoodhull]

I doubt that the logic of removing layers would create any performance penalty at all. In general, you can do as almost much computation as you want, but once you start modifying the DOM, then you have to worry about efficiency. That's why, if you are removing layers, it's much more efficient to redraw than to rerender.

Probably the loganalysis team can describe it better, but my understanding is that they are adding/removing layers without doing a full render because they have implemented semantic zoom on territories, as you can see in their demo. So in this use case, the application would have called removeLayer and then redraw, and it wouldn't have to do with whether there is crossfilter data or not.

It would be cool to use d3 data binding instead of a loop, but again I doubt there is any performance impact, and it's really a code clarity concern.

Anyway, I hope that @mtraynham and the loganalysis team will work together on this refactor for 2.1! This is the area of dc.js that I am least familiar with. (It's not supported by dcplot yet, though my colleagues have requested it.)

[mtraynham]

~~Oof that merge~~. I've updated to handle the layer logic better and satisfy #698. Since it binds the layers in now, it should automatically remove on redraw :)

https://github.com/mtraynham/dc.js/commit/68bbf23b028d2d1667608def48ae216c10b3ff73#diff-ff506ba8d9c0b671b44c576a6aa64674R283

Also, removed most of the unnecessary functions.

[davidcereal]

Zoom is such a cool feature, but when using a flat-surface projection like mercator, it adds strange effects (as i believe geo.zoom intends for). Is it possible to use the new zoom to get a straight normal zoom onto a flat projection?

Thanks!

[mtraynham]

@dberger1989 It's been awhile since I've looked at the code, so it would need a bit of investigation. I do remember that some of the projections do not support rotation and that may be throwing things off. There are a few if-statements that may need to be fixed:

• This one handles fitting the projection on render to bring everything correctly into view. https://github.com/dc-js/dc.js/pull/719/files#diff-ff506ba8d9c0b671b44c576a6aa64674R51
• The d3.geo.zoom function is for interaction, which was derived from Jason Davies implementation. Instead, we could swap out for a flat-zoom like you suggested, maybe based on Bostock's version. https://github.com/dc-js/dc.js/pull/719/files#diff-ff506ba8d9c0b671b44c576a6aa64674R503

Is there a new d3.geo.zoom? I couldn't find one other than the examples from those sites. Davies seemingly has a newer version which looks a bit more fluent, but the source is all minified.

[mtraynham]

Noticed this was getting out of date, rebased against master.

[d3netxer]

so these changes were never pulled in?

[gordonwoodhull]

Hi @d3netxer, that's correct.

I'm in favor of pulling this in for 3.0 but the rebase is probably going to be difficult, given that we've already ported geoChoroplethChart to d3v4 in the 3.0 branch.

I also understand that this may not be backward-compatible. May need more tests too.

So definitely not trivial, but welcome if @mtraynham and/or @kum-deepak want to take it on.

[gordonwoodhull]

Actually probably the easier thing to do would be check if there were any changes made to the original chart and then port this PR to d3v4, replacing this chart. Then we can re-apply #1382