geoChoropleth: Alter underlying topojson WITHOUT re-rendering

[aaronrudkin]

I have a geoChoropleth of the US that shows a political party's control of a given state at a given time. One of the functions I have implemented is a smooth animation where the user can drag a bar through history to an arbitrary year and see how data changes or just press play and watch the advance of time. This looks great--I do it by switching crossfilter groups on the fly and redrawing the map. Smooth transitions, looks great, good performance. For areas of the continent that are not yet states, I simply grey them out so users have a visual cue as to why they are not present in the data.

One limitation of this is that, in point of fact, the actual shapes of states change a good deal during the course of history, beyond just joining and leaving the United States. The partitioning of Virginia is one good example, but actually many states started out having large swaths of mostly unoccupied land which were later carved away into territories and then finally admitted as new states later. As a result, my map is anachronistic up until Hawaii and Alaska join. This is an acceptable limitation for the project, but not an ideal one, so I'd like to fix it.

I can get around this by dynamically loading a new json for each time period. It's not difficult:

d3.json("/static/json/states"+loadNum+".json", function(error, stateboundaries)
        {
                if(!error)
                {
                        var mapTopo = topojson.feature(stateboundaries, stateboundaries.objects.states).features;
                        partyMapChart.overlayGeoJson(mapTopo, 'state', function(d) { return d.id; })
                }
        });

But now, having substituted the data, I need to draw the new map. However, just running partyMapChart.redraw() will not actually update the underlying GeoJson overlay the way it would for a group switch. Instead, I need to force partyMapChart.render(). This is unfortunate, because it breaks the transition animations (suppose a state does not change, and should be coloured dark blue both before and after the switch: instead, we see a flash of white and another fade to dark blue). This creates sort of strobe effect that is very unpleasant. Also, we use a custom legend overlay that we have hooked to a postRender listener, so the legend disappears and flashes in again.

Any advice for the best way to do this? If not, I'd like to suggest the ability to update the json overlay without re-rendering as an enhancement.

The project is private and due to grant restrictions shouldn't be shared before it is ready, so I'd prefer not to give a working example, but if you have difficulty grasping the issue I'm running into I could try to come up with one.

[gordonwoodhull]

Hi @aaronrudkin, that sounds like a neat project!

I think you have analyzed the situation correctly.

This would be a great feature for dc.js. I think it would have to be under an option, because in the general case you are going to need to morph those state shapes, which is obviously more computation than just changing colors.

I'd encourage you to go ahead and start modifying the source for this, and we'll find a way to include it. If it helps, there is a refactored version of the geoChoropleth in PR #719.

Please let us know if you run into trouble - there are a number of people here who can help.

[gordonwoodhull]

Maybe track the old geojson layers and morph the shapes on redraw if there is an oldData for a layer.

I'm surprised not to find good boundary-morphing d3 code in a quick search - lots of examples with circles and where there are the same number of points. I'm sure it's out there.

[aaronrudkin]

As it relates to the project I mentioned:
We ended up implementing this using a fairly clever hack that @JeffreyBLewis came up with; we created a single overlay json file that had every historical state, layered on top of each other. So present West Virginia and Virginia lay on top of greater Virginia from before the partition.

The resulting file is small enough on a state-by-state level that we could do this and still preserve good performance for the page of interest. Each state in the overlay had a range of valid years. I used a postRender hook to use this data and detect states that are out of their temporal bound based on the current time state, making them invisible. The result is not quite visually perfect (and for some reason using d3's transition on path opacities isn't working, so it's a toggle rather than a smooth fade) but it's quite a good deal better than the redrawing solution, performs well, and is compatible with our tooltip overlay.

I'll leave the issue open because I think more broadly this is a good enhancement for dc.js and would be valuable going forward, and not all projects have a clever solution the way we do (and, for example, if we wanted to do a similar map for congressional districts, the data file would be infeasibly large to load at once but probably small enough to dynamically load). Thanks for your help and it's very refreshing to see a dev actively following up and brainstorming on a random user enhancement request!

[gordonwoodhull]

Sure thing. Thanks for sharing your solution @aaronrudkin.

On further reflection, individual morphing of the boundaries would probably not work for showing changing boundaries between states. So just fading out the invalid boundaries is probably best.

Curious that opacity transitions are not working, though. You might consider using a pretransition hook instead of postRender because it catches either a render or redraw before dc.js has applied any transitions.