Department for Business, Innovation & Skills
COMTRADE Map Presentation
30 Apr 2015
Francesco Merletti
@mjs2020
@mjs2020
[email protected]
Doing a bit of Javascript, Node, Angular and D3 among other things.
Main features:
A repository of official trade statistics
Statistics dating back as far as 1962
Offers an API
Structured data
> 3 billion records
Things to cover here:
Free access
Different standards
0000
Why a visualization?
To provide an easier query interface
To give insights into the data
Live access to the data
Things to cover here:
Some visualizations exist already
The database and tables are large and hard to manage
Data is easier to compare
Trade for a reporter
Top exported goods
Top imported goods
Top export markets
Top import markets
Features we're planning
Walkthrough for first time users
SVG/PNG download of charts
Embedding
The main technical overview
Single page application
Responsive
Uses D3js, Crossfilter, Bootstrap, jQuery, Historyjs
Packaged with Grunt, Bower, Requirejs
Things to cover here:
No reloading
API middleware
Usage of few, standardised, widely used libraries
The obstacles we faced
Size and complexity of the data
Query limits
Query complexity
Complex metadata
Need for a query middleware
Things to cover here:
Cannot load all data into browser
Query limits
Duplicate data in overlapping queries
Commodity classification
Country codes
If we had to do it again...
Map out the dataset and all user interaction
Stick to few, multipurpose libraries
Test regularly on different browsers/platforms
Things to cover here:
Better to build more than to get stuck with libraries that do everything for you
Commodity classification
Issue popping up at the last minute with IE
... we would do it better
Accessibility!
More user testing
Making it work
(on the less cutting-edge browsers)*
*Any reference to real browsers, in development or discontinued, is purely coincidental.
Browser requirements
Javascript
SVG
(caniuse.com reports 94.5 support as Apr 2015)
XHR Object with CORS enabled
History API (optional)
Things to cover here:
Javascript not too big an issue but a hard requirement
Server-side rendering
Not all IEs are the same
Look mum no SVG! (2)
Rendering to <canvas>
Better performance across browsers
Easy to export to bitmap files
Less interactivity
No vector export
Performance degrades after a few hundred elements.
TOPOJson detail level
Detect features, not browsers
Customizable: detect only what you need
Shivs and Polyfills
Let libraries do some heavy lifting
Many of the common open-source and widely used libraries like D3, jQuery, Bootstrap, Underscore/lodash level off slight inconsistencies between browsers.
Responsiveness
Bootstrap does a lot of this for you
D3, SVG and responsiveness
Viewport
viewBox="0 0 1920 1080"
preserveAspectRatio="xMidYMid meet"
width="547" height="485"
window resize event
$(window).on('resize', function () {
rowchart.resizeSvg(svg, $chart.width());
});