Blowing Up the Silos
I hit the World Bank today for "Mapping for Results." Putting results front and center is a great idea - as mapping has shot up the hype cycle, it's nice to focus on what it can do beyond putting up pretty pictures. However, I call False Advertising on the conference coordinators.
Maps are a particularly sexy current form of visualization, and there's a lot of great information that can be conveyed that way. (Shameless self-promotion: Like, oh, AfghanistanElectionData.org/.)
The real star of the panels at the event today was not the maps: it was the data backing them. Like the puppeteer pulling the strings, the maps only do what the data tells them to. Panelists returned time and again to the importance of open data and easing access to it.
The World Bank's been a real leader in the Open Data movement; their site, data.worldbank.gov, has thousands of data sets available for download. A lot of neat work has been build on top of their information already. But it's hard, cuz it's yet another data silo. All that info has to be pulled from their site and integrated into your own.
Your organization's silos databases will have your own formats for fields, your own ID numbering scheme for thingies, different elements you find important. It becomes very difficult to shoehorn their square pegs into your round holes.
Enter application programming interfaces (APIs). These let you query the other person's silo in real time, get live answers, and integrate them into your own systems. Whew! Problem solved!
Well, no. You've eased the problems of getting live data into your applications. But you still have the naming and identification problem. Computers work on the basis of numbers; at the end of the day, you have to get the ID numbers for their thingamajigs to match up to ID numbers for your hoosawhatchits.
The next step is to try to work out ways for different organizations to present their information in standard formats. But this is all beautifully network-y - there's no one in charge, and different groups freely connect to each other to get the information they want if it's useful to them. The power of the open data space is that it is entirely organic; my information might be useful for your maps in ways I never would have considered. The negative side is that, well, there's no one in charge. Standard protocols will have to emerge.
But they will. The more connect-able the data is, the more value it has for everyone. It's gonna be messy - remember, the great thing about standards is that you have so many to choose from - but eventually internet-driven applications and open data APIs will be able to play nicely together.