Originally posted by Benjamin Romano on Xconomy
Drawing on an unprecedented amount of government data that is easier to access than ever before, more than 11,000 software developers, entrepreneurs, students, and others across the country devoted part of last weekend to building technologies designed to help local, state, and federal governments solve problems and improve their communities.
The first National Day of Civic Hacking encompassed events in 83 cities including Detroit, where an understaffed city department sought help answering its phones; Denver, where the winning team built an app that helps visitors find legal marijuana dispensaries; and Seattle, where teams developed tools to connect people with events in their neighborhoods.
The civic hacking events are one manifestation of a broad movement toward technology-enabled openness and citizen participation in government. Seattle company Socrata is making a fine business of this "democratization" of government data, providing a cloud-computing platform that helps cities, counties, states, and agencies in the U.S. and abroad manage and publicize troves of data.
The company didn't set out to serve the public sector.
Founder and CEO Kevin Merritt had built an email archiving company that ultimately became part of Microsoft through its 2005 acquisition of FrontBridge Technologies. Merritt left the software giant in 2007 to begin Blist, the company that became Socrata.
The name change corresponded with a significant pivot for the Seattle startup. The idea behind Blist was to make "the world's easiest database," helping non-specialists within an organization use data that had been the realm of programmers and database administrators. (It's a mission we hear from a lot of big data companies these days, including Seattle-based Tableau Software.)
Socrata's data sharing platform was loaded with social features, under the hypothesis "that somebody in isolation wouldn't be able to understand data as well as a group of people in collaboration," Merritt, pictured above, says. And it turned out that rather than using the product as a database, customers were using it as a way to access and share data from other enterprise systems. "It offended us at first," he says.
Even more importantly for the future of the company, the platform had gained a following in the public sector as the "open data" movement--the idea that governments would put data online for reasons of transparency, accountability, efficiency, and good governance--began to gain steam in late 2008 and early 2009.
"They were going through this metamorphosis from a default position of hoarding and only exposing [data] through ... Freedom of Information Act requests, to governments just saying let's put it all online and let's see what happens," Merritt says.
In spring 2009--around the time the Obama Administration began the federal government's open data site, Data.gov--Socrata decided to focus all of its attention on the public sector.
It has since built a roster of more than 70 customers around the world, and from all levels of government. They include the cities of Seattle, New York City, and San Francisco; counties from Snohomish in Washington to Travis in Texas; states including Washington, Hawaii, and Maryland; and major federal agencies like the Centers for Medicare & Medicaid Services, the Consumer Financial Protection Bureau, and the Data.gov site, now with more than 66,300 datasets online. The company has a growing list of international organizations and governments, too.
Governments pay Socrata a monthly subscription fee to use its platform, which can provide everything from a public-facing Web site for interacting with government data to data hosting to application programming interfaces (APIs) for third-party developers to tools for custom information products. The company also hosts a vast quantity of government data that is not made available to the public. More than half of the data on Socrata's network is for internal government use, Merritt says.
The company streams some 14 terabytes of data a month.
Socrata's pricing structure depends on the size of the government customer and the amount of data it puts online. In general, a city of 300,000 to 400,000 people would pay "a few thousand dollars a month" for an open data portal. Smaller jurisdictions can get discounts, and the company is working on engineering improvements that could put the technology in reach of small towns.
Merritt says most governments start with geospatial, expenditure, crime, and transit data--all of which is usually relatively easy to access and publicize--and then move on to other data sets that may require underlying systems be upgraded to feed into the Socrata platform at regular intervals. Few governments can afford to proactively digitize their historical archives, but will post data that is accessed to satisfy Freedom of Information Act requests so that others can see it.
Socrata reported in April that its recurring revenue had grown 126 percent year over year, though it declined to give a revenue or profit figure for this story. However, the company is clearly doing well, having not raised additional capital since 2008 when it took on $6.5 million from Frazier Technology Ventures and Morgenthaler Ventures. The company has 45 employees, mainly in an office that once housed Starbucks' corporate headquarters in Pioneer Square. It also recently opened branches in Washington, D.C., and London. It is backed by Frazier Technology Ventures and Morgenthaler Ventures, which invested $6.5 million in early 2008.
Merritt says Socrata's services are usually an easy sell once governments have made the decision to put data online--and more are enacting policies to make this standard procedure. Last month, President Obama issued an executive order that "the default state of new and modernized government information resources shall be open and machine readable."
Socrata's competition is often a combination of products and services from legacy hardware vendors, system integrators, and internal and external developers. But the company also faces a growing competitor in the open-source Comprehensive Knowledge Archive Network (CKAN), developed by the U.K.-based Open Knowledge Foundation and used for data sites by governments in the U.K., Brazil, the Netherlands, Germany, and most recently, as part of an upgraded Data.gov site. (The software is free. Organizations including the Open Knowledge Foundation offer paid services such as customization and integration.)
The federal government's main open data repository late last month initiated a new unified data catalog based on the CKAN system "that will make it easier to federate with other federal agency catalogs, as well as those of states, cities, and counties," according to a blog post on the change.
The Socrata platform continues to provide data hosting, visualization, and APIs for the Data.gov site, as well as for the open data efforts of several federal agencies. But it can't be thrilled to see arguably its biggest rival gain a foothold on one of the world's foremost government data stores.
(The company's platform is interoperable with CKAN, and Socrata also provides an open source version of its core technology.)
Merritt sees a long way yet to run for the open data movement. He estimates that only about 1 percent of government data is online. And the amount of data captured each day is of course growing as more systems get more sophisticated sensors. Utility smart meters are one example.
In addition to the citizen hackers who went to work last weekend on projects for scores of city and state governments and 21 federal agencies, big companies are doing more with government data.
For example, Socrata is working with Yelp on a pilot project in New York and San Francisco to add restaurant health inspection reports to online reviews.
There are also potential future business opportunities for Socrata in standardizing and aggregating certain kinds of government data for specific industries. Insurance companies, for example, have contacted the company offering to pay for zip-code level crime rates in a uniform format from city to city, Merritt says.
"We're just laying the foundation for what's going to happen over the next two to five years, where we're really going to see ecosystems develop on the back of open data, and we're going to see governments allow civic developers to help them deliver the last mile of services that they want to deliver to the citizens and residents that they serve," Merritt says.
