6. Data Platforms and Exports

There is no common standard for where 911 data is stored or regarding how it is shared. This means that the datasets the Reimagine 911 project reviewed are available in many different formats.

Data Format Categories

Every dataset is available in one or more specific formats. Different formats may dictate what types of analysis can be performed. In some formats (like CSV), the data is stored as plain text that is readable by humans and can be analyzed using common spreadsheet software like Excel. Other formats store data in forms that only computers can read and that require some technical knowledge to be able to analyze.

• Machine readable formats: CSV, XML, JSON, RSS

  • Data that is machine-readable is structured in a way that computers can read and process. This structure makes data analysis possible by allowing researchers to easily segment and analyze data to uncover patterns and insights.

  • 89 out of 135 Reimagine 911 datasets are available in at least one of these machine-readable formats (Fig. 6.1).

• Human readable formats: PDF, HTML

  • Data that is easy for humans to read and understand but that is not structured in a way that computers can process. This makes these the least useful formats for researchers interested in using open 911 data.

• Geodata: KMZ, ESRI, Geodata, GeoJSON

  • Geodata describes location data. It is machine-readable but requires the use of specialized GIS software to analyze. It often visualizes data in map form.

Fig. 6.1 Most datasets are available in a machine-readable format.

Of the different data export formats of open call data, CSV is the most common type (Fig. 4.2). The diversity of different export formats reflects the diversity of different platforms and websites that cities use to share their open data.

Fig. 6.2 70% of datasets reviewed allow download as a CSV file.

Open Data Platforms

A data platform is a software system that collects and stores data. It allows users to “segment” or divide up the data based on specific characteristics such as location or time. Data platforms are often used within organizations to maintain and analyze data generated by their business activities. People outside of an organization would not normally have access to its data platform. The only way the general public could access that data was if an organization made it available or allowed data retrieval through an API.

• Socrata: An open data platform that returns data in a variety of formats including CSV and JSON.

• Web: Websites that do not use one of the other mainstream platforms. In some cases these may be custom-built open data portals. In other cases they may be less helpful and provide separate CSV or PDF downloads—or in the worst case they may publish data as HTML without an option to download.

• ArcGIS: Allows sharing of maps and geodata as well as non-spatial data such as spreadsheets and PDFs.

• CKAN: An open-source data portal that returns data in JSON format.

• Excel: Spreadsheet software that accepts tabular data and also has features for data calculation.

Fig. 6.3 Socrata and ArcGIS are the most commonly used open data portals, with Web being the second most popular method of sharing data (via tables on webpages, PDFs, etc.).

Open Data Exchange via APIs

API’s (application programming interfaces) are standardized data exchange methods between a platform and a program that consumes its data. They can be used to add, modify, delete, or retrieve data from a platform (they are only used for data retrieval in the R911 datasets.) APIs don’t require much human intervention, making them ideal for handling large groups of datasets. While useful in certain situations, APIs require technical knowledge and additional software to set up.

If you are interested in retrieving data using an API, the three most common platforms all provide API access by default. These three platforms (Socrata, ArcGIS, and CKAN) made up 70% of the open data that we found during our review.