This is a summary of some of my recent research on making OpenStreetMap data analysis easy for urban planners. It was also published on the ACSP blog.
OpenStreetMap – a collaborative worldwide mapping project inspired by Wikipedia – has emerged in recent years as a major player both for mapping and acquiring urban spatial data. Though coverage varies somewhat worldwide, its data are of high quality and compare favorably to CIA World Factbook estimates and US Census TIGER/Line data. OpenStreetMap imported the TIGER/Line roads in 2007 and since then its community has made numerous corrections and improvements. In fact, many of these additions go beyond TIGER/Line’s scope, including for example passageways between buildings, footpaths through parks, bike routes, and detailed feature attributes such as finer-grained street classifiers, speed limits, etc.
This presents a fantastic data source to help answer urban planning questions, but OpenStreetMap’s data has been somewhat difficult to work with due to its Byzantine query language and coarse-grained bulk extracts provided by third parties. As part of my dissertation, I developed a tool called OSMnx that allows researchers to download street networks and building footprints for any city name, address, or polygon in the world, then analyze and visualize them. OSMnx democratizes these data and methods to help technical and non-technical planners and researchers use OpenStreetMap data to study urban form, circulation networks, accessibility, and resilience.
Continue reading Urban Form Analysis with OpenStreetMap Data
OSMnx is a Python package for downloading administrative boundary shapes and street networks from OpenStreetMap. It allows you to easily construct, project, visualize, and analyze complex street networks in Python with NetworkX. You can get a city’s or neighborhood’s walking, driving, or biking network with a single line of Python code. Then you can simply visualize cul-de-sacs or one-way streets, plot shortest-path routes, or calculate stats like intersection density, average node connectivity, or betweenness centrality. You can download/cite the paper here.
In a single line of code, OSMnx lets you download, construct, and visualize the street network for, say, Modena Italy:
import osmnx as ox
Continue reading OSMnx: Python for Street Networks
A few months ago, I wrote about the large investments that U.S. universities are making in their football stadiums. This also included a visual analysis of stadium capacity around the country. Outside of North Korea, the 8 largest stadiums in the world are college football stadiums, and the 15 largest college football stadiums are larger than any NFL stadium.
I received a few comments interested in further analysis of the actual attendance of games held in these stadiums. While capacity is interesting because it represents an expectation and sustained investment by the school, attendance represents the utilization of that investment. My stadium capacity data covered every NCAA division I football stadium in the U.S. as of the 2015 college football season. So, I downloaded the NCAA’s 2015 home game attendance data to compare. My data, code, and analysis are in this GitHub repo. First, I visualized the FBS attendance figures themselves:
Continue reading College Football Stadium Attendance
This is a summary of our JPER journal article (available here) about Craigslist rental listings’ insights into U.S. housing markets.
Rentals make up a significant portion of the U.S. housing market, but much of this market activity is poorly understood due to its informal characteristics and historically minimal data trail. The UC Berkeley Urban Analytics Lab collected, validated, and analyzed 11 million Craigslist rental listings to discover fine-grained patterns across metropolitan housing markets in the United States. I’ll summarize our findings below and explain the methodology at the bottom.
But first, 4 key takeaways:
- There are incredibly few rental units below fair market rent in the hottest housing markets. Some metro areas like New York and Boston have only single-digit percentages of Craigslist rental listings below fair market rent. That’s really low.
- This problem doesn’t exclusively affect the poor: the share of its income that the typical household would spend on the typical rent in cities like New York and San Francisco exceeds the threshold for “rent burden.”
- Rents are more “compressed” in soft markets. For example, in Detroit, most of the listed units are concentrated within a very narrow band of rent/ft² values, but in San Francisco rents are much more dispersed. Housing vouchers may end up working very differently in high-cost vs low-cost areas.
- Craigslist listings correspond reasonably well with Dept of Housing and Urban Development (HUD) estimates, but provide up-to-date data including unit characteristics, from neighborhood to national scales. For example, we can see how rents are changing, neighborhood by neighborhood, in San Francisco in a given month.
Continue reading Craigslist and U.S. Rental Housing Markets
Tools like WalkScore visualize how “walkable” a neighborhood is in terms of access to different amenities like parks, schools, or restaurants. It’s easy to create accessibility visualizations like these ad hoc with Python and its pandana library. Pandana (pandas for network analysis – developed by Fletcher Foti during his dissertation research here at UC Berkeley) performs fast accessibility queries over a network. I’ll demonstrate how to use it to visualize urban walkability. My code is in these IPython notebooks in this urban data science course GitHub repo.
First I give pandana a bounding box around Berkeley/Oakland in the East Bay of the San Francisco Bay Area. Then I load the street network and amenities from OpenStreetMap. In this example I’ll look at accessibility to restaurants, bars, and schools. But, you can create any basket of amenities that you are interested in – basically visualizing a personalized “AnythingScore” instead of a generic WalkScore for everyone. Finally I calculate and plot the distance from each node in the network to the nearest amenity:
Continue reading How to Visualize Urban Accessibility and Walkability
The fall semester begins next week at UC Berkeley. For the third year in a row, Paul Waddell and I will be teaching CP255: Urban Informatics and Visualization.
This masters-level course trains students to analyze urban data, develop indicators, conduct spatial analyses, create data visualizations, and build interactive web maps. To do this, we use the Python programming language, open source analysis and visualization tools, and public data.
This course is designed to provide future city planners with a toolkit of technical skills for quantitative problem solving. We don’t require any prior programming experience – we teach this from the ground up – but we do expect prior knowledge of basic statistics and GIS.
Our teaching materials, including IPython Notebooks, tutorials, and guides are available in this GitHub repo, updated as the semester progresses.
Continue reading Urban Informatics and Visualization at UC Berkeley
How big is Greenland? It’s huge, right? At 836,109 square miles in size, Greenland is the largest island and the 12th largest country on Earth. With only 56,000 people living in that enormous area (80% of which is covered by the world’s only extant ice sheet outside of Antarctica), it is also the least densely populated country on Earth.
You can get a sense of how large Greenland is when you look at a map of the world:
It’s huge! Greenland is bigger than the entire continent of Africa! Or is it? The map above uses the common Mercator projection to project the 3-D surface of the Earth onto a 2-D surface suitable for a paper map or an image on your computer screen. But it’s not easy to project the curved surface of a sphere onto a rectangular plane. Compromises must be made. In the case of the Mercator projection, the compromise is that objects’ sizes become increasingly distorted the further they are from the equator. At the poles, the scale and distortion become infinite.
Continue reading Map Projections That Lie
Western Europe gets all the attention, but that means it also gets all the tourists. Here are some of my favorite old cities that I’ve visited on the other side of the continent, along with a few photos I took while there. Granted, a few of these places are now squarely on the backpacker circuit, but many remain underexplored. What they all share is an incredible, exhilarating sense of urbanism — old and new.
Eastern Europe itself is hard to define. Competing designations might include only the former Soviet states, or all the formerly communist European nations. Others might separate a limited Eastern Europe out from Central and Southeastern Europe. Here I will play fast and loose with the geographic boundaries: these are just cities somewhere vaguely toward the eastern side of the continent. Apologies to any readers whose country is usually considered a part of Central or Southern Europe.
First up: Mostar. A small city in the south of Bosnia and Herzegovina, Mostar is most famous for its medieval Ottoman center and its Old Bridge, or Stari Most:
Continue reading The Beautiful Cities of Eastern Europe
I recently co-authored a journal article titled LEED-ND and Livability Revisited, which won the Kaye Bock award. LEED-ND is a system for evaluating neighborhood design that was developed by CNU, USGBC, and NRDC. Many of its criteria, particularly site location and neighborhood pattern, accordingly reflect New Urbanist and Smart Growth principles and are inspired by traditional neighborhood design.
Continue reading LEED-ND and Neighborhood Livability
I recently completed my inside field exam, one of the many steps involved in advancing to candidacy. The three professors on your inside field committee send you six questions – a pair per professor – and you are given 72 hours total to answer one question from each pair. The answers are to be in the form of a scholarly article with thorough citations. Long story short, you’ve got to write 30 pages of academic scholarship in three days.
The exam questions themselves are very interesting. The professors construct them based on their reading of your inside field statement, trying to probe areas that might be particularly rich or a bit weak in the statement. Here are the questions I answered:
Continue reading The Inside Field Exam and Urban Complexity