I previously demonstrated how to create figure-ground square-mile visualizations of urban street networks with OSMnx to consistently compare city patterns, design paradigms, and connectivity. OSMnx downloads, analyzes, and visualizes street networks from OpenStreetMap but it can also get building footprints. If we mash-up these building footprints with the street networks, we get a fascinating comparative window into urban form:
This is a guide for absolute beginners to get started using Python. Since releasing OSMnx a few weeks ago, I’ve received a lot of comments from people who would love to try it out, but don’t know where to begin with Python. I’ll demonstrate how to get Python up and running on your system, how to install packages, and how to run code.
The heart of Allan Jacobs’ classic book on street-level urban form and design, Great Streets, features dozens of hand-drawn figure-ground diagrams in the style of Nolli maps. Each depicts one square mile of a city’s street network. Drawing these cities at the same scale provides a revealing spatial objectivity in visually comparing their street networks and urban forms.
We can recreate these visualizations automatically with Python and the OSMnx package, which I developed as part of my dissertation. With OSMnx we can download a street network from OpenStreetMap for anywhere in the world in just one line of code. Here are the square-mile diagrams of Portland, San Francisco, Irvine, and Rome created and plotted automatically by OSMnx:
A spatial index such as R-tree can drastically speed up GIS operations like intersections and joins. Spatial indices are key features of spatial databases like PostGIS, but they’re also available for DIY coding in Python. I’ll introduce how R-trees work and how to use them in Python and its geopandas library. All of my code is in this notebook in this urban data science GitHub repo.
Tools like WalkScore visualize how “walkable” a neighborhood is in terms of access to different amenities like parks, schools, or restaurants. It’s easy to create accessibility visualizations like these ad hoc with Python and its pandana library. Pandana (pandas for network analysis – developed by Fletcher Foti during his dissertation research here at UC Berkeley) performs fast accessibility queries over a network. I’ll demonstrate how to use it to visualize urban walkability. My code is in these IPython notebooks in this urban data science course GitHub repo.
First I give pandana a bounding box around Berkeley/Oakland in the East Bay of the San Francisco Bay Area. Then I load the street network and amenities from OpenStreetMap. In this example I’ll look at accessibility to restaurants, bars, and schools. But, you can create any basket of amenities that you are interested in – basically visualizing a personalized “AnythingScore” instead of a generic WalkScore for everyone. Finally I calculate and plot the distance from each node in the network to the nearest amenity:
I recently wrote about visualizing my Foursquare check-in history and mapping my Google location history, and it inspired me to mount a more substantial project: mapping everywhere I’ve ever been in my life (!!). I’ve got 4 years of Foursquare check-ins and Google location history data. For everything pre-smart phone, I typed up a simple spreadsheet of places I’d visited in the past and then geocoded it with the Google Maps API. All my Python and Leaflet code is available in this GitHub repo and is easy to re-purpose to visualize your own location history.
I’ll show the maps first, then run through the process I followed, below. First off, I used Python and matplotlib basemap to create this map of everywhere I’ve ever been:
I recently wrote about visualizing my Foursquare check-in history and it inspired me to map my entire Google location history data – about 1.2 million GPS coordinates from my Android phone between 2012 and 2016. I used Python and its pandas, matplotlib, and basemap libraries. The Python code is available in this notebook in this GitHub repo, and it’s simple to re-use to visualize your own location history.
Just download your JSON file from Google then run the code. First I load the JSON file and parse the latitude, longitude, and timestamp with pandas. Then I map my worldwide data set:
Last.fm is a web site that tracks your music listening history across devices (computer, phone, iPod, etc) and services (Spotify, iTunes, Google Play, etc). I’ve been using Last.fm for nearly 10 years now, and my tracked listening history goes back even further when you consider all my pre-existing iTunes play counts that I scrobbled (ie, submitted to my Last.fm database) when I joined Last.fm.
Using Python, pandas, matplotlib, and leaflet, I downloaded my listening history from Last.fm’s API, analyzed and visualized the data, downloaded full artist details from the Musicbrainz API, then geocoded and mapped all the artists I’ve played. All of my code used to do this is available in this GitHub repo, and is easy to re-purpose for exploring your own Last.fm history. All you need is an API key.
First I visualized my most-played artists, above. Across the dataset, I have 279,769 scrobbles (aka, song plays). I’ve listened to 26,761 different artists and 66,377 different songs across 38,026 different albums from when I first started using iTunes circa 2005 through the present day. This includes pretty close to every song I’ve played on anything other than vinyl during that time. Continue reading Analyzing Last.fm Listening History
I started using Foursquare at the end of 2012 and kept with it even after it became the pointless muck that is Swarm. Since I’ve now got 4 years of location history (ie, check-ins) data, I decided to visualize and map it with Python, matplotlib, and basemap. The code is available in this GitHub repo. It’s easy to re-purpose to visualize your own check-in history: you just need to plug in your Foursquare OAuth token then run the notebook.
First the notebook downloads all my check-ins from the Foursquare API. Then I mapped all of them, using matplotlib basemap.
The fall semester begins next week at UC Berkeley. For the third year in a row, Paul Waddell and I will be teaching CP255: Urban Informatics and Visualization.
This masters-level course trains students to analyze urban data, develop indicators, conduct spatial analyses, create data visualizations, and build interactive web maps. To do this, we use the Python programming language, open source analysis and visualization tools, and public data.
This course is designed to provide future city planners with a toolkit of technical skills for quantitative problem solving. We don’t require any prior programming experience – we teach this from the ground up – but we do expect prior knowledge of basic statistics and GIS.
Our teaching materials, including IPython Notebooks, tutorials, and guides are available in this GitHub repo, updated as the semester progresses.