Tag: scikit-learn

Categories
Urban

Urban Street Network Orientation

My new article, Urban Spatial Order: Street Network Orientation, Configuration, and Entropy, has just been published in one of my favorite journals: Applied Network Science (download free PDF). This study explores the spatial signatures of urban evolution and central planning. It examines street network orientation, connectivity, granularity, and entropy in 100 cities around the world using OpenStreetMap data and OSMnx for modeling and visualization:

City street network grid orientations, order, disorder, entropy, rose plot, polar histogram made with Python, OSMnx, OpenStreetMap, matplotlib.

So, who’s got a grid and who doesn’t? Each of the cities above is represented by a polar histogram (aka rose diagram) depicting how its streets orient. Each bar’s direction represents the compass bearings of the streets (in that histogram bin) and its length represents the relative frequency of streets with those bearings. The cities above are in alphabetical order. Here they are again, re-sorted from most-ordered/gridded city (Chicago) to most-disordered (Charlotte):

Continue reading “Urban Street Network Orientation”
Categories
Data

Street Network Analysis in a Docker Container

Containerization is the way of the future present. I’ve heard feedback from some folks over the past few months who would like to play around with OSMnx for street network analysis, transport modeling, and urban design—but can’t because they can’t install Python and its data science stack on their computers. Furthermore, it would be nice to have a consistent reference environment to deploy on AWS or elsewhere in the cloud.

So, I’ve created a docker image containing OSMnx, Jupyter, and the rest of the Python geospatial data science stack, available on docker hub alongside additional usage instructions. If you’re starting from scratch, you can get started in four simple steps:

Continue reading “Street Network Analysis in a Docker Container”
Categories
Data

Network-Based Spatial Clustering

Jobs, establishments, and other amenities tend to agglomerate and cluster in cities. To identify these agglomerations and explore their causes and effects, we often use spatial clustering algorithms. However, urban space cannot simply be traversed as-the-crow-flies: human mobility is network-constrained. To properly model agglomeration along a city’s street network, we must use network-based spatial clustering.

The code for this example can be found in this GitHub repo. We use OSMnx to download and assemble the street network for a small city. We also have a dataframe of points representing the locations of (fake) restaurants in this city. Our restaurants cluster into distinct districts, as many establishments and industries tend to do:

firm locations on the street network to be clustered: python, osmnx, matplotlib, scipy, scikit-learn, geopandas

Continue reading “Network-Based Spatial Clustering”
Categories
Data

OSMnx Features Round-Up

OSMnx is a Python package for quickly and easily downloading, modeling, analyzing, and visualizing street networks and other spatial data from OpenStreetMap. Here’s a quick round-up of recent updates to OSMnx. I’ll try to keep this up to date as a single reference source. A lot of new features have appeared in the past few months, and people have been asking about what’s new and what’s possible. You can:

  • Download and model street networks or other networked infrastructure anywhere in the world with a single line of code
  • Download any other spatial geometries, place boundaries, building footprints, or points of interest as a GeoDataFrame
  • Download by city name, polygon, bounding box, or point/address + network distance
  • Download drivable, walkable, bikeable, or all street networks
  • Download node elevations and calculate edge grades (inclines)
  • Impute missing speeds and calculate graph edge travel times
  • Simplify and correct the network’s topology to clean-up nodes and consolidate intersections
  • Fast map-matching of points, routes, or trajectories to nearest graph edges or nodes
  • Save networks to disk as shapefiles, geopackages, and GraphML
  • Save/load street network to/from a local .osm xml file
  • Conduct topological and spatial analyses to automatically calculate dozens of indicators
  • Calculate and visualize street bearings and orientations
  • Calculate and visualize shortest-path routes that minimize distance, travel time, elevation, etc
  • Visualize street networks as a static map or interactive leaflet web map
  • Visualize travel distance and travel time with isoline and isochrone maps
  • Plot figure-ground diagrams of street networks and building footprints
Continue reading “OSMnx Features Round-Up”
Categories
Tech

Describing Cities with Computer Vision

What does artificial intelligence see when it looks at your city? I recently created a Twitter bot in Python called CityDescriber that takes popular photos of cities from Reddit and describes them using Microsoft’s computer vision AI. The bot typically does pretty well with straightforward images of city skylines and street scenes:

Some are even kind of wryly poetic, such as this description of Los Angeles:

Continue reading “Describing Cities with Computer Vision”
Categories
Data

Mapping Everywhere I’ve Ever Been in My Life

I recently wrote about visualizing my Foursquare check-in history and mapping my Google location history, and it inspired me to mount a more substantial project: mapping everywhere I’ve ever been in my life (!!). I’ve got 4 years of Foursquare check-ins and Google location history data. For everything pre-smart phone, I typed up a simple spreadsheet of places I’d visited in the past and then geocoded it with the Google Maps API. All my Python and Leaflet code is available in this GitHub repo and is easy to re-purpose to visualize your own location history.

I’ll show the maps first, then run through the process I followed, below. First off, I used Python and matplotlib basemap to create this map of everywhere I’ve ever been:

Location History World Map, data from Foursquare and Google, made with Python matplotlib basemap

Continue reading “Mapping Everywhere I’ve Ever Been in My Life”
Categories
Tech

Scientific Python for Raspberry Pi

Raspberry Pi 3 Model BA guide to setting up the Python scientific stack, well-suited for geospatial analysis, on a Raspberry Pi 3. The whole process takes just a few minutes.

The Raspberry Pi 3 was announced two weeks ago and presents a substantial step up in computational power over its predecessors. It can serve as a functional Wi-Fi connected Linux desktop computer, albeit underpowered. However it’s perfectly capable of running the Python scientific computing stack including Jupyter, pandas, matplotlib, scipy, scikit-learn, and OSMnx.

Despite (or because of?) its low power, it’s ideal for low-overhead and repetitive tasks that researchers and engineers often face, including geocoding, web scraping, scheduled API calls, or recurring statistical or spatial analyses (with small-ish data sets). It’s also a great way to set up a simple server or experiment with Linux. This guide is aimed at newcomers to the world of Raspberry Pi and Linux, but who have an interest in setting up a Python environment on these $35 credit card sized computers. We’ll run through everything you need to do to get started (if your Pi is already up and running, skip steps 1 and 2).

Continue reading “Scientific Python for Raspberry Pi”
Categories
Academia

Urban Informatics and Visualization at UC Berkeley

The fall semester begins next week at UC Berkeley. For the third year in a row, Paul Waddell and I will be teaching CP255: Urban Informatics and Visualization, and this is my first year as co-lead instructor.

This masters-level course trains students to analyze urban data, develop indicators, conduct spatial analyses, create data visualizations, and build Paris open datainteractive web maps. To do this, we use the Python programming language, open source analysis and visualization tools, and public data.

This course is designed to provide future city planners with a toolkit of technical skills for quantitative problem solving. We don’t require any prior programming experience – we teach this from the ground up – but we do expect prior knowledge of basic statistics and GIS.

Update, September 2017: I am no longer a Berkeley GSI, but Paul’s class is ongoing. Check out his fantastic teaching materials in his GitHub repo. From my experiences here, I have developed a course series on urban data science with Python and Jupyter, available in this GitHub repo.

Continue reading “Urban Informatics and Visualization at UC Berkeley”
Categories
Data

Clustering to Reduce Spatial Data Set Size

Read/cite the paper here.

In this tutorial, I demonstrate how to reduce the size of a spatial data set of GPS latitude-longitude coordinates using Python and its scikit-learn implementation of the DBSCAN clustering algorithm. All my code is in this IPython notebook in this GitHub repo, where you can also find the data.

Traditionally it’s been a problem that researchers did not have enough spatial data to answer useful questions or build compelling visualizations. Today, however, the problem is often that we have too much data. Too many scattered points on a map can overwhelm a viewer looking for a simple narrative. Furthermore, rendering a JavaScript web map (like Leaflet) with millions of data points on a mobile device can swamp the processor and be unresponsive.

Continue reading “Clustering to Reduce Spatial Data Set Size”