Mapping Everywhere I’ve Ever Been in My Life

I recently wrote about visualizing my Foursquare check-in history and mapping my Google location history, and it inspired me to mount a more substantial project: mapping everywhere I’ve ever been in my life (!!). I’ve got 4 years of Foursquare check-ins and Google location history data. For everything pre-smart phone, I typed up a simple spreadsheet of places I’d visited in the past and then geocoded it with the Google Maps API. All my Python and Leaflet code is available in this GitHub repo and is easy to re-purpose to visualize your own location history.

I’ll show the maps first, then run through the process I followed, below. First off, I used Python and matplotlib basemap to create this map of everywhere I’ve ever been:

I also made an interactive web map with Leaflet that you can zoom into and click any point for details:

Overview of the data and process

I downloaded, processed, combined, and mapped my location history data from three sources:

1. My Google location history
2. My Foursquare/Swarm check-ins
3. A simple spreadsheet of pre-smart phone travels

As for the process itself, here is an overview of the steps (described in detail in the following sections):

1. Download and process my Foursquare/Swarm check-in history
2. Download my Google location history from their Takeout site
3. Cluster my Google location data set to reduce its size (for Leaflet)
4. Reverse-geocode this reduced Google location history data set
5. Create a simple spreadsheet of pre-smart phone travels, then geocode it
6. Combine the Foursquare check-in history, the Google location history, and the previous travels data and then map it with Python and Leaflet.

That’s the overview. Here are the details…

Foursquare and Google location data

First, I downloaded my entire Foursquare/Swarm check-in history from their API, wrangled it, and analyzed it with Python and matplotlib, as described in this post. I looked at the countries, cities, venues, and venue categories that I have checked-into the most often. Then I analyzed my check-in history over time by visualizing the check-ins per month, per day of the week, and per hour of the day.

Next I downloaded my Google location history from the Google Takeout site. It contains approximately 1.2 million GPS coordinates from my Android phone between 2012 and 2016. This huge JSON file works great as-is for mapping and visualizing in its entirety with Python, as I describe in this post.  I mapped my worldwide Google location history, as well as closer looks at California and the San Francisco Bay Area.

Clustering and reverse-geocoding

For this current project, I wanted to additionally visualize my Google location history data as an interactive Leaflet web map, but I can’t do that with 1.2 million data points in a 300mb data file. But most of these GPS coordinates are redundant, such as hundreds of thousands of duplicate points at my house accumulating over time.

So, I drastically reduced the size of this Google location history data set with DBSCAN clustering, using the code in this notebook. And see this prior blog post on reducing spatial data with clustering for full details. Clustering cut my data set down from 1.2 million points to the 3,500 most spatially-representative points (that’s a 99.7% reduction). Not only are 3,500 points easily mapped in JavaScript with Leaflet, but they’re also easily reverse-geocoded to get the neighborhood, city, state, and country for each data point, without spending weeks making a million API calls.

Accordingly, I reverse-geocoded each point in my cluster-reduced Google location history data set with the Google Maps reverse-geocoding API, using the code in this notebook. I added local caching of results so you only have to call the API once ever for a given set of coordinates. This makes repeated runs of the notebook nearly instantaneous (and no repeated API calls to count against your daily quota).

Putting it all together and mapping it

Finally, I created a simple spreadsheet of my pre-smart phone travels with columns for the place’s name, the city, country, and date. I geocoded this spreadsheet using the Google Maps geocoding API with the code in this notebook. Then I put all three processed data sets together: the Foursquare check-in history, the Google location history, and the previous travels. I mapped the combined location history data with Python’s matplotlib basemap toolkit and saved my data as GeoJSON for Leaflet web mapping – all using the code in this notebook. The final product can be seen at the beginning of this post.

My entire project is in this GitHub repo. Feel free to re-purpose it to visualize your own location history. You might also be interested in:

• Simply mapping your Google location history with Python
• Visualizing and mapping your Foursquare check-in history
• Clustering to reduce spatial data set size
• Reverse-geocoding latitude-longitude to city + country
• Exporting pandas DataFrames to GeoJSON
• Our course at UC Berkeley that teaches these skills and tools