Categories
Tech

Scientific Python for Raspberry Pi

Raspberry Pi 3 Model BA guide to setting up the Python scientific stack, well-suited for geospatial analysis, on a Raspberry Pi 3. The whole process takes just a few minutes.

The Raspberry Pi 3 was announced two weeks ago and presents a substantial step up in computational power over its predecessors. It can serve as a functional Wi-Fi connected Linux desktop computer, albeit underpowered. However it’s perfectly capable of running the Python scientific computing stack including Jupyter, pandas, matplotlib, scipy, scikit-learn, and OSMnx.

Despite (or because of?) its low power, it’s ideal for low-overhead and repetitive tasks that researchers and engineers often face, including geocoding, web scraping, scheduled API calls, or recurring statistical or spatial analyses (with small-ish data sets). It’s also a great way to set up a simple server or experiment with Linux. This guide is aimed at newcomers to the world of Raspberry Pi and Linux, but who have an interest in setting up a Python environment on these $35 credit card sized computers. We’ll run through everything you need to do to get started (if your Pi is already up and running, skip steps 1 and 2).

Step 1: Get the hardware

Assuming you have an available phone charger, HDMI cable, mouse, and keyboard, you can buy everything else you need to get up and running for under raspberry-pi-3-b$45. Here’s what you’ll need:

  1. A Raspberry Pi ($35)
  2. A 5-volt 1-amp power supply (I just used an old Android charger with a micro-USB cable, otherwise about $5)
  3. A micro SD card with a full-size SD adapter (about $9)
  4. An HDMI cable to connect your monitor (I already had one, otherwise about $5)
  5. USB mouse/keyboard (I already had them, otherwise about $15 for a basic set)
  6. Optional: if you’re running your Raspberry Pi side-by-side with a desktop computer, you can get a cheap USB switch to switch your mouse/keyboard from the computer to the Raspberry Pi and back

Step 2: Install Raspbian OS on the Raspberry Pi

Now we’ll install Raspbian (the Debian Linux OS tailored for Raspberry Pi) the easy way, then boot the OS and connect to Wi-Fi.

  1. Pop your micro SD card into its full-size SD adapter sleeve and stick it in your computer
  2. Download the NOOBS installer for Raspbian and un-zip it to your desktop
  3. Download SDFormatter and install it (this tool is particularly useful if you eventually want to refresh your Raspberry Pi system, as the Linux partitions otherwise might be tricky to work with on your desktop computer)
  4. Open SDFormatter, choose the SD card drive, click format
  5. When the formatting is done, copy the NOOBS files from your desktop to the SD card
  6. Pop the SD card out of the adapter sleeve and into the Raspberry Pi
  7. Connect your mouse, keyboard, HDMI, and power supply to the Raspberry Pi
  8. Once NOOBS boots up, choose your language, select Raspbian, then click install

When the installation is finished, click ok and the new OS will boot up. The Raspberry Pi 3 has Wi-Fi: in the top-right of the screen, click the Wi-Fi networks panel item and choose your network to connect.

Step 3: Update packages

Next we update the existing software. Open a terminal window and run the following commands, one at a time. The first line fetches updated package lists from the repositories, and the second then fetches new versions of installed packages. The last two lines list the installed system packages and installed Python packages and dump them to files, just for reference.

sudo apt-get update
sudo apt-get upgrade
dpkg -l > ~/Desktop/packages.list
pip freeze > ~/Desktop/pip-freeze-initial.list

apt-get is a Debian tool to install and update software packages. We’ll use it instead of pip wherever we can because the packages come pre-compiled, meaning they take seconds rather than minutes to install. When a Python package isn’t available via apt-get, we’ll fall back on using pip to install (and compile) it.

Step 4: Install the Python basics

As we saw in the previous step’s file output, the Raspberry Pi comes with several Python packages already installed. We need to supplement it with a few more prerequisites. In the terminal window, run this command:

sudo apt-get install build-essential python-dev python-distlib python-setuptools python-pip python-wheel libzmq-dev libgdal-dev

The build-essential package is required for building Debian packages; python-dev, python-distlib, and python-setuptools provide several Python development and packaging tools; python-pip and python-wheel are useful for installing Python packages; libzmq-dev is needed for Jupyter notebooks; libgdal-dev is needed for geospatial analysis with geopandas.

Step 5: Install pandas dependencies

Pandas has several recommended and optional dependencies that unlock functionality or provide significant performance enhancements. To install them all, run the following two commands:

sudo apt-get install xsel xclip libxml2-dev libxslt-dev python-lxml python-h5py python-numexpr python-dateutil python-six python-tz python-bs4 python-html5lib python-openpyxl python-tables python-xlrd python-xlwt cython python-sqlalchemy python-xlsxwriter python-jinja2 python-boto python-gflags python-googleapi python-httplib2 python-zmq libspatialindex-dev
sudo pip install bottleneck rtree

The first command uses apt-get to install the available recommended dependencies, and the second command uses pip to install the two that are not available in the apt repositories.

Step 6: Install the scientific Python stack

Fortunately we can use apt-get to install all the massive, complex packages that make up the Python scientific stack without having to compile everything. This makes the process much, much faster.

sudo apt-get install python-numpy python-matplotlib python-mpltoolkits.basemap python-scipy python-sklearn python-statsmodels python-pandas

If you need a specific version of these packages or want a more up-to-date version than exists in the Debian repositories, you can use pip to install it, but be prepared for a slow compilation process.

Step 7 (optional): Install other useful packages

We’re all done! But if you optionally would like to install a few more useful packages, run the following two commands:

sudo apt-get install python-requests python-pil python-scrapy python-geopy python-shapely python-pyproj
sudo pip install jupyter geopandas osmnx

The requests Python package provides a graceful interface for making HTTP requests, pil provides Python imaging capabilities, scrapy is a web scraping framework, geopy provides geocoding and geodesic distance functions, shapely provides 2D geometry manipulation, and pyproj provides cartographic transformations. In the second command, jupyter provides interactive coding notebooks, geopandas spatializes pandas, and OSMnx lets you work with OpenStreetMap street networks.

Wrapping up

Our Python scientific stack is now all ready to use on the Raspberry Pi. Launch a Jupyter notebook, load up some data with pandas, or plot a map with basemap. Due to the Raspberry Pi’s memory constraints, you cannot load huge data sets, but everything else works great. It’s particularly good for repetitive, scheduled, or low-overhead tasks such as geocoding and web scraping.

25 replies on “Scientific Python for Raspberry Pi”

Very useful, thank you. Would be something similar for python3? I guess that changing “python-package” for “python3-package” would do the trick, but I am not sure it is possible to make the full stack without problems.

In the other hand, it is useful to remember that we could use pandas “chuncksize” to overcome a little bit the RAM limitation.

Yes, great point about chunksize. In general, “python3-package” should give you the python 3 version of most of these packages. Haven’t tested it yet but I will soon. Interested to hear if anyone’s had succcess with python 3 on the Pi…

I was able to install the Python 3 versions of all the packages using the python3-package method on a Raspberry Pi 3B with the latest version of the ‘legacy’ OS. As far as I know (and I have tried many things) this is the only approach to installing Python data science packages on Raspberry Pi that works.

As of today, these are the package versions this gave me:
python: 3.7.3
numpy: 1.16.2
scipy: 1.1.0
matplotlib: 3.0.2
pandas: 0.23.3
statsmodels: 0.8.0
sklearn: 0.20.2

Thanks very much.

Nice!

It will be most interesting to see how this board works when full 64 bit support is enabled. There is enough additional hardware that I’m hoping things would run a bit faster.

Unfortunately, only a small handful of packages are available through Anaconda/Miniconda on Raspberry Pi, and those built for the new ARM hardware in the Pi 3 are even more limited as of now.

Thanks for laying this out so clearly.
I got an error running this:
sudo pip install bottleneck rtree

ImportError: cannot import name nanmedian

Any advice?

Weird error. Try installing each package separately. That should at least isolate the source of the error. Bottom line, you don’t *need* bottleneck or rtree – they just enhance pandas/geopandas speed and functionality.

Hi thank you for the super clear guide! I am trying to launch a Jupiter notebook (by giving the terminal the command Jupiter notebook, but it doesn’t open anything it just answer me command not found) can you help me? Should I install something else to use it?

Geoff:

One more thanks for your contribution. I like having all this info in one place. I am a long-time user of Matlab and am slowly converting my favorite matlab routines to Python.

Chuck Young (professor emeritus of geophysical engineering,
Michigan Technological University).

Thanks so much. I was getting frustrated with my inability to install sklearn on my Pi 3. Lots of instructions on the web failed because they made assumptions about my previous installations. I appreciate the thorough walk through and descriptions.

Thanks for the instruction.
I think your instruction is the best i can find so far. I’m very disappointed on pi3’s limited ability to work with python, especially the lack of support from anaconda. I brought my pi3 specifically to make it a small, quite, low energy consumption data mining server. It seems like there going to be a lot of challenges I need to overcome.

Great. I have used this a few times but today I tried something different. Downloaded and installed Miniconda (3) onto my Raspberry Pi 2. It’s a slim version of Continuum’s Anaconda package manager. I wouldn’t say it was painless but it seems to work now and should make the updating easier.

Thanks for this guide. I don’t usually use my Pi for this sort of thing, I have some reasonable kit at work to play with, (6x Dell workstations with Xeon CPU and Nvidia 192 core GPU :) ), but I started playing with opencv on the Pi and I needed to get some of these packages installed but was having issues. All ok now!

I’ve been using this procedure for a while and it’s great but my numpy is only version 1.8.2 (pandas 0.14.1). How do I upgrade them all to a more recent version? I tried ‘sudo apt-get update’ and ‘sudo apt-get upgrade’ but that doesn’t change the numpy version. Thanks.

Having followed the outline precisely to the end on my RPi 3B+ running (NOOBS) Raspbian, at the end, this error:
‘jupyter-console requires Python ‘>=3.5′ but the running Python is 2.7.13’

Simply adding a 3 after pip; i.e.,
‘sudo pip3 install jupyter geopandas osmnx’
produced additional errors:
‘Command “python setup.py egg_info” failed with error code 1 in /tmp/pip-build-gk_1qso5/pyproj/’

I’m at a loss as to what to try next, or how to purge and clean, if necessary, in order to have another go at it. Please consider updating this article to address the conflicts which only manifested at the tail end of your outline.

Thanks so much!!

Leave a Reply to GhostZodickCancel reply