Allen Downey and Alexander Fengler
In this tutorial, we explore Bayesian regression using PyMC -- the primary library for Bayesian sampling in Python -- focusing on survey data and other datasets with categorical outcomes. Starting with logistic regression, we'll build up to categorical and ordered logistic regression, showcasing how Bayesian approaches provide versatile tools for developing and evaluating complex models. Participants will leave with practical skills for implementing Bayesian regression models in PyMC, along with a deeper appreciation for the power of Bayesian inference in real-world data analysis.
The examples use data from the General Social Survey (GSS).
The tutorial site has the notebooks with solutions. This is one of my workshops and tutorials. Its companion, Survey Data with Pandas, uses the same GSS data.
Level: intermediate. You should be comfortable with Python, NumPy and pandas, and basic statistics. You don't need any experience with Bayesian methods or PyMC.
Length: about 90 minutes.
- Introduction to Bayesian sampling: using PyMC as a random number generator, then building up a model from an uncertain probability, a latent propensity, and observed data.
- Logistic regression: modeling a binary outcome, whether people say other people can be trusted, as a function of time and political party. Priors, the posterior distribution, sampling diagnostics, centering, and generating predictions.
- Categorical regression: extending logistic regression to an unordered outcome with more than two categories, the 2020 presidential vote, as a function of age and gender.
- Ordinal regression: modeling an ordered outcome, how happy people say they are, with cutpoints, as a function of age and birth cohort.
Each notebook comes in two versions: one with blanks for the exercises, and one with solutions.
| Notebook | Exercises | Solutions |
|---|---|---|
| 0. Introduction to Bayesian sampling | Run on Colab | Run on Colab |
| 1. Logistic regression | Run on Colab | Run on Colab |
| 2. Categorical regression | Run on Colab | Run on Colab |
| 3. Ordinal regression | Run on Colab | Run on Colab |
There is also a bonus notebook, still a draft, on a hierarchical categorical model of vote by gender identity.
Slides for this tutorial are here.
The notebooks download gss_extract_2022_4.hdf, an extract of the GSS (2022 release 4) made in the GssExtract repository.
We presented this tutorial at ODSC East 2025 (May 13, 2025). It was not recorded. The version of the code we presented is tagged odsc-east-2025.
The easiest way to run the notebooks is on Colab: click one of the links above. You need a Google account, but you don't have to install anything. The notebooks install nutpie, the sampler they use, and download the data they need.
To run them on your own computer, you need Python 3 with Jupyter, NumPy, pandas, Matplotlib, SciPy, PyTables, PyMC 5, ArviZ (before version 1), nutpie, and Graphviz. You can download the code in this zip file, or clone the repository:
git clone https://github.com/AllenDowney/SurveyDataPyMC
cd SurveyDataPyMC
Then install the packages with pip:
pip install -r requirements.txt jupyter
Or, if you use conda or mamba, create an environment:
mamba env create -f environment.yml
mamba activate SurveyDataPyMC
Then run jupyter lab and open the notebooks in the notebooks folder.
- Survey Data with Pandas, the companion tutorial: exploring the same GSS data with pandas and StatsModels.
- Think Bayes, which is free online.
- The PyMC documentation and example gallery.
- Statistical Rethinking by Richard McElreath.
- My other workshops.
The code is available under the MIT License. The text is available under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.