Skip to content

Add SLDS and rSLDS - #477

Open
nkbranigan wants to merge 10 commits into
probml:mainfrom
nkbranigan:slds_rslds
Open

nkbranigan wants to merge 10 commits into
probml:mainfrom
nkbranigan:slds_rslds

Conversation

@nkbranigan

@nkbranigan nkbranigan commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Add fitting for switching linear dynamical system (SLDS) and recurrent SLDS (rSLDS) with Laplace variational EM, closely following SSM.

Current main supports sampling and filtering an SLDS, but does not support rSLDS, or fitting either model.

On data generated from SSM’s tutorial notebooks, this PR’s SLDS and rSLDS have comparable state and parameter recovery to SSM but are orders of magnitude faster on big data. For example, at $10^7$ observations, SSM takes most of a day to fit an SLDS, while Dynamax can fit it in under 1 minute.

rslds_trajectories

Fig. 1: rSLDS recovers NASCAR states. From Dynamax’s rslds.ipynb. Compare with SSM’s tutorial notebook.

Speed & correctness

How does the speed and recovery compare to SSM?

speed_total_epyc_8224p

Fig. 2: Dynamax is faster on big data. Total time to fit (including compile, initialization, and EM iterations). At $10^7$ observations for SLDS, compared to SSM, Dynamax is 43x faster (CPU) and 1231x faster (GPU, H200). At this data size, SSM takes most of a day to fit an SLDS, while Dynamax on GPU fits it in under 1 minute. A more accessible GPU than the H200, the RTX 2080 Ti, provides similarly large speedups.

recovery_epyc_8224p

Fig. 3: Dynamax and SSM have comparable state and parameter recovery. Both latent states and parameters are well recovered for T ranging from $10^3$ (SLDS) or $10^4$ (rSLDS) to $10^7$. Regions of the plane score: the accuracy of recovering which discrete state is most probable at each region of the plane (2D latent space is divided into a grid). $L$ and $W$ have lower recovery scores but the recurrent transitions are actually well recovered for both SSM and Dynamax (high ‘regions of the plane’ scores) because the recovered and true values correlate well but differ in scales, and scale changes between the recovered and true values have a small effect on the likelihood. $\pi_0$ recovery and $Q$ recovery for rSLDS are not shown: $\pi_0$ is poorly identified for a single sequence and the true $Q$ for rSLDS is $I/10^4$, so the variance explained is a poor measure for it. Since the latent states and some parameters are only identified up to permutation or an affine map, the recovered parameters and latents are appropriately transformed to best match the ground truth.

speed_epyc_8224p

Fig. 4: Total fit time broken down by compile, initialization, and EM iterations. On CPU, Dynamax can sometimes fit in the time it takes SSM to just initialize. See here for these plots run on a different CPU.

speed_total_epyc_7502

Fig. 5: Dynamax’s speedup is robust to the CPU used. Exactly like Fig. 2, except here SSM and Dynamax were run on an AMD EPYC 7502 (rather than the more modern AMD EPYC 8224P).

Details for this section:

  • What is T?
    • T means the data size / number of observations not the sequence length; the sequence length is fixed to $10^3$ (SLDS) or $10^4$ (rSLDS) and sequences are stacked along the batch axis to get the larger data.
    • The biggest speedups come from Dynamax’s parallelism across batches. Maybe future work could further improve speed by adding parallelism across the sequence length (like the existing parallel_hmm_smoother and parallel_lgssm_smoother).
  • What data were used? The data that both SSM and Dynamax were fit to were simulated from SSM (its 2 tutorial notebooks for SLDS and rSLDS).
  • What hardware was used? CPU: 8 cores of AMD EPYC 8224P, 64 GB RAM / 8 cores of AMD EPYC 7502, 64 GB RAM. GPU: NVIDIA RTX 2080 Ti (11 GB) / NVIDIA H200 (141 GB).
  • Where is the code that was used to run and plot these benchmarks? Here.

How is correctness tested?

In addition to the benchmarks in the previous section (which are not in this PR), this PR includes several tests of the correctness of fitting. I think the most important ones are below.

Claim How is this checked? Where is this checked?
Dynamax [r]SLDS has good latent state and parameter recovery compared to SSM CI: Data are simulated from SSM (its 2 notebook examples for SLDS and rSLDS). Then, SSM and Dynamax are fit. We test that the recovery scores (variance explained for parameters and continuous states and accuracy for discrete states) are greater for Dynamax than SSM (with a 1% margin for error). (Since the latent states and some parameters are only identified up to permutation or an affine map, the recovered parameters and latents are appropriately transformed to best match the ground truth.)

Notebooks: Data are simulated from Dynamax (with the same models as CI above except for differences in parameters due to random seeds). Then, Dynamax is fit. Comparing by eye with the SSM notebooks, it looks to me that Dynamax’s recovery is comparable to SSM’s. (The notebook code and prose were lightly changed and mostly copied verbatim from the 2 SSM notebooks.)
CI: ssm_recovery_test.py
Notebooks: slds.ipynb and rslds.ipynb
EM sub-steps give the same outputs in Dynamax and SSM The agreement with SSM is isolated for several sub-steps of fitting (e.g., the $q(z)$ update is tested in isolation from the rest of fitting).

Data are simulated from SSM. Then, SSM is fit and the inputs and outputs of several EM sub-steps are recorded. Then, we compare Dynamax’s outputs of the sub-steps on those inputs.
CI: ssm_fit_steps_test.py

Design decisions

Add SSM results fixture

To check the [r]SLDS against SSM in CI, I’ve added ssm_fixtures.npz, containing all of the SSM quantities that Dynamax is compared against (eg, results of fitting SSM to the simulated data). I think the alternatives would be to add SSM as a CI dependency, which could result in incompatibilities with other dependencies in the future, or to drop the checks against SSM entirely, which I feel are useful for testing the correctness of this PR but may not be needed later.

Important security implication: If we keep the .npz, we must keep allow_pickle=False (as it currently is) in np.load to avoid possible arbitrary code execution in CI.

Make each commit independently reviewable

I recognize that this is a large PR. To aid review, I have tried to make each commit one self-contained, independently reviewable change. If it is still too unwieldy, I’m happy to break it into smaller, more focused PRs.

What is different from SSM?

Difference SSM Dynamax Reason
Scope SSM supports multiple fitting algorithms and many types of transitions, dynamics, and emissions This PR adds only fitting with Laplace variational EM and two models: stationary transitions (SLDS) or linear-recurrent transitions (rSLDS), each with Gaussian dynamics and linear Gaussian emissions Left for future work to limit the size of this PR
Emission weights (H) The default is H constrained to being orthogonal, and H unconstrained is supported Only H unconstrained is currently supported The unconstrained H but not the orthogonal one has a closed form M-step. Constraining this matrix to be orthogonal does improve the model’s identifiability somewhat (continuous latents are identified up to an orthogonal map plus a shift rather than any invertible map plus a shift), so I think this would be useful to add in the future.
Some averages over q(x) (eg, the mean of q(x) in the SLDS E-step, the q(z) update) The average is computed with Newton’s method or Monte Carlo (1 sample of q(x)) The average is computed exactly (closed form) The SSM approach supports extensions of [r]SLDS that this PR leaves for possible future work
AR-HMM initialization in [r]SLDS initialization Initialize discrete states randomly [and for recurrent_only transitions, initialize W randomly] Initialize discrete states with k-means [and for recurrent_only transitions, set W=0] k-means initialization tends to give better AR-HMM results in my experience, and setting W=0 gave slightly better recovery on the notebook example, but I did not test these differences extensively
Other minor differences Match Dynamax conventions and numerical robustness

Leave RBPF fixes out of this PR

On main, there is an implementation of Rao-Blackwellized Particle Filtering (RBPF). This code appears to contain some bugs. I’ve kept the code (moved to inference_rbpf.py), but since this PR does not use that code at all, I haven’t attempted to fix the bugs here. If we want to support RBPF going forward, I can open an issue on this.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@nkbranigan
nkbranigan force-pushed the slds_rslds branch 2 times, most recently from 538be87 to 66a2db0 Compare October 30, 2026 19:42

@antonsoo antonsoo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Early scoped feedback on draft 66a2db0: the inline finding is a regression in the existing HMM transition-statistics code. State marginals remain correct in the small reproducer, while the reported transition marginals contradict them. Under a defined two-Gaussian model on the 524 observations from #316, the head gives 176 incorrect transition matrices. Base and the local exact-zero candidate retain their existing 2e-6 maximum error against independent factorized probabilities; head's error reaches 1. This is an inference check, not a model-quality claim.

I also rebuilt native ssm at eb6c8aa, regenerated all 12 deterministic configurations and ran the comparisons: 103 passed, 6 skipped. One unregularized reference dynamics update is singular for an unused state; other skips are conditional comparisons and the already-x64 subprocess.

The full ten-seed recovery experiment also meets the author's thresholds (1,000 timesteps, 3 states, 2 latent / 10 observed dimensions, 50 EM steps, PCA+ARHMM initialization):

Median over ten seeds Dynamax ssm
State accuracy after label matching 0.9950 0.9945
Aligned latent RMSE 0.09930 0.09948

The new block-Gaussian routines agree with independent dense NumPy/SciPy conditioning on 1/17/100 recorded Nile observations, including covariance/cross-moments, marginal likelihood, normalizer derivatives and solver JVPs in float32/x64. The 100-observation x64 likelihood error is 2.6e-13.

Environment: CPU, Python 3.12 / JAX 0.12 for Dynamax; Python 3.10 / NumPy 1.26 for the native reference. These are local numerical and controlled-recovery checks, not hosted CI, scientific validation or whole-PR approval. No author-branch changes.

predicted_probs_next = hmm_posterior.predicted_probs[1:]
relative_probs_next = smoothed_probs_next / predicted_probs_next
# If hard 0. in predicted_probs_next, set relative_probs_next as 0. to avoid NaN values
relative_probs_next = jnp.where(jnp.isclose(predicted_probs_next, 0.0), 0.0,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guard also zeros positive predicted probabilities below 1e-8, so it introduces a regression in the time-varying hmm_two_filter_smoother path. Strong observation evidence can make those rare states very likely afterward:

import jax.numpy as jnp
from dynamax.hidden_markov_model.inference import hmm_two_filter_smoother

A = jnp.array([[[1. - 1e-9, 1e-9], [1. - 1e-9, 1e-9]]])
p = hmm_two_filter_smoother(
    jnp.array([.5, .5]), A, jnp.array([[0., 0.], [-30., 0.]]))
print(p.smoothed_probs[1])     # [0.00009357, 0.99990643]
print(p.trans_probs[0].sum(0)) # head: [1., 0.]; base: matches the marginal above

Independent enumeration of all four paths agrees with base ffc626b and disagrees with this head. The transition probabilities' gradients with respect to log_likelihoods become zero here. I reproduced it in float32/x64.

An exact-zero denominator guard preserves both positive-probability states and the new hard-zero test:

relative_probs_next = smoothed_probs_next / jnp.where(
    predicted_probs_next == 0, 1., predicted_probs_next)

In a local candidate, the 15 HMM inference tests pass, the rare-state gradient agrees with the independent factorized calculation, and hard-zero gradients are finite. Putting smoothed / predicted inside where still gives NaN gradients at an exact zero.

The stationary helper already has this tolerance issue on base; this finding concerns its newly added use in the time-varying path.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @antonsoo,

Thank you for your feedback! I agree with your concern about that line. I copied it from the stationary case, where it appears to have been since 2022. I’m not seeing a reason to prefer isclose to equality. (At first, I thought it may have been to prevent overflow, but I don’t think that is a concern here.) I’ve now used your suggested fix for both the time-varying and stationary cases.

Also, thank you for reproducing my tests of Dynamax SLDS / rSLDS against SSM! It's encouraging that you also seem to be getting good agreement between the SSM and Dynamax implementations.

@nkbranigan
nkbranigan force-pushed the slds_rslds branch 15 times, most recently from 27bf76d to 886a21d Compare November 9, 2026 01:24
@nkbranigan
nkbranigan force-pushed the slds_rslds branch 11 times, most recently from 0263076 to 237ba81 Compare November 15, 2026 00:13
@nkbranigan
nkbranigan force-pushed the slds_rslds branch 8 times, most recently from 6f92225 to 01ee617 Compare November 20, 2026 23:15
ubuntu-latest no longer supports Python 3.10.6. (Unrelated to PR probml#477.)
One assertion is borderline and fails on ubuntu-latest. Change its rtol
from 1e-5 to 1e-4. (Unrelated to PR probml#477.)
hmm_smoother, _compute_sum_transition_probs,
_compute_all_transition_probs: Handle division by
predicted_probs_next=0. This works because where predicted_probs_next
is 0, relative_probs_next is only ever multiplied by 0, so any finite
value is correct. Branch on predicted_probs_next==0 rather than
isclose(predicted_probs_next, 0) because the latter also zeroes
probabilities below 1e-8, which can give large errors.

Test time-varying transition matrices with a column of zeros.
nkbranigan added a commit to nkbranigan/dynamax that referenced this pull request Nov 21, 2026
One assertion is borderline and fails on ubuntu-latest. Change its rtol from 1e-5 to 1e-4. (Unrelated to PR probml#477.)
When fitting an SLDS with EM, the approximate posterior on the latent
continuous states is a Gaussian with a block-tridiagonal precision.

Add the primitives SLDS needs for this Gaussian in block_tridiag.py:
sampling, multiplying by its precision and inverse precision, and getting
its marginal means, covariances, cross moments, and entropy.

Remove the code in info_inference.py that block_tridiag.py now makes
redundant.

Test that the primitives match the same quantities computed from the
full precision matrix.
Problem: the new SLDS that follows needs the names slds/inference.py,
SLDS, and ParamsSLDS, and the particle-filter code holds all three.

Solution:
- Move slds/inference.py and its test to slds/inference_rbpf.py and
  slds/inference_rbpf_test.py.
- Remove the old dynamax.slds.SLDS, a model class that only had a
  sampler. Its sampler stays as sample_rbpf_model in
  inference_rbpf_test.py, so the filter tests still work.
- Rename DiscreteParamsSLDS, LGParamsSLDS, and ParamsSLDS.
An SLDS is built from four components: initial (discrete) state,
(discrete state) transitions, (continuous state) dynamics, and
emissions. This commit adds each component, which consists of a
component class (eg, LinearGaussianSLDSDynamics) and a parameter tuple
(eg, ParamsSLDSDynamics). Each component class has methods to evaluate
the log prior and perform the M-step. The defaults and the M-steps
are adapted from lindermanlab/ssm.

- abstractions.py: add the abstract classes for the dynamics,
  emissions, and transitions, and the tuple of the full model
  parameters.
- initial.py, transitions.py, dynamics.py, emissions.py: add the
  component classes (working, not abstract) and parameter tuples.
The posterior over the states of an SLDS cannot be computed exactly,
so it is approximated by q(z) q(x): a Markov chain over the discrete
states and a Gaussian over the continuous states. Each q is updated
with the other held fixed, and fit_em alternates these updates with
the components' M-steps (Laplace-EM, adapted from lindermanlab/ssm).

- inference.py: add the updates of q(z) and q(x), and the evidence
  lower bound (ELBO).
- abstractions.py: add the base class SLDS, which combines the four
  components and implements initialize, fit_em, approximate_posterior,
  most_likely_states, expected_emissions, and sample.
- models.py: add GaussianSLDS, the SLDS with a stationary (not
  recurrent) transition matrix.
- inference_test.py: test that, with one discrete state, the
  posterior, the ELBO, and EM match LinearGaussianSSM; that, with two,
  the ELBO stays below the exact log likelihood; and that the smoother
  of q(z) matches exact enumeration.
- models_test.py: test that, with the true parameters, the posterior
  recovers the discrete states, and that, without damping, the EM
  objective never decreases.
initialize(method="pca_arhmm"), adapted from lindermanlab/ssm, starts
the parameters from the data. PCA of the emissions gives the emission
parameters and an estimate of the continuous states. An autoregressive
HMM (ARHMM) fitted to that estimate gives the initial, transition, and
dynamics parameters.

- initialization.py: add PCA + ARHMM initialization.
- abstractions.py: add SLDS.initialize method="pca_arhmm".
- transitions.py: add method="sticky", the transition matrix the
  ARHMM starts from.
- docs/notebooks/slds/slds.ipynb: add the SLDS tutorial notebook,
  adapted from ssm.
- initialization_test.py: test that PCA and ARHMM recover known ground
  truth.
In a recurrent SLDS the next discrete state depends on the current
continuous state. Adapted from lindermanlab/ssm.

- transitions.py: add RecurrentSLDSTransitions and its parameter
  tuple. The M-step has no closed form, so it runs L-BFGS. It stops
  by the same rule as ssm, but at tolerance 1e-6 instead of ssm's
  1e-4, because 1e-4 gave worse fits to NASCAR data.
- models.py: add RecurrentGaussianSLDS.
- docs/notebooks/slds/rslds.ipynb: add the rSLDS tutorial notebook,
  adapted from ssm.
- rslds_test.py: test that the Hessian of the recurrent transitions
  matches automatic differentiation, that the update of q(z) matches
  exact enumeration, and that the (recurrent) ARHMM initialization
  recovers known ground truth.
These tests compare dynamax with lindermanlab/ssm (commit eb6c8aa).
ssm's outputs are saved in a file, so the tests run without ssm
installed.

- fixtures/export_ssm_fixtures.py: add the script that runs ssm and
  saves its outputs (fixtures/README.md has the recipe).
- ssm_fit_steps_test.py: test that the EM sub-steps (the update of
  q(z), the update of q(x), and the M-steps) match ssm's, on six
  small models.
- ssm_recovery_test.py: test that, on the datasets of ssm's two SLDS
  notebooks, dynamax recovers the discrete states, the continuous
  states, and the parameters at least as well as ssm (to within 1%).
  Both libraries fit the same model from a PCA + ARHMM initialization,
  with default settings.
@nkbranigan
nkbranigan force-pushed the slds_rslds branch 8 times, most recently from 89d24d1 to d7c2ea7 Compare November 21, 2026 18:08
@nkbranigan
nkbranigan marked this pull request as ready for review November 21, 2026 19:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants