Repository navigation
Add SLDS and rSLDS - #477
nkbranigan wants to merge 10 commits into
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
538be87 to
66a2db0
Compare
antonsoo
left a comment
There was a problem hiding this comment.
Early scoped feedback on draft 66a2db0: the inline finding is a regression in the existing HMM transition-statistics code. State marginals remain correct in the small reproducer, while the reported transition marginals contradict them. Under a defined two-Gaussian model on the 524 observations from #316, the head gives 176 incorrect transition matrices. Base and the local exact-zero candidate retain their existing 2e-6 maximum error against independent factorized probabilities; head's error reaches 1. This is an inference check, not a model-quality claim.
I also rebuilt native ssm at eb6c8aa, regenerated all 12 deterministic configurations and ran the comparisons: 103 passed, 6 skipped. One unregularized reference dynamics update is singular for an unused state; other skips are conditional comparisons and the already-x64 subprocess.
The full ten-seed recovery experiment also meets the author's thresholds (1,000 timesteps, 3 states, 2 latent / 10 observed dimensions, 50 EM steps, PCA+ARHMM initialization):
| Median over ten seeds | Dynamax | ssm |
|---|---|---|
| State accuracy after label matching | 0.9950 | 0.9945 |
| Aligned latent RMSE | 0.09930 | 0.09948 |
The new block-Gaussian routines agree with independent dense NumPy/SciPy conditioning on 1/17/100 recorded Nile observations, including covariance/cross-moments, marginal likelihood, normalizer derivatives and solver JVPs in float32/x64. The 100-observation x64 likelihood error is 2.6e-13.
Environment: CPU, Python 3.12 / JAX 0.12 for Dynamax; Python 3.10 / NumPy 1.26 for the native reference. These are local numerical and controlled-recovery checks, not hosted CI, scientific validation or whole-PR approval. No author-branch changes.
| predicted_probs_next = hmm_posterior.predicted_probs[1:] | ||
| relative_probs_next = smoothed_probs_next / predicted_probs_next | ||
| # If hard 0. in predicted_probs_next, set relative_probs_next as 0. to avoid NaN values | ||
| relative_probs_next = jnp.where(jnp.isclose(predicted_probs_next, 0.0), 0.0, |
There was a problem hiding this comment.
This guard also zeros positive predicted probabilities below 1e-8, so it introduces a regression in the time-varying hmm_two_filter_smoother path. Strong observation evidence can make those rare states very likely afterward:
import jax.numpy as jnp
from dynamax.hidden_markov_model.inference import hmm_two_filter_smoother
A = jnp.array([[[1. - 1e-9, 1e-9], [1. - 1e-9, 1e-9]]])
p = hmm_two_filter_smoother(
jnp.array([.5, .5]), A, jnp.array([[0., 0.], [-30., 0.]]))
print(p.smoothed_probs[1]) # [0.00009357, 0.99990643]
print(p.trans_probs[0].sum(0)) # head: [1., 0.]; base: matches the marginal aboveIndependent enumeration of all four paths agrees with base ffc626b and disagrees with this head. The transition probabilities' gradients with respect to log_likelihoods become zero here. I reproduced it in float32/x64.
An exact-zero denominator guard preserves both positive-probability states and the new hard-zero test:
relative_probs_next = smoothed_probs_next / jnp.where(
predicted_probs_next == 0, 1., predicted_probs_next)In a local candidate, the 15 HMM inference tests pass, the rare-state gradient agrees with the independent factorized calculation, and hard-zero gradients are finite. Putting smoothed / predicted inside where still gives NaN gradients at an exact zero.
The stationary helper already has this tolerance issue on base; this finding concerns its newly added use in the time-varying path.
There was a problem hiding this comment.
Hi @antonsoo,
Thank you for your feedback! I agree with your concern about that line. I copied it from the stationary case, where it appears to have been since 2022. I’m not seeing a reason to prefer isclose to equality. (At first, I thought it may have been to prevent overflow, but I don’t think that is a concern here.) I’ve now used your suggested fix for both the time-varying and stationary cases.
Also, thank you for reproducing my tests of Dynamax SLDS / rSLDS against SSM! It's encouraging that you also seem to be getting good agreement between the SSM and Dynamax implementations.
27bf76d to
886a21d
Compare
0263076 to
237ba81
Compare
6f92225 to
01ee617
Compare
ubuntu-latest no longer supports Python 3.10.6. (Unrelated to PR probml#477.)
01ee617 to
4bd3c23
Compare
One assertion is borderline and fails on ubuntu-latest. Change its rtol from 1e-5 to 1e-4. (Unrelated to PR probml#477.)
hmm_smoother, _compute_sum_transition_probs, _compute_all_transition_probs: Handle division by predicted_probs_next=0. This works because where predicted_probs_next is 0, relative_probs_next is only ever multiplied by 0, so any finite value is correct. Branch on predicted_probs_next==0 rather than isclose(predicted_probs_next, 0) because the latter also zeroes probabilities below 1e-8, which can give large errors. Test time-varying transition matrices with a column of zeros.
One assertion is borderline and fails on ubuntu-latest. Change its rtol from 1e-5 to 1e-4. (Unrelated to PR probml#477.)
4bd3c23 to
01b1495
Compare
When fitting an SLDS with EM, the approximate posterior on the latent continuous states is a Gaussian with a block-tridiagonal precision. Add the primitives SLDS needs for this Gaussian in block_tridiag.py: sampling, multiplying by its precision and inverse precision, and getting its marginal means, covariances, cross moments, and entropy. Remove the code in info_inference.py that block_tridiag.py now makes redundant. Test that the primitives match the same quantities computed from the full precision matrix.
Problem: the new SLDS that follows needs the names slds/inference.py, SLDS, and ParamsSLDS, and the particle-filter code holds all three. Solution: - Move slds/inference.py and its test to slds/inference_rbpf.py and slds/inference_rbpf_test.py. - Remove the old dynamax.slds.SLDS, a model class that only had a sampler. Its sampler stays as sample_rbpf_model in inference_rbpf_test.py, so the filter tests still work. - Rename DiscreteParamsSLDS, LGParamsSLDS, and ParamsSLDS.
An SLDS is built from four components: initial (discrete) state, (discrete state) transitions, (continuous state) dynamics, and emissions. This commit adds each component, which consists of a component class (eg, LinearGaussianSLDSDynamics) and a parameter tuple (eg, ParamsSLDSDynamics). Each component class has methods to evaluate the log prior and perform the M-step. The defaults and the M-steps are adapted from lindermanlab/ssm. - abstractions.py: add the abstract classes for the dynamics, emissions, and transitions, and the tuple of the full model parameters. - initial.py, transitions.py, dynamics.py, emissions.py: add the component classes (working, not abstract) and parameter tuples.
The posterior over the states of an SLDS cannot be computed exactly, so it is approximated by q(z) q(x): a Markov chain over the discrete states and a Gaussian over the continuous states. Each q is updated with the other held fixed, and fit_em alternates these updates with the components' M-steps (Laplace-EM, adapted from lindermanlab/ssm). - inference.py: add the updates of q(z) and q(x), and the evidence lower bound (ELBO). - abstractions.py: add the base class SLDS, which combines the four components and implements initialize, fit_em, approximate_posterior, most_likely_states, expected_emissions, and sample. - models.py: add GaussianSLDS, the SLDS with a stationary (not recurrent) transition matrix. - inference_test.py: test that, with one discrete state, the posterior, the ELBO, and EM match LinearGaussianSSM; that, with two, the ELBO stays below the exact log likelihood; and that the smoother of q(z) matches exact enumeration. - models_test.py: test that, with the true parameters, the posterior recovers the discrete states, and that, without damping, the EM objective never decreases.
initialize(method="pca_arhmm"), adapted from lindermanlab/ssm, starts the parameters from the data. PCA of the emissions gives the emission parameters and an estimate of the continuous states. An autoregressive HMM (ARHMM) fitted to that estimate gives the initial, transition, and dynamics parameters. - initialization.py: add PCA + ARHMM initialization. - abstractions.py: add SLDS.initialize method="pca_arhmm". - transitions.py: add method="sticky", the transition matrix the ARHMM starts from. - docs/notebooks/slds/slds.ipynb: add the SLDS tutorial notebook, adapted from ssm. - initialization_test.py: test that PCA and ARHMM recover known ground truth.
In a recurrent SLDS the next discrete state depends on the current continuous state. Adapted from lindermanlab/ssm. - transitions.py: add RecurrentSLDSTransitions and its parameter tuple. The M-step has no closed form, so it runs L-BFGS. It stops by the same rule as ssm, but at tolerance 1e-6 instead of ssm's 1e-4, because 1e-4 gave worse fits to NASCAR data. - models.py: add RecurrentGaussianSLDS. - docs/notebooks/slds/rslds.ipynb: add the rSLDS tutorial notebook, adapted from ssm. - rslds_test.py: test that the Hessian of the recurrent transitions matches automatic differentiation, that the update of q(z) matches exact enumeration, and that the (recurrent) ARHMM initialization recovers known ground truth.
These tests compare dynamax with lindermanlab/ssm (commit eb6c8aa). ssm's outputs are saved in a file, so the tests run without ssm installed. - fixtures/export_ssm_fixtures.py: add the script that runs ssm and saves its outputs (fixtures/README.md has the recipe). - ssm_fit_steps_test.py: test that the EM sub-steps (the update of q(z), the update of q(x), and the M-steps) match ssm's, on six small models. - ssm_recovery_test.py: test that, on the datasets of ssm's two SLDS notebooks, dynamax recovers the discrete states, the continuous states, and the parameters at least as well as ssm (to within 1%). Both libraries fit the same model from a PCA + ARHMM initialization, with default settings.
89d24d1 to
d7c2ea7
Compare
Add fitting for switching linear dynamical system (SLDS) and recurrent SLDS (rSLDS) with Laplace variational EM, closely following SSM.
Current main supports sampling and filtering an SLDS, but does not support rSLDS, or fitting either model.
On data generated from SSM’s tutorial notebooks, this PR’s SLDS and rSLDS have comparable state and parameter recovery to SSM but are orders of magnitude faster on big data. For example, at$10^7$ observations, SSM takes most of a day to fit an SLDS, while Dynamax can fit it in under 1 minute.
Fig. 1: rSLDS recovers NASCAR states. From Dynamax’s rslds.ipynb. Compare with SSM’s tutorial notebook.
Speed & correctness
How does the speed and recovery compare to SSM?
Fig. 2: Dynamax is faster on big data. Total time to fit (including compile, initialization, and EM iterations). At$10^7$ observations for SLDS, compared to SSM, Dynamax is 43x faster (CPU) and 1231x faster (GPU, H200). At this data size, SSM takes most of a day to fit an SLDS, while Dynamax on GPU fits it in under 1 minute. A more accessible GPU than the H200, the RTX 2080 Ti, provides similarly large speedups.
Fig. 3: Dynamax and SSM have comparable state and parameter recovery. Both latent states and parameters are well recovered for T ranging from$10^3$ (SLDS) or $10^4$ (rSLDS) to $10^7$ . Regions of the plane score: the accuracy of recovering which discrete state is most probable at each region of the plane (2D latent space is divided into a grid). $L$ and $W$ have lower recovery scores but the recurrent transitions are actually well recovered for both SSM and Dynamax (high ‘regions of the plane’ scores) because the recovered and true values correlate well but differ in scales, and scale changes between the recovered and true values have a small effect on the likelihood. $\pi_0$ recovery and $Q$ recovery for rSLDS are not shown: $\pi_0$ is poorly identified for a single sequence and the true $Q$ for rSLDS is $I/10^4$ , so the variance explained is a poor measure for it. Since the latent states and some parameters are only identified up to permutation or an affine map, the recovered parameters and latents are appropriately transformed to best match the ground truth.
Fig. 4: Total fit time broken down by compile, initialization, and EM iterations. On CPU, Dynamax can sometimes fit in the time it takes SSM to just initialize. See here for these plots run on a different CPU.
Fig. 5: Dynamax’s speedup is robust to the CPU used. Exactly like Fig. 2, except here SSM and Dynamax were run on an AMD EPYC 7502 (rather than the more modern AMD EPYC 8224P).
Details for this section:
How is correctness tested?
In addition to the benchmarks in the previous section (which are not in this PR), this PR includes several tests of the correctness of fitting. I think the most important ones are below.
Notebooks: Data are simulated from Dynamax (with the same models as CI above except for differences in parameters due to random seeds). Then, Dynamax is fit. Comparing by eye with the SSM notebooks, it looks to me that Dynamax’s recovery is comparable to SSM’s. (The notebook code and prose were lightly changed and mostly copied verbatim from the 2 SSM notebooks.)
Notebooks: slds.ipynb and rslds.ipynb
Data are simulated from SSM. Then, SSM is fit and the inputs and outputs of several EM sub-steps are recorded. Then, we compare Dynamax’s outputs of the sub-steps on those inputs.
Design decisions
Add SSM results fixture
To check the [r]SLDS against SSM in CI, I’ve added ssm_fixtures.npz, containing all of the SSM quantities that Dynamax is compared against (eg, results of fitting SSM to the simulated data). I think the alternatives would be to add SSM as a CI dependency, which could result in incompatibilities with other dependencies in the future, or to drop the checks against SSM entirely, which I feel are useful for testing the correctness of this PR but may not be needed later.
Important security implication: If we keep the .npz, we must keep allow_pickle=False (as it currently is) in np.load to avoid possible arbitrary code execution in CI.
Make each commit independently reviewable
I recognize that this is a large PR. To aid review, I have tried to make each commit one self-contained, independently reviewable change. If it is still too unwieldy, I’m happy to break it into smaller, more focused PRs.
What is different from SSM?
Leave RBPF fixes out of this PR
On main, there is an implementation of Rao-Blackwellized Particle Filtering (RBPF). This code appears to contain some bugs. I’ve kept the code (moved to inference_rbpf.py), but since this PR does not use that code at all, I haven’t attempted to fix the bugs here. If we want to support RBPF going forward, I can open an issue on this.