Skip to content
tryhard-26Public

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

49 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Spatio-Temporal Traffic Flow Prediction on Highway Networks

A Deep Learning comparative study benchmarking Dilated 1D-CNN, Temporal Transformer, Spatio-Temporal Graph Neural Networks (STGCN / ChebNet), and Multi-Layer LSTM on the METR-LA traffic speed benchmark.


1. Team & Architecture Matrix

Real Name Registration No. GitHub Profile Architecture Paradigm Trainable Params Status
Adriteyo Das 230953244 @tryhard-26 Temporal Transformer (Multi-Head Self-Attention + Node Embeddings) 246,701 Completed
Varanasi Naga Akhil 230953496 @AkCodes23 Spatio-Temporal GNN (Chebyshev Spectral Graph Conv $K=3$ + Temporal GLU) 304,941 Completed
Jayesh Agarwal 230953348 @itjayesh Dilated 1D-CNN (Stacked Causal Dilated Convolutions $d \in {1,2,4,8}$) 150,061 Completed
Mihika Bardhan 230953122 @mihikabardhan Multi-Layer LSTM (Recurrent Sequence Modeling + LayerNorm) 213,388 Completed

2. Dataset Specifications

  • Benchmark Dataset: METR-LA (Los Angeles County Highway Network via Caltrans PeMS)
  • Sensors (Nodes): 207 loop detector stations
  • Timesteps: 34,272 time steps sampled at 5-minute intervals (March 1, 2012 – June 27, 2012)
  • Total Observations: 7,094,304 speed readings (mph)
  • Graph Topology: $207 \times 207$ weighted directed adjacency matrix with 1,722 spatial edges (4.02% density, 95.98% sparsity)
  • Data Partitions: 70% Train (23,978 samples), 10% Validation (3,408 samples), 20% Test (6,835 samples) (chronological split)
  • Forecasting Task: Input past 12 steps (1 hour) $\to$ Predict next 12 steps (15, 30, and 60 minutes ahead)

Exploratory Data Analysis (EDA) Highlights

Temporal Dynamics (Rush Hour Dips) Spatial Sensor Adjacency Network
Temporal Patterns Spatial Network

3. Comprehensive Benchmark Results (All 4 Architectures)

All 4 models were trained on Apple Silicon Metal Performance Shaders (mps) across 25 epochs using Masked MAE loss and Adam optimizer with ReduceLROnPlateau scheduling.

Master Evaluation Table (Unseen Test Partition)

Model Architecture Member Horizon Masked MAE (mph) Masked RMSE (mph) Masked MAPE (%) $R^2$ Score
Temporal Transformer Adriteyo Das 15 min (H3) 2.79 5.38 7.35% 0.8447
30 min (H6) 3.17 6.40 8.85% 0.7802
60 min (H12) 3.55 7.36 10.45% 0.7090
Overall Average 3.12 6.31 8.69% 0.7865
Spatio-Temporal GNN Varanasi Naga Akhil 15 min (H3) 2.86 5.50 7.65% 0.8377
(Dual-Graph STGCN) 30 min (H6) 3.27 6.54 9.25% 0.7706
60 min (H12) 3.72 7.53 11.01% 0.6955
Overall Average 3.22 6.45 9.07% 0.7771
Dilated 1D-CNN Jayesh Agarwal 15 min (H3) 2.88 5.57 7.65% 0.8339
(Adaptive Graph WaveNet) 30 min (H6) 3.31 6.61 9.34% 0.7656
60 min (H12) 3.79 7.66 11.18% 0.6849
Overall Average 3.26 6.52 9.14% 0.7718
Multi-Layer LSTM Mihika Bardhan 15 min (H3) 2.86 5.65 7.61% 0.8287
(Recurrent Sequence Modeling) 30 min (H6) 3.30 6.78 9.38% 0.7533
60 min (H12) 3.82 8.00 11.44% 0.6564
Overall Average 3.26 6.73 9.22% 0.7573

4. Literature Comparison on METR-LA

Category Model Architecture 15-min MAE (mph) 30-min MAE (mph) 60-min MAE (mph) Spatial Modeling Mechanism
Classical Baselines Historical Average (HA) 4.16 4.16 4.16 None
ARIMA 3.99 5.15 6.90 None (Univariate)
Vector Auto-Regression (VAR) 4.42 5.41 6.52 Linear cross-sensor
Feed-Forward Net (FNN) 3.99 4.23 4.49 Fully connected
Recurrent Baseline FC-LSTM (Seq2Seq) 3.44 3.77 4.37 Recurrent state memory
Our Architectures Temporal Transformer (Ours) 2.79 3.17 3.55 Spatial-Temporal Dual Attention + Residual Velocity
Spatio-Temporal GNN (Ours) 2.86 3.27 3.72 Dual-Graph STGCN (ChebNet $K=3$ + Dynamic Adaptive Graph)
Dilated 1D-CNN (Ours) 2.88 3.31 3.79 Graph WaveNet (Adaptive Graph Diffusion + Dilated Causal Conv)
Multi-Layer LSTM (Ours) 2.86 3.30 3.82 Multi-Layer LSTM Sequence Modeling + LayerNorm
Published SOTA GNNs DCRNN (ICLR 2018) 2.77 3.15 3.60 Diffusion Graph Conv + GRU
Graph WaveNet (IJCAI 2019) 2.69 3.07 3.53 Adaptive Graph Conv + Dilated Conv
STGCN (IJCAI 2018) 2.64 3.01 3.47 Spectral Graph Conv + Gated Conv
GMAN (AAAI 2020) 2.77 3.07 3.40 Spatial-Temporal Self-Attention

5. Visualizations Gallery

A. Multi-Model Benchmark Comparison (MAE & RMSE across Horizons)

Model Comparison Benchmark

B. Convergence Curves Across Model Architectures

Dilated 1D-CNN (Jayesh) Temporal Transformer (Adriteyo)
CNN Loss Transformer Loss
Multi-Layer LSTM (Mihika) Spatio-Temporal GNN (Naga Akhil)
LSTM Loss GNN Loss

C. Actual vs. Predicted Traffic Speeds (24-Hour Test Window)

Dilated 1D-CNN Predictions Temporal Transformer Predictions
CNN Predictions Transformer Predictions
Multi-Layer LSTM Predictions Spatio-Temporal GNN Predictions
LSTM Predictions GNN Predictions

D. Failure Case & Diurnal Error Breakdown (Temporal Transformer)

Failure Case Analysis

  • Diurnal Regime Dynamics: Morning Peak (07:00–09:00, MAE 3.64 mph), Evening Rush (16:30–18:30, MAE 4.51 mph), and Night Free-Flow (22:00–05:00, MAE 2.41 mph).
  • Congestion Shockwave Failure: Under abrupt deceleration ($\Delta = 66.9\text{ mph}$, speed collapse from $69.4 \to 2.4\text{ mph}$), self-attention models display an intrinsic 15–20 minute phase lag due to reliance on historical autoregressive velocity windows.
  • Spatial Bottleneck Variance: Complex highway interchange merge sensors exhibit up to 2.4× higher prediction error compared to mainline straight-segment sensors.

E. Architectural Ablation Study (Component Contributions Across All 4 Paradigms)

Architectural Ablation Study

Master Architectural Ablation Matrix

Model Family Member Ablation Variant 15m MAE 30m MAE 60m MAE Overall MAE $\Delta$ MAE Degradation
Temporal Transformer Adriteyo Das Full Model (Base) 2.79 3.17 3.55 3.12 0.00 Baseline
w/o Residual Skip Head 3.84 4.55 5.56 4.54 +1.42 +45.5%
w/o Spatial Diffusion 3.97 4.96 6.37 4.95 +1.83 +58.8%
w/o Positional Encoding 3.96 4.98 6.41 4.98 +1.86 +59.8%
Spatio-Temporal GNN Varanasi Naga Akhil ChebNet $K=3$ (Base) 2.86 3.27 3.72 3.22 0.00 Baseline
ChebNet $K=1$ (1-Hop) 3.82 4.83 6.29 4.83 +1.61 +50.0%
ChebNet $K=2$ (2-Hop) 3.91 4.84 6.29 4.87 +1.65 +51.3%
Static Graph Only (w/o Adaptive Graph) 3.90 4.94 6.50 4.94 +1.72 +53.5%
Dilated 1D-CNN Jayesh Agarwal Dilations $[1,2,4,8]$ (Base) 2.88 3.31 3.79 3.26 0.00 Baseline
No Dilation $[1,1,1,1]$ 3.68 4.61 5.84 4.58 +1.32 +40.6%
Shallow Dilation $[1,2,4]$ 3.85 4.72 5.87 4.66 +1.40 +42.9%
w/o Adaptive Graph Diffusion 3.69 4.63 6.08 4.65 +1.40 +42.8%
Multi-Layer LSTM Mihika Bardhan 3-Layer + Embeddings (Base) 2.86 3.30 3.82 3.26 0.00 Baseline
2-Layer w/o Node Embeddings 4.31 5.01 6.03 5.00 +1.75 +53.7%
1-Layer w/o Node Embeddings 4.34 5.20 6.26 5.16 +1.90 +58.3%

6. Quickstart & Execution Guide

1. Environment Setup

# Sync dependencies using uv
uv sync

2. Download Dataset

uv run python scripts/setup.py

3. Run Exploratory Data Analysis (EDA)

uv run python src/eda.py --device auto --output-dir output

4. Train Any Architecture (25 Epochs)

# 1. Dilated 1D-CNN (Jayesh Agarwal)
uv run python src/train.py --model cnn --device mps --epochs 25 --batch-size 64

# 2. Temporal Transformer (Adriteyo Das)
uv run python src/train.py --model temporal_transformer --device mps --epochs 25 --batch-size 64

# 3. Multi-Layer LSTM (Mihika Bardhan)
uv run python src/train.py --model lstm --device mps --epochs 25 --batch-size 64

# 4. Spatio-Temporal GNN (Varanasi Naga Akhil)
uv run python src/train.py --model gnn --device mps --epochs 25 --batch-size 64

5. Run In-Depth Failure Case & Error Analysis

uv run python scripts/error_analysis.py --device mps --checkpoint output/run-temporal_transformer-20260826_222415/best_model.pt

6. Run Systematic Architectural Component Ablation Study

uv run python scripts/ablation_study.py --device mps

7. Output & Log Architecture

Each training execution creates an isolated timestamped directory with weights, configurations, and publication-ready plots:

.logs/
├── run-cnn-<timestamp>.log
├── run-lstm-<timestamp>.log
├── run-temporal_transformer-<timestamp>.log
└── run-gnn-<timestamp>.log

output/
└── run-<model>-<timestamp>/                        # Run artifacts
    ├── run_command.txt                             # Command string used for reproduction
    ├── run_config.json                             # Hyperparameters & metadata
    ├── best_model.pt                               # Best model checkpoint weights
    ├── training_history.csv                        # Per-epoch loss & LR history
    ├── horizon_metrics.csv                         # Formatted test metrics table
    ├── metrics_summary.json                        # Comprehensive test metrics JSON
    ├── 01_loss_curve.png                           # Train vs Val loss curves
    ├── 02_horizon_metrics_bar.png                  # Multi-horizon comparison bar chart
    ├── 03_predictions_vs_actual.png                # Actual vs Predicted speed curves
    └── 04_sensor_error_distribution.png            # Sensor-wise error distributions

8. Repository Structure

.
├── pyproject.toml                     # UV project configuration
├── uv.lock                            # Locked dependency tree
├── .gitignore                         # Git ignore rules (data, output, .logs, temp)
├── README.md                          # Project documentation and results
├── AGENTS.md                          # Architecture guidelines and coding standards
├── assets/                            # Tracked figures for README presentation
│   ├── 00_model_comparison_benchmark.png
│   ├── 05_failure_case_analysis.png
│   ├── 06_ablation_study.png
│   ├── cnn_loss_curve.png
│   ├── cnn_predictions.png
│   ├── lstm_loss_curve.png
│   ├── lstm_predictions.png
│   ├── transformer_loss_curve.png
│   ├── transformer_predictions.png
│   ├── gnn_loss_curve.png
│   ├── gnn_predictions.png
│   ├── eda_spatial_network.png
│   └── eda_temporal_patterns.png
├── scripts/
│   ├── setup.py                       # Dataset download & verification script
│   ├── error_analysis.py              # Diurnal regimes & shockwave failure analysis
│   └── ablation_study.py              # Cross-architecture component ablation suite
├── src/
│   ├── dataset.py                     # Sliding window generator, standard scaler, DataLoaders
│   ├── metrics.py                     # Masked MAE, RMSE, MAPE, R2 metrics
│   ├── eda.py                         # Accelerated EDA analysis script
│   ├── train.py                       # Unified training, evaluation, and plotting engine
│   └── models/
│       ├── __init__.py                # Model registry
│       ├── cnn_model.py               # Dilated 1D-CNN (Jayesh Agarwal)
│       ├── temporal_transformer.py    # Temporal Transformer (Adriteyo Das)
│       ├── lstm_model.py              # Multi-Layer LSTM (Mihika Bardhan)
│       └── gnn_model.py               # Spatio-Temporal GNN (Varanasi Naga Akhil)
├── data/                              # Ignored in git (downloaded via setup.py)
├── output/                            # Ignored in git (generated run artifacts)
└── .logs/                             # Ignored in git (run log files)

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages