Pólya-Gamma Model Backend Profiling: numpy vs JAX (sparse)

This notebook profiles the reduced-form Pólya–Gamma spatial models, SARLogit and SARNegBin, across the two Gibbs execution backends:

  • numpy — sparse factorizations of I − ρW through scipy and scikit-sparse, NumPy Pólya–Gamma draws, and a NumPy log-determinant.

  • jax — the sweep compiled once into an XLA program, with sparse LU solves of I − ρW through sparsax (W is never densified), Pólya–Gamma draws from augax, and the log-determinant passed in as data.

The two W regimes (this is the important axis)

The backend split that matters is the log-determinant log|I − ρW|, and it hinges on whether W is D-symmetrizable, not on whether the matrix is literally symmetric:

  • Symmetric adjacency → row-standardized (rook/queen contiguity): W = D⁻¹A with A symmetric, so W is D-symmetrizable even though the row-standardized matrix is not literally symmetric. → exact Cholesky logdet (logdet_decomp="chol").

  • Asymmetric adjacency → row-standardized (directed KNN, travel time, migration): the underlying adjacency is not symmetric, so W is not D-symmetrizable and a Cholesky logdet does not exist. → exact LU logdet (aaa, KLU-seeded).

Both logdet methods are exact (machine precision); they differ only in the factorization the matrix admits. max|W − Wᵀ| is not the discriminator — a row-standardized rook W and a row-standardized KNN W can have identical raw asymmetry while landing on different logdet paths. We therefore key on the auto-resolved logdet method reported by the model.

Question

Does jax beat numpy on CPU, and at what n? backend="auto" resolves to jax for both models; this benchmark is the evidence for that default.

import time
import warnings
from dataclasses import dataclass

import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from libpysal import graph

from neighbayes import dgp
from neighbayes.models import SARLogit, SARNegBin

warnings.filterwarnings("ignore")
/home/runner/micromamba/envs/test/lib/python3.14/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
def build_w(gdf, kind: str, k: int = 6):
    """Row-standardized W. ``kind="sym"`` = rook contiguity (D-symmetrizable →
    Cholesky logdet); ``kind="asym"`` = directed KNN on centroids
    (non-D-symmetrizable → LU/aaa logdet)."""
    if kind == "sym":
        return graph.Graph.build_contiguity(gdf, rook=True).transform("r")
    pts = gdf.copy()
    pts["geometry"] = gdf.geometry.centroid
    return graph.Graph.build_knn(pts, k=k).transform("r")


def make_dataset(model: str, n_side: int, kind: str, seed: int = 2026):
    """Simulate one PG dataset on an ``n_side × n_side`` grid with the chosen W."""
    gdf = dgp.simulate_sar(n_side=n_side, create_gdf=True, seed=seed)
    W = build_w(gdf, kind)
    sim = (dgp.simulate_sar_logit if model == "logit" else dgp.simulate_sar_negbin)(
        W=W, gdf=gdf, rho=0.4, seed=seed, latent=False
    )
    y = sim["y"].to_numpy() if hasattr(sim["y"], "to_numpy") else np.asarray(sim["y"])
    X_cols = [c for c in sim.columns if c.startswith("X_")]
    X = sim[X_cols].to_numpy()
    return y, X, W


def fit_and_time(model: str, y, X, W, backend: str, draws, tune, chains, seed=2026):
    """Fit one PG model on one backend; return timing + ESS + resolved logdet method."""
    Model = SARLogit if model == "logit" else SARNegBin
    m = Model(y=y, X=X, W=W, latent=False)  # the reduced form's PG sampler
    t0 = time.perf_counter()
    idata = m.fit(
        draws=draws,
        tune=tune,
        chains=chains,
        random_seed=seed,
        progressbar=False,
        backend=backend,
        sampler="gibbs",
    )
    elapsed = time.perf_counter() - t0
    ess = float(az.summary(idata, var_names=["rho"]).loc["rho", "ess_bulk"])
    rho_hat = float(idata.posterior["rho"].mean())
    return {
        "total_s": elapsed,
        "ess_rho": ess,
        "ess_per_s": ess / elapsed,
        "rho_hat": rho_hat,
        "logdet_method": m._resolved_logdet_method,
    }
@dataclass
class Config:
    # grid sides -> obs = side²: 20→400, 30→900, 45→2025, 71→5041 (≥5000)
    sides: tuple = (20, 30, 45, 71)
    models: tuple = ("logit", "nb")
    w_kinds: tuple = ("sym", "asym")
    backends: tuple = ("numpy", "jax")
    draws: int = 400
    tune: int = 400
    chains: int = 2
    seed: int = 2026


cfg = Config()
rows = []
for side in cfg.sides:
    n = side * side
    for model in cfg.models:
        for kind in cfg.w_kinds:
            y, X, W = make_dataset(model, side, kind, seed=cfg.seed)
            for backend in cfg.backends:
                print(
                    f"n={n:5d} | {model:5s} | W={kind:4s} | backend={backend} ...",
                    flush=True,
                )
                try:
                    r = fit_and_time(
                        model,
                        y,
                        X,
                        W,
                        backend,
                        cfg.draws,
                        cfg.tune,
                        cfg.chains,
                        cfg.seed,
                    )
                    r.update(n_obs=n, model=model, w_kind=kind, backend=backend)
                    rows.append(r)
                except Exception as exc:  # keep the sweep going
                    print(f"    FAILED: {type(exc).__name__}: {exc}")

res = pd.DataFrame(rows)
res
n=  400 | logit | W=sym  | backend=numpy ...
n=  400 | logit | W=sym  | backend=jax ...
n=  400 | logit | W=asym | backend=numpy ...
n=  400 | logit | W=asym | backend=jax ...
n=  400 | nb    | W=sym  | backend=numpy ...
n=  400 | nb    | W=sym  | backend=jax ...
n=  400 | nb    | W=asym | backend=numpy ...
n=  400 | nb    | W=asym | backend=jax ...
n=  900 | logit | W=sym  | backend=numpy ...
n=  900 | logit | W=sym  | backend=jax ...
n=  900 | logit | W=asym | backend=numpy ...
n=  900 | logit | W=asym | backend=jax ...
n=  900 | nb    | W=sym  | backend=numpy ...
n=  900 | nb    | W=sym  | backend=jax ...
n=  900 | nb    | W=asym | backend=numpy ...
n=  900 | nb    | W=asym | backend=jax ...
n= 2025 | logit | W=sym  | backend=numpy ...
n= 2025 | logit | W=sym  | backend=jax ...
n= 2025 | logit | W=asym | backend=numpy ...
n= 2025 | logit | W=asym | backend=jax ...
n= 2025 | nb    | W=sym  | backend=numpy ...
n= 2025 | nb    | W=sym  | backend=jax ...
n= 2025 | nb    | W=asym | backend=numpy ...
n= 2025 | nb    | W=asym | backend=jax ...
n= 5041 | logit | W=sym  | backend=numpy ...
n= 5041 | logit | W=sym  | backend=jax ...
n= 5041 | logit | W=asym | backend=numpy ...
n= 5041 | logit | W=asym | backend=jax ...
n= 5041 | nb    | W=sym  | backend=numpy ...
n= 5041 | nb    | W=sym  | backend=jax ...
n= 5041 | nb    | W=asym | backend=numpy ...
n= 5041 | nb    | W=asym | backend=jax ...
total_s ess_rho ess_per_s rho_hat logdet_method n_obs model w_kind backend
0 4.842934 528.670291 109.163230 0.160000 eigenvalue 400 logit sym numpy
1 2.619110 726.165233 277.256457 0.154656 eigenvalue 400 logit sym jax
2 10.869791 609.198975 56.045143 0.130328 eigenvalue 400 logit asym numpy
3 2.514596 545.293228 216.851227 0.128933 eigenvalue 400 logit asym jax
4 6.003300 440.322449 73.346740 0.309341 eigenvalue 400 nb sym numpy
5 2.328481 527.860841 226.697469 0.307686 eigenvalue 400 nb sym jax
6 5.750110 578.274294 100.567527 0.481518 eigenvalue 400 nb asym numpy
7 2.379150 531.891301 223.563620 0.483036 eigenvalue 400 nb asym jax
8 5.489050 624.807655 113.828008 0.488573 chol_aaa 900 logit sym numpy
9 2.601276 520.756814 200.192854 0.489691 chol_aaa 900 logit sym jax
10 8.664342 446.927669 51.582415 0.462511 lu_aaa 900 logit asym numpy
11 2.622217 636.960301 242.909069 0.456870 lu_aaa 900 logit asym jax
12 6.190954 573.920806 92.703126 0.320246 chol_aaa 900 nb sym numpy
13 2.831745 638.243666 225.388840 0.320430 chol_aaa 900 nb sym jax
14 6.892935 568.991549 82.547064 0.371075 lu_aaa 900 nb asym numpy
15 2.970273 444.393619 149.613706 0.376566 lu_aaa 900 nb asym jax
16 6.181957 490.844189 79.399485 0.300904 chol_aaa 2025 logit sym numpy
17 2.920237 607.298079 207.961881 0.300721 chol_aaa 2025 logit sym jax
18 8.215058 583.466330 71.024005 0.341244 lu_aaa 2025 logit asym numpy
19 3.227348 572.931671 177.524010 0.342960 lu_aaa 2025 logit asym jax
20 7.294384 424.479580 58.192660 0.370420 chol_aaa 2025 nb sym numpy
21 3.736432 504.498578 135.021472 0.368809 chol_aaa 2025 nb sym jax
22 7.518789 494.936441 65.826615 0.404559 lu_aaa 2025 nb asym numpy
23 3.778756 550.430753 145.664552 0.405283 lu_aaa 2025 nb asym jax
24 8.304927 472.874395 56.939020 0.335606 chol_aaa 5041 logit sym numpy
25 3.692545 609.415387 165.039407 0.336724 chol_aaa 5041 logit sym jax
26 8.635595 682.187156 78.997126 0.304462 lu_aaa 5041 logit asym numpy
27 4.874219 496.245171 101.810185 0.300102 lu_aaa 5041 logit asym jax
28 10.847178 523.579003 48.268683 0.387372 chol_aaa 5041 nb sym numpy
29 4.820179 523.105344 108.524060 0.385733 chol_aaa 5041 nb sym jax
30 10.078985 395.112171 39.201584 0.393597 lu_aaa 5041 nb asym numpy
31 4.833566 446.739827 92.424490 0.392613 lu_aaa 5041 nb asym jax
# Confirm the two W regimes land on the two exact logdet methods.
if not res.empty:
    method_map = (
        res.groupby(["w_kind"])["logdet_method"].unique().apply(sorted).to_dict()
    )
    print("Resolved logdet method by W regime:")
    for kind, methods in method_map.items():
        print(f"  W={kind:4s} -> {methods}")
    display(
        res.pivot_table(
            index=["model", "n_obs"], columns=["w_kind", "backend"], values="total_s"
        ).round(2)
    )
Resolved logdet method by W regime:
  W=asym -> ['eigenvalue', 'lu_aaa']
  W=sym  -> ['chol_aaa', 'eigenvalue']
w_kind asym sym
backend jax numpy jax numpy
model n_obs
logit 400 2.51 10.87 2.62 4.84
900 2.62 8.66 2.60 5.49
2025 3.23 8.22 2.92 6.18
5041 4.87 8.64 3.69 8.30
nb 400 2.38 5.75 2.33 6.00
900 2.97 6.89 2.83 6.19
2025 3.78 7.52 3.74 7.29
5041 4.83 10.08 4.82 10.85
if not res.empty:
    fig, axes = plt.subplots(2, 2, figsize=(13, 9), constrained_layout=True)
    styles = {"numpy": dict(marker="o", ls="-"), "jax": dict(marker="s", ls="--")}
    colors = {"sym": "#1f77b4", "asym": "#d62728"}
    for j, model in enumerate(("logit", "nb")):
        for backend in cfg.backends:
            for kind in cfg.w_kinds:
                sub = res[
                    (res.model == model)
                    & (res.backend == backend)
                    & (res.w_kind == kind)
                ].sort_values("n_obs")
                if sub.empty:
                    continue
                lbl = f"{backend}·{kind}"
                axes[0, j].plot(
                    sub.n_obs,
                    sub.total_s,
                    color=colors[kind],
                    alpha=0.9 if backend == "jax" else 0.5,
                    label=lbl,
                    **styles[backend],
                )
                axes[1, j].plot(
                    sub.n_obs,
                    sub.ess_per_s,
                    color=colors[kind],
                    alpha=0.9 if backend == "jax" else 0.5,
                    label=lbl,
                    **styles[backend],
                )
        axes[0, j].set(
            title=f"{model}: total time vs n",
            xlabel="observations",
            ylabel="total time (s)",
            yscale="log",
        )
        axes[1, j].set(
            title=f"{model}: ESS(rho)/sec vs n",
            xlabel="observations",
            ylabel="ESS per second",
            yscale="log",
        )
        for ax in (axes[0, j], axes[1, j]):
            ax.grid(True, alpha=0.3)
            ax.legend(fontsize=8)
    plt.show()
../_images/41649433091126755a576fd75aa7b7ebe4772c8583de0e386f8967eb2cf91bb9.png
# Correctness: numpy and jax must agree (within MC noise) on the posterior.
# Deterministic side-check of the JAX exact logdet is in logdet_profiling.ipynb;
# here we confirm the end-to-end samplers agree on rho.
if not res.empty:
    agree = res.pivot_table(
        index=["model", "w_kind", "n_obs"], columns="backend", values="rho_hat"
    ).dropna()
    agree["abs_diff"] = (agree["numpy"] - agree["jax"]).abs()
    print("Posterior mean rho: numpy vs jax (should agree within MC noise)")
    display(agree.round(3))
Posterior mean rho: numpy vs jax (should agree within MC noise)
backend jax numpy abs_diff
model w_kind n_obs
logit asym 400 0.129 0.130 0.001
900 0.457 0.463 0.006
2025 0.343 0.341 0.002
5041 0.300 0.304 0.004
sym 400 0.155 0.160 0.005
900 0.490 0.489 0.001
2025 0.301 0.301 0.000
5041 0.337 0.336 0.001
nb asym 400 0.483 0.482 0.002
900 0.377 0.371 0.005
2025 0.405 0.405 0.001
5041 0.393 0.394 0.001
sym 400 0.308 0.309 0.002
900 0.320 0.320 0.000
2025 0.369 0.370 0.002
5041 0.386 0.387 0.002

Reading the results

  • Two exact logdet paths, auto-selected by W regime. The sym (rook) datasets resolve to the Cholesky logdet; the asym (directed KNN) datasets resolve to the LU-based aaa logdet. Both are exact — the split is which factorization W admits, driven by D-symmetrizability, not raw matrix asymmetry.

  • Backend crossover. numpy/CHOLMOD historically won on CPU at small–moderate n (dispatch-free per-iteration factorization). The fully-sparse jax path pays a one-time JIT-compile cost (folded into total time here) but then runs a single fused kernel per sweep; its advantage grows with n. The crossover point is the practically important number.

  • ess_per_s is the fair summary — it nets out any per-iteration cost differences and answers “which backend gives more effective draws per wall-second.”

  • Same posterior. numpy and jax use different RNG streams, so exact agreement is not expected, but the rho posterior means should match within Monte Carlo noise.

  • auto_backend policy. Both PG models register auto_backend="numpy". If jax wins on ess_per_s above some n, the auto-selector should prefer jax beyond that threshold (mirroring the size-based logdet auto-selection).

Caveats. Single-run wall-clock at benchmark sizes; NUTS-free Gibbs so timings are dominated by the per-sweep linear algebra. The JAX total time includes compilation — amortized away across longer chains than the modest draws used here.