Pólya-Gamma Model Backend Profiling: numpy vs JAX (sparse)¶
This notebook profiles the reduced-form Pólya–Gamma spatial models, SARLogit and SARNegBin, across the two Gibbs execution backends:
numpy— sparse factorizations ofI − ρWthrough scipy and scikit-sparse, NumPy Pólya–Gamma draws, and a NumPy log-determinant.jax— the sweep compiled once into an XLA program, with sparse LU solves ofI − ρWthrough sparsax (Wis never densified), Pólya–Gamma draws from augax, and the log-determinant passed in as data.
The two W regimes (this is the important axis)¶
The backend split that matters is the log-determinant log|I − ρW|, and it
hinges on whether W is D-symmetrizable, not on whether the matrix is
literally symmetric:
Symmetric adjacency → row-standardized (rook/queen contiguity):
W = D⁻¹AwithAsymmetric, soWis D-symmetrizable even though the row-standardized matrix is not literally symmetric. → exact Cholesky logdet (logdet_decomp="chol").Asymmetric adjacency → row-standardized (directed KNN, travel time, migration): the underlying adjacency is not symmetric, so
Wis not D-symmetrizable and a Cholesky logdet does not exist. → exact LU logdet (aaa, KLU-seeded).
Both logdet methods are exact (machine precision); they differ only in the
factorization the matrix admits. max|W − Wᵀ| is not the discriminator — a
row-standardized rook W and a row-standardized KNN W can have identical raw
asymmetry while landing on different logdet paths. We therefore key on the
auto-resolved logdet method reported by the model.
Question¶
Does jax beat numpy on CPU, and at what n? backend="auto" resolves to
jax for both models; this benchmark is the evidence for that default.
import time
import warnings
from dataclasses import dataclass
import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from libpysal import graph
from neighbayes import dgp
from neighbayes.models import SARLogit, SARNegBin
warnings.filterwarnings("ignore")
/home/runner/micromamba/envs/test/lib/python3.14/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
def build_w(gdf, kind: str, k: int = 6):
"""Row-standardized W. ``kind="sym"`` = rook contiguity (D-symmetrizable →
Cholesky logdet); ``kind="asym"`` = directed KNN on centroids
(non-D-symmetrizable → LU/aaa logdet)."""
if kind == "sym":
return graph.Graph.build_contiguity(gdf, rook=True).transform("r")
pts = gdf.copy()
pts["geometry"] = gdf.geometry.centroid
return graph.Graph.build_knn(pts, k=k).transform("r")
def make_dataset(model: str, n_side: int, kind: str, seed: int = 2026):
"""Simulate one PG dataset on an ``n_side × n_side`` grid with the chosen W."""
gdf = dgp.simulate_sar(n_side=n_side, create_gdf=True, seed=seed)
W = build_w(gdf, kind)
sim = (dgp.simulate_sar_logit if model == "logit" else dgp.simulate_sar_negbin)(
W=W, gdf=gdf, rho=0.4, seed=seed, latent=False
)
y = sim["y"].to_numpy() if hasattr(sim["y"], "to_numpy") else np.asarray(sim["y"])
X_cols = [c for c in sim.columns if c.startswith("X_")]
X = sim[X_cols].to_numpy()
return y, X, W
def fit_and_time(model: str, y, X, W, backend: str, draws, tune, chains, seed=2026):
"""Fit one PG model on one backend; return timing + ESS + resolved logdet method."""
Model = SARLogit if model == "logit" else SARNegBin
m = Model(y=y, X=X, W=W, latent=False) # the reduced form's PG sampler
t0 = time.perf_counter()
idata = m.fit(
draws=draws,
tune=tune,
chains=chains,
random_seed=seed,
progressbar=False,
backend=backend,
sampler="gibbs",
)
elapsed = time.perf_counter() - t0
ess = float(az.summary(idata, var_names=["rho"]).loc["rho", "ess_bulk"])
rho_hat = float(idata.posterior["rho"].mean())
return {
"total_s": elapsed,
"ess_rho": ess,
"ess_per_s": ess / elapsed,
"rho_hat": rho_hat,
"logdet_method": m._resolved_logdet_method,
}
@dataclass
class Config:
# grid sides -> obs = side²: 20→400, 30→900, 45→2025, 71→5041 (≥5000)
sides: tuple = (20, 30, 45, 71)
models: tuple = ("logit", "nb")
w_kinds: tuple = ("sym", "asym")
backends: tuple = ("numpy", "jax")
draws: int = 400
tune: int = 400
chains: int = 2
seed: int = 2026
cfg = Config()
rows = []
for side in cfg.sides:
n = side * side
for model in cfg.models:
for kind in cfg.w_kinds:
y, X, W = make_dataset(model, side, kind, seed=cfg.seed)
for backend in cfg.backends:
print(
f"n={n:5d} | {model:5s} | W={kind:4s} | backend={backend} ...",
flush=True,
)
try:
r = fit_and_time(
model,
y,
X,
W,
backend,
cfg.draws,
cfg.tune,
cfg.chains,
cfg.seed,
)
r.update(n_obs=n, model=model, w_kind=kind, backend=backend)
rows.append(r)
except Exception as exc: # keep the sweep going
print(f" FAILED: {type(exc).__name__}: {exc}")
res = pd.DataFrame(rows)
res
n= 400 | logit | W=sym | backend=numpy ...
n= 400 | logit | W=sym | backend=jax ...
n= 400 | logit | W=asym | backend=numpy ...
n= 400 | logit | W=asym | backend=jax ...
n= 400 | nb | W=sym | backend=numpy ...
n= 400 | nb | W=sym | backend=jax ...
n= 400 | nb | W=asym | backend=numpy ...
n= 400 | nb | W=asym | backend=jax ...
n= 900 | logit | W=sym | backend=numpy ...
n= 900 | logit | W=sym | backend=jax ...
n= 900 | logit | W=asym | backend=numpy ...
n= 900 | logit | W=asym | backend=jax ...
n= 900 | nb | W=sym | backend=numpy ...
n= 900 | nb | W=sym | backend=jax ...
n= 900 | nb | W=asym | backend=numpy ...
n= 900 | nb | W=asym | backend=jax ...
n= 2025 | logit | W=sym | backend=numpy ...
n= 2025 | logit | W=sym | backend=jax ...
n= 2025 | logit | W=asym | backend=numpy ...
n= 2025 | logit | W=asym | backend=jax ...
n= 2025 | nb | W=sym | backend=numpy ...
n= 2025 | nb | W=sym | backend=jax ...
n= 2025 | nb | W=asym | backend=numpy ...
n= 2025 | nb | W=asym | backend=jax ...
n= 5041 | logit | W=sym | backend=numpy ...
n= 5041 | logit | W=sym | backend=jax ...
n= 5041 | logit | W=asym | backend=numpy ...
n= 5041 | logit | W=asym | backend=jax ...
n= 5041 | nb | W=sym | backend=numpy ...
n= 5041 | nb | W=sym | backend=jax ...
n= 5041 | nb | W=asym | backend=numpy ...
n= 5041 | nb | W=asym | backend=jax ...
| total_s | ess_rho | ess_per_s | rho_hat | logdet_method | n_obs | model | w_kind | backend | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 4.842934 | 528.670291 | 109.163230 | 0.160000 | eigenvalue | 400 | logit | sym | numpy |
| 1 | 2.619110 | 726.165233 | 277.256457 | 0.154656 | eigenvalue | 400 | logit | sym | jax |
| 2 | 10.869791 | 609.198975 | 56.045143 | 0.130328 | eigenvalue | 400 | logit | asym | numpy |
| 3 | 2.514596 | 545.293228 | 216.851227 | 0.128933 | eigenvalue | 400 | logit | asym | jax |
| 4 | 6.003300 | 440.322449 | 73.346740 | 0.309341 | eigenvalue | 400 | nb | sym | numpy |
| 5 | 2.328481 | 527.860841 | 226.697469 | 0.307686 | eigenvalue | 400 | nb | sym | jax |
| 6 | 5.750110 | 578.274294 | 100.567527 | 0.481518 | eigenvalue | 400 | nb | asym | numpy |
| 7 | 2.379150 | 531.891301 | 223.563620 | 0.483036 | eigenvalue | 400 | nb | asym | jax |
| 8 | 5.489050 | 624.807655 | 113.828008 | 0.488573 | chol_aaa | 900 | logit | sym | numpy |
| 9 | 2.601276 | 520.756814 | 200.192854 | 0.489691 | chol_aaa | 900 | logit | sym | jax |
| 10 | 8.664342 | 446.927669 | 51.582415 | 0.462511 | lu_aaa | 900 | logit | asym | numpy |
| 11 | 2.622217 | 636.960301 | 242.909069 | 0.456870 | lu_aaa | 900 | logit | asym | jax |
| 12 | 6.190954 | 573.920806 | 92.703126 | 0.320246 | chol_aaa | 900 | nb | sym | numpy |
| 13 | 2.831745 | 638.243666 | 225.388840 | 0.320430 | chol_aaa | 900 | nb | sym | jax |
| 14 | 6.892935 | 568.991549 | 82.547064 | 0.371075 | lu_aaa | 900 | nb | asym | numpy |
| 15 | 2.970273 | 444.393619 | 149.613706 | 0.376566 | lu_aaa | 900 | nb | asym | jax |
| 16 | 6.181957 | 490.844189 | 79.399485 | 0.300904 | chol_aaa | 2025 | logit | sym | numpy |
| 17 | 2.920237 | 607.298079 | 207.961881 | 0.300721 | chol_aaa | 2025 | logit | sym | jax |
| 18 | 8.215058 | 583.466330 | 71.024005 | 0.341244 | lu_aaa | 2025 | logit | asym | numpy |
| 19 | 3.227348 | 572.931671 | 177.524010 | 0.342960 | lu_aaa | 2025 | logit | asym | jax |
| 20 | 7.294384 | 424.479580 | 58.192660 | 0.370420 | chol_aaa | 2025 | nb | sym | numpy |
| 21 | 3.736432 | 504.498578 | 135.021472 | 0.368809 | chol_aaa | 2025 | nb | sym | jax |
| 22 | 7.518789 | 494.936441 | 65.826615 | 0.404559 | lu_aaa | 2025 | nb | asym | numpy |
| 23 | 3.778756 | 550.430753 | 145.664552 | 0.405283 | lu_aaa | 2025 | nb | asym | jax |
| 24 | 8.304927 | 472.874395 | 56.939020 | 0.335606 | chol_aaa | 5041 | logit | sym | numpy |
| 25 | 3.692545 | 609.415387 | 165.039407 | 0.336724 | chol_aaa | 5041 | logit | sym | jax |
| 26 | 8.635595 | 682.187156 | 78.997126 | 0.304462 | lu_aaa | 5041 | logit | asym | numpy |
| 27 | 4.874219 | 496.245171 | 101.810185 | 0.300102 | lu_aaa | 5041 | logit | asym | jax |
| 28 | 10.847178 | 523.579003 | 48.268683 | 0.387372 | chol_aaa | 5041 | nb | sym | numpy |
| 29 | 4.820179 | 523.105344 | 108.524060 | 0.385733 | chol_aaa | 5041 | nb | sym | jax |
| 30 | 10.078985 | 395.112171 | 39.201584 | 0.393597 | lu_aaa | 5041 | nb | asym | numpy |
| 31 | 4.833566 | 446.739827 | 92.424490 | 0.392613 | lu_aaa | 5041 | nb | asym | jax |
# Confirm the two W regimes land on the two exact logdet methods.
if not res.empty:
method_map = (
res.groupby(["w_kind"])["logdet_method"].unique().apply(sorted).to_dict()
)
print("Resolved logdet method by W regime:")
for kind, methods in method_map.items():
print(f" W={kind:4s} -> {methods}")
display(
res.pivot_table(
index=["model", "n_obs"], columns=["w_kind", "backend"], values="total_s"
).round(2)
)
Resolved logdet method by W regime:
W=asym -> ['eigenvalue', 'lu_aaa']
W=sym -> ['chol_aaa', 'eigenvalue']
| w_kind | asym | sym | |||
|---|---|---|---|---|---|
| backend | jax | numpy | jax | numpy | |
| model | n_obs | ||||
| logit | 400 | 2.51 | 10.87 | 2.62 | 4.84 |
| 900 | 2.62 | 8.66 | 2.60 | 5.49 | |
| 2025 | 3.23 | 8.22 | 2.92 | 6.18 | |
| 5041 | 4.87 | 8.64 | 3.69 | 8.30 | |
| nb | 400 | 2.38 | 5.75 | 2.33 | 6.00 |
| 900 | 2.97 | 6.89 | 2.83 | 6.19 | |
| 2025 | 3.78 | 7.52 | 3.74 | 7.29 | |
| 5041 | 4.83 | 10.08 | 4.82 | 10.85 | |
if not res.empty:
fig, axes = plt.subplots(2, 2, figsize=(13, 9), constrained_layout=True)
styles = {"numpy": dict(marker="o", ls="-"), "jax": dict(marker="s", ls="--")}
colors = {"sym": "#1f77b4", "asym": "#d62728"}
for j, model in enumerate(("logit", "nb")):
for backend in cfg.backends:
for kind in cfg.w_kinds:
sub = res[
(res.model == model)
& (res.backend == backend)
& (res.w_kind == kind)
].sort_values("n_obs")
if sub.empty:
continue
lbl = f"{backend}·{kind}"
axes[0, j].plot(
sub.n_obs,
sub.total_s,
color=colors[kind],
alpha=0.9 if backend == "jax" else 0.5,
label=lbl,
**styles[backend],
)
axes[1, j].plot(
sub.n_obs,
sub.ess_per_s,
color=colors[kind],
alpha=0.9 if backend == "jax" else 0.5,
label=lbl,
**styles[backend],
)
axes[0, j].set(
title=f"{model}: total time vs n",
xlabel="observations",
ylabel="total time (s)",
yscale="log",
)
axes[1, j].set(
title=f"{model}: ESS(rho)/sec vs n",
xlabel="observations",
ylabel="ESS per second",
yscale="log",
)
for ax in (axes[0, j], axes[1, j]):
ax.grid(True, alpha=0.3)
ax.legend(fontsize=8)
plt.show()
# Correctness: numpy and jax must agree (within MC noise) on the posterior.
# Deterministic side-check of the JAX exact logdet is in logdet_profiling.ipynb;
# here we confirm the end-to-end samplers agree on rho.
if not res.empty:
agree = res.pivot_table(
index=["model", "w_kind", "n_obs"], columns="backend", values="rho_hat"
).dropna()
agree["abs_diff"] = (agree["numpy"] - agree["jax"]).abs()
print("Posterior mean rho: numpy vs jax (should agree within MC noise)")
display(agree.round(3))
Posterior mean rho: numpy vs jax (should agree within MC noise)
| backend | jax | numpy | abs_diff | ||
|---|---|---|---|---|---|
| model | w_kind | n_obs | |||
| logit | asym | 400 | 0.129 | 0.130 | 0.001 |
| 900 | 0.457 | 0.463 | 0.006 | ||
| 2025 | 0.343 | 0.341 | 0.002 | ||
| 5041 | 0.300 | 0.304 | 0.004 | ||
| sym | 400 | 0.155 | 0.160 | 0.005 | |
| 900 | 0.490 | 0.489 | 0.001 | ||
| 2025 | 0.301 | 0.301 | 0.000 | ||
| 5041 | 0.337 | 0.336 | 0.001 | ||
| nb | asym | 400 | 0.483 | 0.482 | 0.002 |
| 900 | 0.377 | 0.371 | 0.005 | ||
| 2025 | 0.405 | 0.405 | 0.001 | ||
| 5041 | 0.393 | 0.394 | 0.001 | ||
| sym | 400 | 0.308 | 0.309 | 0.002 | |
| 900 | 0.320 | 0.320 | 0.000 | ||
| 2025 | 0.369 | 0.370 | 0.002 | ||
| 5041 | 0.386 | 0.387 | 0.002 |
Reading the results¶
Two exact logdet paths, auto-selected by W regime. The
sym(rook) datasets resolve to the Cholesky logdet; theasym(directed KNN) datasets resolve to the LU-basedaaalogdet. Both are exact — the split is which factorizationWadmits, driven by D-symmetrizability, not raw matrix asymmetry.Backend crossover.
numpy/CHOLMOD historically won on CPU at small–moderaten(dispatch-free per-iteration factorization). The fully-sparsejaxpath pays a one-time JIT-compile cost (folded into total time here) but then runs a single fused kernel per sweep; its advantage grows withn. The crossover point is the practically important number.ess_per_sis the fair summary — it nets out any per-iteration cost differences and answers “which backend gives more effective draws per wall-second.”Same posterior. numpy and jax use different RNG streams, so exact agreement is not expected, but the
rhoposterior means should match within Monte Carlo noise.auto_backendpolicy. Both PG models registerauto_backend="numpy". Ifjaxwins oness_per_sabove somen, the auto-selector should preferjaxbeyond that threshold (mirroring the size-based logdet auto-selection).
Caveats. Single-run wall-clock at benchmark sizes; NUTS-free Gibbs so timings are dominated by the per-sweep linear algebra. The JAX total time includes compilation — amortized away across longer chains than the modest draws used here.