How to fit zero-inflated SAR negative binomial models

Count data with more zeros than a negative binomial can account for, where the excess zeros come from a separate process — units that are inactive altogether, not merely low-intensity. SARZINB fits the zero and count parts jointly, each with its own spatial structure and its own weights matrix, and decomposes the observed zeros between the two.

The unmarked SARZINB is the latent model, each part propagating its own disturbance; this guide fits the reduced form, SARZINB(latent=False), whose spatial lags act on the linear predictors only. Both are Gibbs-only. Equations and constructor arguments are in Supported Models.

Important

Check that you need zero inflation Excess zeros are often ordinary count zeros from a small mean, not a separate process — Santos Silva & Tenreyro (2011) argue exactly this for trade flows. Fit a plain negative binomial first and compare before reaching for this model.

import arviz as az
import matplotlib.pyplot as plt
import numpy as np

from neighbayes.dgp import simulate_sar_zinb
from neighbayes.models import SARZINB

az.style.use("arviz-darkgrid")
plt.rcParams["figure.figsize"] = (10, 6)
/home/runner/micromamba/envs/test/lib/python3.14/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Simulate a known process

The simulate_sar_zinb function generates data from a two-part spatial model:

  • Zero part: a reduced-form SAR logit with parameter rho_zero and covariates X_zero

  • Count part: a reduced-form SAR NB with parameter rho, covariates X, and dispersion alpha

We use a 50×50 grid (2500 observations) for a demo.

# Simulate data from the ZINB DGP
# Use n=50 (50x50 grid → 2500 observations) for reliable parameter recovery.
# Spatial parameters (rho, rho_zero) require n ≥ 900 for reliable recovery in
# cross-sectional count models. The model raises a UserWarning when n < 900.
data = simulate_sar_zinb(
    n_side=50,  # 50x50 grid → 2500 observations
    rho=0.4,  # Count SAR parameter
    rho_zero=0.3,  # Zero-part SAR parameter
    alpha=5.0,  # NB dispersion (larger = less overdispersion)
    beta=np.array([1.0, 0.6]),  # Count coefficients [intercept, slope]
    beta_zero=np.array([0.3, 1.0]),  # Zero-part coefficients [intercept, slope]
    target_pi=0.7,  # Target activation probability
    seed=42,
    latent=False,
)

y = data["y"]
X = data["X"]
X_zero = data["X_zero"]
W_graph = data["W_graph"]
W_zero_graph = data["W_zero_graph"]

print(f"Observations: {len(y)}")
print(f"Zero fraction: {(y == 0).mean():.2%}")
print(f"Mean (y>0): {y[y > 0].mean():.1f}")
print(f"Max: {y.max():.0f}")
print("\nTrue parameters:")
print(f"  ρ (count SAR):  {data['params_true']['rho']}")
print(f"  ρ (zero SAR):   {data['params_true']['rho_zero']}")
print(f"  α (dispersion): {data['params_true']['alpha']}")
print(f"  β (count coef): {data['params_true']['beta']}")
print(f"  β (zero coef):  {data['params_true']['beta_zero']}")
Observations: 2500
Zero fraction: 34.12%
Mean (y>0): 6.6
Max: 61

True parameters:
  ρ (count SAR):  0.4
  ρ (zero SAR):   0.3
  α (dispersion): 5.0
  β (count coef): [1.  0.6]
  β (zero coef):  [0.71603727 1.        ]

Fit the model

The SARZINB model class accepts:

  • y: non-negative integer counts

  • X: count covariate matrix

  • X_zero: zero-part covariate matrix (defaults to X if not provided)

  • W: spatial weights for the count part

  • W_zero: spatial weights for the zero part (defaults to W)

The fit() method runs a 9-block Gibbs sampler with profile-log-likelihood initialisation.

# Create and fit the ZINB-SAR model
model = SARZINB(
    y=y,
    X=X,
    X_zero=X_zero,
    W=W_graph,
    W_zero=W_zero_graph,
    latent=False,
)

idata = model.fit(
    draws=2000,
    tune=1000,
    chains=4,
    random_seed=123,
    n_jobs=-1,
    progressbar=True,
)
Gibbs sampling (zinb_sar): 4 chains for 1,000 tune and 2,000 draw iterations (4 x 3,000 = 12,000 draws total)
/home/runner/micromamba/envs/test/lib/python3.14/site-packages/rich/live.py:260: UserWarning: install "ipywidgets" 
for Jupyter support
  warnings.warn('install "ipywidgets" for Jupyter support')

Sampling took 42s (285 draws/s)

Read the two equations separately

Compare posterior means to the true parameter values used in the DGP.

# Posterior summary
summary = az.summary(idata, var_names=["rho", "rho_zero", "alpha", "beta", "beta_zero"])
print(summary)

# Compare with true values
print("\n--- Parameter Recovery ---")
print(
    f"ρ (count SAR):  true={data['params_true']['rho']:.2f}, "
    f"posterior mean={float(idata.posterior['rho'].mean()):.3f} "
    f"± {float(idata.posterior['rho'].std()):.3f}"
)
print(
    f"ρ (zero SAR):   true={data['params_true']['rho_zero']:.2f}, "
    f"posterior mean={float(idata.posterior['rho_zero'].mean()):.3f} "
    f"± {float(idata.posterior['rho_zero'].std()):.3f}"
)
print(
    f"α (dispersion): true={data['params_true']['alpha']:.2f}, "
    f"posterior mean={float(idata.posterior['alpha'].mean()):.3f} "
    f"± {float(idata.posterior['alpha'].std()):.3f}"
)

# Note: spatial parameters (rho, rho_zero) require large samples (n ≥ 900)
# for reliable recovery in cross-sectional count models. The posterior
# is typically wide and may be attenuated toward zero at moderate n.
# The model raises a UserWarning when n < 900.
                 mean      sd eti89_lb eti89_ub ess_bulk ess_tail r_hat mcse_mean  mcse_sd
rho             0.387  0.0405     0.32     0.45     6293     5841  1.00   0.00051  0.00043
rho_zero        0.214   0.089    0.067     0.35     4637     5208  1.00    0.0013  0.00095
alpha            5.06   0.372      4.5      5.7     4932     5879  1.00    0.0053   0.0038
beta[x0]         1.03   0.067     0.93      1.1     6232     5934  1.00   0.00085  0.00071
beta[x1]       0.5826  0.0179     0.55     0.61     5364     6698  1.00   0.00024  0.00015
beta_zero[z0]   0.792   0.102     0.64     0.96     4070     5062  1.00    0.0016    0.001
beta_zero[z1]   1.009   0.065     0.91      1.1     2228     3740  1.00    0.0014  0.00065

--- Parameter Recovery ---
ρ (count SAR):  true=0.40, posterior mean=0.387 ± 0.040
ρ (zero SAR):   true=0.30, posterior mean=0.214 ± 0.089
α (dispersion): true=5.00, posterior mean=5.057 ± 0.372

What to check before trusting the output

Check R-hat and effective sample sizes to verify MCMC convergence.

# Convergence diagnostics
az.plot_trace_dist(idata, var_names=["rho", "rho_zero", "alpha"], compact=False)
plt.tight_layout()
plt.show()

# R-hat and ESS
diagnostics = az.summary(idata, var_names=["rho", "rho_zero", "alpha"])
print(diagnostics[["r_hat", "ess_bulk", "ess_tail"]])
/tmp/ipykernel_6201/1222109080.py:3: UserWarning: The figure layout has changed to tight
  plt.tight_layout()
../_images/d493aa0505d71e57a1cbe88b32aa68d31d679ceb41ff45ce5e0f48e0b046a881.png
         r_hat ess_bulk ess_tail
rho       1.00     6293     5841
rho_zero  1.00     4637     5208
alpha     1.00     4932     5879

Split the zeros between the two processes

The ZINB model decomposes observed zeros into two types:

  • Structural zeros: the observation is inactive (d=0)

  • Sampling zeros: the observation is active (d=1) but the NB draw was zero

The zero_attribution() method computes the posterior probability of each type for every zero observation.

# Zero attribution
attribution = model.zero_attribution()

print(f"Number of zero observations: {len(attribution['zero_indices'])}")
print(f"\nMean P(structural | y=0): {attribution['structural_prob'].mean():.3f}")
print(f"Mean P(sampling | y=0):   {attribution['sampling_prob'].mean():.3f}")

# Histogram of structural zero probabilities
fig, ax = plt.subplots(1, 2, figsize=(12, 4))
ax[0].hist(attribution["structural_prob"], bins=20, edgecolor="white")
ax[0].set_xlabel("P(structural zero | y=0)")
ax[0].set_ylabel("Count")
ax[0].set_title("Structural Zero Probabilities")

ax[1].hist(attribution["sampling_prob"], bins=20, edgecolor="white", color="C1")
ax[1].set_xlabel("P(sampling zero | y=0)")
ax[1].set_ylabel("Count")
ax[1].set_title("Sampling Zero Probabilities")
plt.tight_layout()
plt.show()
Number of zero observations: 853

Mean P(structural | y=0): 0.887
Mean P(sampling | y=0):   0.113
/tmp/ipykernel_6201/1143200135.py:19: UserWarning: The figure layout has changed to tight
  plt.tight_layout()
../_images/072c0e629e9d6e1d75877e3c4bb3babc7f39944fc328b508ca26182d13079249.png

Get per-observation activation probabilities

The activation_probabilities() method returns the posterior-mean probability that each observation is “active” (i.e., in the count regime rather than the always-zero regime).

# Activation probabilities
pi = model.activation_probabilities()

print(f"Mean activation probability:{pi.mean():.3f}")
print(f"Min activation probability: {pi.min():.3f}")
print(f"Max activation probability: {pi.max():.3f}")

# Compare with true d
d_true = data["d"]
print(f"\nTrue activation rate: {d_true.mean():.3f}")
print(f"Estimated activation rate: {pi.mean():.3f}")

# Scatter: pi vs observed y
fig, ax = plt.subplots(figsize=(8, 5))
ax.scatter(pi, y, alpha=0.5, s=20)
ax.set_xlabel("Estimated P(active)")
ax.set_ylabel("Observed count y")
ax.set_title("Activation Probability vs Observed Count")
ax.axhline(0, color="grey", linestyle="--", alpha=0.5)
plt.tight_layout()
plt.show()
Mean activation probability:0.698
Min activation probability: 0.084
Max activation probability: 0.990

True activation rate: 0.705
Estimated activation rate: 0.698
/tmp/ipykernel_6201/949050388.py:20: UserWarning: The figure layout has changed to tight
  plt.tight_layout()
../_images/aea5110a4ac0a2c0a61c609bda1ba6ca2b2f6a80a503de344aa728d7cb9551dc.png

Predict expected counts

The fitted_values() method returns E[y_i] = π_i · exp(η_i^cnt), the posterior-mean expected count for each observation.

# Fitted mean counts
fitted = model.fitted_values()

print(f"Mean fitted count: {fitted.mean():.2f}")
print(f"Mean observed count: {y.mean():.2f}")

# Scatter: fitted vs observed
fig, ax = plt.subplots(figsize=(8, 5))
ax.scatter(fitted, y, alpha=0.5, s=20)
ax.plot(
    [0, max(y.max(), fitted.max())],
    [0, max(y.max(), fitted.max())],
    "r--",
    alpha=0.5,
    label="45° line",
)
ax.set_xlabel("Fitted E[y]")
ax.set_ylabel("Observed y")
ax.set_title("Fitted vs Observed Counts")
ax.legend()
plt.tight_layout()
plt.show()
Mean fitted count: 4.37
Mean observed count: 4.33
/tmp/ipykernel_6201/2483833273.py:21: UserWarning: The figure layout has changed to tight
  plt.tight_layout()
../_images/f764e6cea5529090ab890afd15b2e35a2cbe244703f06740fe31cb0f3717538d.png

Inspect the posteriors

Visualize the posterior distributions of the spatial parameters rho and rho_zero, and the NB dispersion alpha, with true values marked.

# Posterior distributions with true values
truth = data["params_true"]
pc = az.plot_dist(
    idata, var_names=["rho", "rho_zero", "alpha"], figure_kwargs={"figsize": (14, 4)}
)
az.add_lines(pc, {name: truth[name] for name in ("rho", "rho_zero", "alpha")})
plt.show()

# Note: spatial parameters (rho, rho_zero) require large samples (n ≥ 900)
# for reliable recovery in cross-sectional count models. The posterior
# is typically wide and may be attenuated toward zero at moderate n.
../_images/26265774f006aeb90c73e5ed0073b8e48770706038629d2b34c96fee14621a7e.png

See also