neighbayes.models.SDEM¶
-
class neighbayes.models.SDEM(formula=
None, data=None, y=None, X=None, W=None, *, unit_effects=None, time_effects=False, dynamic=False, centered=False, unit_col=None, time_col=None, N=None, T=None, priors=None, logdet_method=None, robust=False, durbin_vars=None, logdet_decomp='auto', logdet_adapt=True)[source]¶ Bayesian Spatial Durbin Error Model.
Combines spatial lags of the regressors \(X\) with a spatial autoregressive disturbance:
\[y = X\beta + WX\theta + u, \quad u = \lambda Wu + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2 I).\]The sampled coefficient vector stacks the local and lagged-regressor blocks as \([\beta, \theta]\). The likelihood includes the spatial Jacobian \(\log|I - \lambda W|\).
- Parameters:¶
- formula : str, optional¶
Wilkinson-style formula, e.g.
"y ~ x1 + x2". Requiresdata. Intercept is included by default.- data : pandas.DataFrame or geopandas.GeoDataFrame, optional¶
Data source for formula mode.
- y : array-like, optional¶
Dependent variable of shape
(n,). Required in matrix mode.- X : array-like or pandas.DataFrame, optional¶
Design matrix. Required in matrix mode. DataFrame columns are preserved as feature names.
- W : libpysal.graph.Graph or scipy.sparse matrix¶
Spatial weights of shape
(n, n). Accepts alibpysal.graph.Graphor anyscipy.sparsematrix. The legacylibpysal.weights.Wobject is not accepted; passw.sparseorlibpysal.graph.Graph.from_W(w). Should be row-standardized; aUserWarningis raised otherwise.- priors : dict, optional¶
Override default priors. Supported keys:
lam_lower(float, default -1.0): Lower bound of the Uniform prior on \(\lambda\).lam_upper(float, default 1.0): Upper bound of the Uniform prior on \(\lambda\).beta_mu,beta_sigma(float or array, default Gelman et al. 2008): Normal prior on \([\beta, \theta]\), scaled tosd(y)and each column’s sd.sigma2_alpha(float, default 2.0): Shape of the InverseGamma prior on \(\sigma^2\).sigma2_beta(float, defaultVar(y)): Scale of the InverseGamma prior on \(\sigma^2\).nu(float, default 4.0): Fixed Student-t degrees of freedom (only used whenrobust=True).
- logdet_method : {"eigenvalue", "cheb", "aaa", "spline", "stochastic"}, optional¶
How
log|I - rho*W|is computed over rho: summed over the eigenvalues ofW; interpolated (Chebyshev, AAA rational, or cubic spline) through values from a sparse factorization, exact to the interpolant’s tolerance; or expanded stochastically with no factorization, the one method with sampling error.None(default) selects from the size and symmetry ofW:"eigenvalue"forn <= 500;"aaa"up ton = 60000(NEIGHBAYES_LOGDET_FACTOR_MAX_N) unless the factorization’s fill-in would dominate;"stochastic"beyond.- logdet_decomp : {"auto", "chol", "lu"}, default "auto"¶
The factorization an interpolating method uses: sparse Cholesky (undirected
W; scikit-sparse CHOLMOD) or sparse LU (KLU or UMFPACK, chosen by the sparsity pattern)."auto"takes"chol"where the method has it andWis undirected, else"lu". Ignored by"eigenvalue"and"stochastic";"spline"on"chol"raises.- logdet_adapt : bool, default True¶
Halfway through warmup, let the Gibbs sampler fit the log-determinant to where the chains have found the posterior rather than to the prior interval. An interpolant on Cholesky or LU values (
"cheb","aaa") is refit on that range, padded by ten warmup standard deviations; the window becomes the sampler’s support, is recorded inidata.attrs["logdet_refit_window"], and a warning is raised if the retained draws reach an edge it introduced."aaa"also adds exact factorizations when the posterior lies near a singularity (idata.attrs["logdet_aaa_nodes"])."stochastic"grows its Hutchinson probes from 50 until the bias they leave in the posterior mean of the spatial parameter is expected to stay under half the run’s own Monte Carlo error,1/sqrt(ESS)estimated in warmup: at most 1,000 probes up to 100,000 units, fewer beyond (idata.attrs["logdet_probes"]).Falsekeeps the fit on the prior interval.- robust : bool, default False¶
If True, replace the Normal disturbance with Student-t. See Robust regression below.
- durbin_vars : list of str, optional¶
Names of X columns to spatially lag. By default all non-constant columns are lagged. Pass a subset to restrict which variables receive a spatial lag, e.g.
durbin_vars=["income", "density"].- time_col : str, optional¶
Formula-mode panel: the period and unit columns of
data. Given time, the model is a panel, stacked by period, then unit.- unit_col : str, optional¶
Formula-mode panel: the period and unit columns of
data. Given time, the model is a panel, stacked by period, then unit.- N : int, optional¶
Matrix-mode panel: the numbers of units and periods;
T=1is the cross-section.- T : int, optional¶
Matrix-mode panel: the numbers of units and periods;
T=1is the cross-section.- unit_effects : {None, "fixed", "random"}, default None¶
Panel unit effects:
"fixed"applies the within transformation (with Lee and Yu’s 2010 likelihood dimensions),"random"adds a normal effect per unit,unit_effectwith sdunit_effect_sd(priorsunit_effect_sd_nuandunit_effect_sd_scale, half-Cauchy onsd(y)by default).- time_effects : bool, default False¶
Fixed time effects (panels).
- dynamic : bool, default False¶
Add the time lag \(\phi y_{t-1}\) (priors
phi_lower,phi_upper; thelambounds default to±0.95). A dynamic panel takes no unit effects (Nickell 1981).- centered : bool, default False¶
With random unit effects, sample them centered under NUTS rather than as \(\alpha_i = \sigma_\alpha z_i\); better when every unit has many observations.
Notes
Because the spatial autoregression enters only through the disturbance, direct effects equal \(\beta\) and indirect effects equal \(\theta\) (no global spillover multiplier).
Robust regression
When
robust=True, the spatially-filtered innovation is Student-t:\[\varepsilon = (I - \lambda W)(y - X\beta - WX\theta) \sim t_\nu(0, \sigma^2 I)\]where \(\nu\) is a fixed hyperparameter set by
priors={"nu": value}(default 4, LeSage’srval); larger values approach the Normal. Values must exceed 2 so the variance exists.-
__init__(formula=
None, data=None, y=None, X=None, W=None, *, unit_effects=None, time_effects=False, dynamic=False, centered=False, **kwargs)[source]¶
Methods
__init__([formula, data, y, X, W, ...])fit([draws, tune, chains, target_accept, ...])Draw samples from the posterior.
Return fitted values at posterior mean parameters.
impacts([return_posterior_samples])Compute Bayesian inference for direct, indirect, and total impacts.
predict(X_new, W[, oos, thin, random_seed])Posterior predictive distribution of
yat new units.predict_in_sample(*[, thin, random_seed])Leave-one-out conditional predictive of each fitted unit.
Return residuals
y - fitted_values.Run Bayesian LM specification tests and return a summary table.
spatial_diagnostics_decision([alpha, format])Return a model-selection decision from Bayesian LM test results.
summary([var_names])Return posterior summary table.
Attributes
Return the ArviZ DataTree from the most recent fit.
Return the PyMC model object built for the most recent fit.
-
fit(draws=
2000, tune=1000, chains=4, target_accept=None, random_seed=None, progressbar=True, sampler=None, backend=None, thin=1, n_jobs=-1, idata_kwargs=None, **sample_kwargs)[source]¶ Draw samples from the posterior.
Dispatches to this model’s Gibbs sampler (
sampler="gibbs") or NUTS (sampler="nuts"). WhensamplerisNone(default), Gibbs is used if the model has a registered Gibbs sampler, otherwise NUTS.- Parameters:¶
- draws : int¶
Post-warmup draws, warmup steps, and number of chains.
- tune : int¶
Post-warmup draws, warmup steps, and number of chains.
- chains : int¶
Post-warmup draws, warmup steps, and number of chains.
- target_accept : float, optional¶
Target acceptance rate for NUTS. NUTS-only: passing it with the Gibbs sampler raises
TypeError. Defaults to0.9for NUTS.- random_seed : int, optional¶
Seed for reproducibility.
- progressbar : bool, default True¶
Show progress bar(s) during sampling.
- sampler : {"gibbs", "nuts", None}, default None¶
Sampling method.
Noneauto-selects Gibbs when this model has one, else NUTS.- backend : str, optional¶
What executes the sampler. For Gibbs,
"auto"(the default),"jax"or"numpy";"auto"takes JAX when it is installed and supported by the family, else NumPy. For NUTS,"pymc"(the default),"nutpie","numpyro"or"blackjax".- thin : int, default 1¶
Keep every
thin-th post-warmup Gibbs draw (Gibbs only).- n_jobs : int, default -1¶
Parallel workers for the NumPy Gibbs path (Gibbs only).
- idata_kwargs : dict, optional¶
{"log_likelihood": True}stores the complete Jacobian-corrected pointwise log-likelihood thataz.loo/az.compareneed, for Gibbs and NUTS alike. Off by default, as in PyMC: it holds one value per draw, chain, and observation (16 GB at n = 250,000 with 4 × 2,000 draws). For NUTS the dict is also passed topm.sample.- **sample_kwargs¶
For NUTS, forwarded to
pm.sample; for Gibbs, the family’s declared options (an unsupported key raises).
- Return type:¶
xarray.DataTree
-
impacts(return_posterior_samples=
False)[source]¶ Compute Bayesian inference for direct, indirect, and total impacts.
Computes impact measures for each posterior draw, then summarizes the posterior distribution with means, 95% credible intervals, and Bayesian p-values. This is the fully Bayesian analog of the simulation-based approach in LeSage and Pace [2009] and the asymptotic variance formulas in Arbia et al. [2020].
Models without a spatial lag on y do not exhibit global feedback propagation through \((I-\\rho W)^{-1}\). However, models with spatially lagged covariates (SLX, SDEM) can still have non-zero neighbor spillovers captured in the indirect term.
- Parameters:¶
- Returns:¶
If return_posterior_samples is
False(default), returns a DataFrame indexed by feature names with columns for posterior means, credible-interval bounds, and Bayesian p-values.If return_posterior_samples is
True, returns(DataFrame, dict)where the dict has keys"direct","indirect","total", each mapping to a(G, k)array of posterior draws.- Return type:¶
-
predict(X_new, W, oos=
None, *, thin=1, random_seed=None)[source]¶ Posterior predictive distribution of
yat new units.For each posterior draw the outcome at the new units \(O\), given the fitted outcomes \(y_S\), is exactly
\[y_O \mid y_S \sim N\bigl(\mu_O - \Lambda_{OO}^{-1} \Lambda_{OS}(y_S - \mu_S),\; \Lambda_{OO}^{-1}\bigr), \qquad \Lambda = (I - \theta W)^{\top}(I - \theta W)/\sigma^2,\]the best predictor (BP) of Goulard et al. [2017] (eq. 9) at that draw. \(\theta\) is \(\rho\) for SAR and SDM, \(\lambda\) for SEM and SDEM, and zero for OLS and SLX, whose predictive is \(N(\mu_O, \sigma^2 I)\). The marginal mean \(\mu\) is \((I - \rho W)^{-1} Z\beta\) for the lag models and \(Z\beta\) otherwise, computed on the union graph.
- Parameters:¶
- X_new : DataFrame or array of shape (n_new, k)¶
Covariates of the new units. For a model built from a formula, a DataFrame with the variables the formula uses; the formula’s transforms are replayed on it. Otherwise, columns in the order of the fitted
X(or a DataFrame with its column names).- W : libpysal.graph.Graph or scipy.sparse matrix¶
Weights over the union of fitted and new units,
n_fit + n_newsquare, standardized the way the fittedWwas. For SLX, SDM and SDEM the lagged covariates of every unit are recomputed on this graph, since a fitted unit with a new neighbor gets a newWXrow.- oos : array of int, optional¶
Positions of the new units in
W, in the row order ofX_new; the fitted units fill the remaining positions in their fitted order. Defaults to the lastn_newpositions.- thin : int, default 1¶
Use every
thin-th draw of each chain.- random_seed : int, optional¶
Seed for the predictive draws.
- Returns:¶
A
predictionsgroup with dimensions(chain, draw, obs_new)holdingy(posterior predictive draws),y_bp(the conditional mean at each draw; its average over draws is the posterior predictive mean) andy_tc(the trend predictor \(\mu_O\), which ignores the observed neighbors), and apredictions_constant_datagroup withX_new. Comparingy_bpwithy_tcshows what the spatial correction adds.- Return type:¶
xarray.DataTree
Notes
The parameters were estimated on the fitted units’
W, and the prediction uses the unionW: the feasible practice of Goulard et al. [2017] (their eq. 8).
-
predict_in_sample(*, thin=
1, random_seed=None)[source]¶ Leave-one-out conditional predictive of each fitted unit.
For each unit \(i\) and posterior draw, the distribution of \(y_i\) given every other fitted outcome is
\[y_i \mid y_{-i} \sim N\bigl(y_i - (\Lambda r)_i/\Lambda_{ii}, \; 1/\Lambda_{ii}\bigr), \qquad r = y - \mu,\]the in-sample BP of Goulard et al. [2017] (eq. 14). All units together cost one sparse matrix-vector product per draw.
The draws condition on the posterior from the full data, which includes \(y_i\), so this is a smoother, not cross-validation; use
neighbayes.diagnostics.spatial_kfold()to score held-out predictive accuracy.- Parameters:¶
- Returns:¶
A
predictionsgroup with dimensions(chain, draw, obs)holdingy(draws),y_bp(the leave-one-out conditional mean) andy_tc(the marginal mean \(\mu\)). Each unit’s draws follow its own conditional; they are marginally correct per unit but not a joint draw across units.- Return type:¶
xarray.DataTree
- property pymc_model[source]¶
Return the PyMC model object built for the most recent fit.
For a Gibbs-fitted cross-section the PyMC model is not constructed during sampling; it is built lazily on first access so that downstream consumers (e.g. bridge sampling for marginal likelihoods) can evaluate
logpand the prior under the same model definition used by the NUTS path.
- residuals()[source]¶
Return residuals
y - fitted_values.- Returns:¶
Residual vector
y - fitted_valueson the same scale asfitted_values().- Return type:¶
np.ndarray
- spatial_diagnostics()[source]¶
Run Bayesian LM specification tests and return a summary table.
Looks up the diagnostic suite registered for this model class and calls each test function on this fitted model, collecting the results into a tidy DataFrame. The set of tests depends on the model type — for example, an OLS model runs LM-Lag, LM-Error, LM-SDM-Joint, and LM-SLX-Error-Joint, while an SAR model runs LM-Error, LM-WX, and Robust-LM-WX. Panel models run the
Panel--prefixed analogues (e.g. Panel-LM-Lag).Requires the model to have been fit (
.fit()called) and a spatial weights matrixWto have been supplied at construction time.- Returns:¶
DataFrame indexed by test name with columns:
Column
Description
statistic
Posterior mean of the LM statistic
median
Posterior median of the LM statistic
df
Degrees of freedom for the \(\chi^2\) reference
p_value
Bayesian p-value:
1 - chi2.cdf(mean, df)ci_lower
Lower bound of 95% credible interval (2.5%)
ci_upper
Upper bound of 95% credible interval (97.5%)
The DataFrame has
attrs["model_type"](class name) andattrs["n_draws"](total posterior draws) metadata.- Return type:¶
pandas.DataFrame
- Raises:¶
RuntimeError – If the model has not been fit yet.
ValueError – If no spatial weights matrix
Wwas supplied.
See also
spatial_diagnostics_decisionModel-selection decision based on the test results.
impactsPosterior inference for direct/indirect/total impacts.
Examples
>>> ols = OLS(formula="price ~ income + crime", data=df, W=w) >>> ols.fit() >>> ols.spatial_diagnostics() statistic median df p_value ci_lower ci_upper LM-Lag 3.21 2.98 1 0.073 0.12 8.54 LM-Error 5.67 5.34 1 0.017 0.34 12.10 LM-SDM-Joint 7.89 7.12 4 0.096 1.23 18.32 LM-SLX-Error-Joint 6.45 5.98 4 0.168 0.89 15.67
-
spatial_diagnostics_decision(alpha=
0.05, format='graphviz')[source]¶ Return a model-selection decision from Bayesian LM test results.
One rule reads every fitted model’s tests: the specific-to-general rule of Anselin et al. [1996], which
spreg’sstge_classicimplements. A model that leaves two directions untested runs a plain and a robust LM test for each:Neither plain test is significant → keep the model.
One is → add that direction.
Both are → add the direction whose robust test is significant; if both or neither are, the one with the smaller robust p-value.
A plain test is valid when the other direction is absent and a robust one is valid either way, at a cost in power, so the robust pair only chooses between two plain rejections. The SDM and SDEM leave one direction, and its test decides. The directions, and the models they lead to:
OLS: lag → SAR, error → SEM,
WX→ SLX. With three directions untested, the lag and the error together go to their robust pair whether or notWXalso rejects (the SAR or SEM then tests it); the lag andWXgo to the robust pair of Koley and Bera [2024], whose lag test is LM-Error at the OLS fit; the error andWX, whose scores are uncorrelated there, to the smaller p-value.SAR:
WX→ SDM, error → SARAR.SEM: lag → SARAR,
WX→ SDEM.SLX: lag → SDM, error → SDEM.
SDM: error → GNS. SDEM: lag → GNS.
SARAR and GNS are where the tests point; neighbayes does not fit them. The
WXtests are those of Koley and Bera [2024]. Panel models run thePanel-tests through the same trees ([Elhorst, 2014]), and the Tobit classes the same trees with Tobit leaves. The logit, probit and NB at their aspatial null read their tests as a pair of their own.- Parameters:¶
- alpha : float, default 0.05¶
Significance level for the Bayesian p-values.
- format : {"graphviz", "ascii", "model"}, default "graphviz"¶
Output format.
"model"returns the recommended-model name string."ascii"returns an indented box-drawing rendering of the full decision tree with the chosen path highlighted."graphviz"returns agraphviz.Digraphobject that renders inline in Jupyter; if the optionalgraphvizpackage is not installed aUserWarningis issued and the ASCII rendering is returned instead.
- Returns:¶
Recommended model name when
format="model", an ASCII tree string whenformat="ascii", or agraphviz.Digraphwhenformat="graphviz"(with ASCII fallback on missing dep).- Return type:¶
str or graphviz.Digraph
See also
spatial_diagnosticsCompute the Bayesian LM test statistics.
References
Koley and Bera [2024], Anselin et al. [1996], Elhorst [2014]