Priors, and how much they matter¶
A student who has never seen a Beta distribution should still be able to state an assumption and be held to it. So priors are addressed by name, with the numbers they resolve to printed in every summary and one keystroke away for anyone who wants them.
import bayesplain as bp
print(bp.priors.describe())
uninformed Beta(1, 1) every rate between 0% and 100% starts out equally plausible
jeffreys Beta(0.5, 0.5) the conventional reference choice; slightly favours rates near 0 and 1, and is invariant to how the question is parameterised
gentle Beta(2, 2) mild pull toward the middle, worth about 2 observations either way; rules out nothing but keeps tiny samples from implying certainty
skeptical Beta(10, 10) assumes the rate is probably near 50% and takes real data to move; worth about 20 observations
The names describe what the assumption says, not what family it belongs to. None of them is neutral, and the summary output says so.
Five families, one argument¶
Each analysis takes prior=, and what it means depends on what is being
estimated.
Analysis |
Prior on |
Presets |
|---|---|---|
|
the rate, as |
|
|
cell probabilities, as |
same four names |
|
standardised effect size, as |
|
|
the correlation, as a stretched beta |
|
|
the rate, as |
|
|
standardised slope, as |
same four as means |
|
category shares, as |
same four as proportions |
The rate presets carry no exposure, so neither says anything about how large a
rate is — only how much to lean when events are rare. To say something about
size, use an earlier period:
bp.priors.from_previous_period(events=30, exposure=5).
The first two line up exactly rather than by analogy: for a 2×2 table of
successes and failures, concentration a on the columns is a Beta(a, a)
prior on each group’s rate.
bp.priors.resolve_proportion("gentle"), bp.priors.resolve_table("gentle")
(BetaPrior(a=2.0, b=2.0, name='gentle'),
ConcentrationPrior(a=2.0, name='gentle'))
Saying it in your own numbers¶
bp.proportion(34, 220, prior=(2, 5)).prior.label
'custom — Beta(2, 5)'
prior = bp.priors.from_previous_study(successes=12, n=80)
prior, prior.prior_mean, prior.prior_weight
(BetaPrior(a=13.0, b=69.0, name='prior study'), 0.15853658536585366, 82.0)
An earlier study is treated as data observed under a flat prior, which makes the strength of the assumption legible: 80 prior cases are worth 80 cases, no more.
Watching the prior stop mattering¶
import pandas as pd
rows = []
for n in (20, 100, 500, 2000):
for name in bp.priors.SENSITIVITY_LADDER:
res = bp.proportion(int(0.3 * n), n, prior=name)
rows.append({"n": n, "prior": name, "estimate": round(res.point(), 4)})
pd.DataFrame(rows).pivot(index="n", columns="prior", values="estimate")
| prior | gentle | jeffreys | skeptical | uninformed |
|---|---|---|---|---|
| n | ||||
| 20 | 0.3286 | 0.3034 | 0.3983 | 0.3126 |
| 100 | 0.3065 | 0.3007 | 0.3324 | 0.3026 |
| 500 | 0.3013 | 0.3001 | 0.3074 | 0.3005 |
| 2000 | 0.3003 | 0.3000 | 0.3019 | 0.3001 |
At n = 20 the choice moves the answer in the second decimal place. By n = 2000 the columns are indistinguishable. This is the week-3 lesson in one table: the prior matters exactly when the data are thin, which is exactly when you should be saying so out loud.
Reporting it honestly¶
.sensitivity() is a first-class method rather than an advanced footnote,
because prior sensitivity is the strongest legitimate objection to canned
Bayesian analysis, and reporting it by default converts that objection into a
scheduled lecture.
res = bp.compare_proportions([34, 51], [220, 240], labels=["A", "B"])
print(res.sensitivity())
HOW MUCH DOES THE PRIOR MATTER?
quantity: difference in rate (B − A)
interval in percentage points
prior 95% interval P(dir) BF10
--------------------------------------------------------------------------
uninformed — Beta(1, 1) −1.2 to 12.9 0.945 0.322
jeffreys — Beta(0.5, 0.5) −1.2 to 12.9 0.946 0.27
gentle — Beta(2, 2) −1.3 to 12.8 0.942 0.284
skeptical — Beta(10, 10) −1.8 to 12.3 0.922 0.017
Verdict: the estimate barely moves. The direction probability shifts by
less than 0.05 across every assumption tried and the interval
width by less than 1%, so this conclusion is driven by the data.
Report it, and note that you checked.
But note: the Bayes factor is a different story: it moves by a factor of
19 over the same priors, while the interval hardly budged. That
gap is not a bug in either number — it is why this package leads
with the estimate. Locating a value is a question the data can
mostly answer on its own; grading the evidence for one model
against another cannot be separated from what you assumed the
alternative looked like.
Two things to notice. The interval barely moves — this conclusion is driven by the data. The Bayes factor moves by a factor of nearly twenty over the same range. Both facts are true, and reporting only the second would be misleading in one direction while reporting only the first would be misleading in the other.
The cleanest demonstration¶
For means, the prior does not touch the estimate at all. The posterior uses the
standard reference prior and is fixed by the data; prior= enters only the
Bayes factor.
import numpy as np
rng = np.random.default_rng(0)
x, y = rng.normal(0, 1, 50), rng.normal(0.4, 1, 50)
rows = []
for name in bp.priors.EFFECT_SENSITIVITY_LADDER:
res = bp.compare_means(x, y, prior=name)
lo, hi = res.interval()
rows.append({
"prior": name,
"interval": f"{lo:.4f} to {hi:.4f}",
"BF10": round(float(np.exp(res.log_bf10)), 3),
})
pd.DataFrame(rows)
| prior | interval | BF10 | |
|---|---|---|---|
| 0 | modest | -0.0811 to 0.6921 | 0.769 |
| 1 | conventional | -0.0811 to 0.6921 | 0.624 |
| 2 | uninformed | -0.0811 to 0.6921 | 0.483 |
| 3 | generous | -0.0811 to 0.6921 | 0.361 |
Identical intervals to four decimal places; a Bayes factor that moves by half again. Locating a value is a question the data can mostly answer on its own. Grading the evidence for one model against another cannot be separated from what you assumed the alternative looked like.