16  Segregation Measures

Code
import os

import geopandas as gpd
import matplotlib.pyplot as plt
import networkx as nx
import pandas as pd
import seaborn as sns
import contextily as ctx
from factor_analyzer import FactorAnalyzer
from geosnap import DataStore
from geosnap import io as gio
from networkx.drawing.nx_agraph import graphviz_layout
from segregation.batch import batch_compute_multigroup, batch_compute_singlegroup
from segregation.local import LocalDistortion, MultiLocationQuotient
from segregation.multigroup import MultiInfoTheory, MultiGini, MultiDiversity
from segregation.singlegroup import Dissim, Gini, Entropy
from sklearn.cluster import AffinityPropagation, AgglomerativeClustering, KMeans
from sklearn.metrics import silhouette_score
from scipy.stats import zscore

%load_ext watermark
%watermark -iv -a "eli knaap"
OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Author: eli knaap

segregation    : 2.5.3.dev3+g0cb426cf2.d20251118
contextily     : 1.6.2
scipy          : 1.16.3
factor_analyzer: 0.5.1
networkx       : 3.5
geosnap        : 0.15.3
seaborn        : 0.13.2
pandas         : 2.3.3
matplotlib     : 3.10.8
geopandas      : 1.1.1
sklearn        : 1.7.2

Segregation is in many ways the core of urban inequality research; any discussion of urban analytics is incomplete without decent coverage of segregation measurement. One of the most obvious and long-studied avenues for creating inequality between groups is to partition access to different resources. This is also one of the most important legacies of institutionalized racism in the United States, where the unequal distribution of public resources has repeatedly been shown to be illegal. In other words, there is an important legacy of urban inequality research closely related to legal challenges against federal policy; if policies governing public resources (like education, housing, transportation infrastructure, clean air and water, etc.) result in segregation (for some protected class), then they are inherently unequal (and ultimately unconstitutional, thanks to the equal protection clause).

Segregation of white and colored children in public schools has a detrimental effect upon the colored children. The impact is greater when it has the sanction of the law, for the policy of separating the races is usually interpreted as denoting the inferiority of the Negro group… Any language in contrary to this finding is rejected. We conclude that in the field of public education the doctrine of ‘separate but equal’ has no place. Separate educational facilities are inherently unequal.

Warren (1954)

In concept, segregation is about separation; when we measure residential segregation, we are asking whether people belonging to different groups share the same space, often conceived as the same ‘neighborhood’. This is a more ambiguous task than measuring, for example, educational segregation, where the shared resource such as schools are very well-defined. Residential space can be measured at the scale of the room, housing unit, building, collection of buildings, neighborhood, city, region, and all the way on (we explore this variable concept of scale in the next section).

Thus in the case of residential segregation we are forced to rely on the fuzzy notion of neighborhoods. In practice, this means that researchers simply adopt the tract or blockgroup as a placeholder for the neighborhood, then examine segregation as defined by these units. Here, we’ll use PySAL’s segregation module to analyze residential segregation by race and ethnicity in Southern California, and we begin by collecting data for the entire region, then partitioning it into the coastal and inland sections.

Code
datasets = DataStore()

socal = gio.get_acs(
    datasets,
    county_fips=["06037", "06025", "06059", "06071", "06073", "06065", "06111"],
    years=[2018],
)
socal = socal.to_crs(socal.estimate_utm_crs())
socal[["p_hispanic_persons", "geometry"]].assign(geometry=socal.geometry.simplify(100)).explore(
    column="p_hispanic_persons",
    scheme="quantiles",
    cmap="Blues",
    k=8,
    tooltip=["p_hispanic_persons"],
    style_kwds={"weight": 0.5},
    tiles='CartoDB Positron'
)
socal["county"] = socal.geoid.str[:5]
county_names = [
    "Imperial",
    "Los Angeles",
    "Orange",
    "Riverside",
    "San Bernadino",
    "San Diego",
    "Ventura",
]
county_fips = ["06025", "06037", "06059", "06065", "06071", "06073", "06111"]
namer = dict(zip(county_fips, county_names))
socal['county'] = socal.county.replace(to_replace=namer)

coastal = socal[socal.county.isin(["Los Angeles", "Orange", "San Diego", "Ventura"])]
inland = socal[socal.county.isin(['Riverside', "San Bernadino", "Imperial"])]

f, ax = plt.subplots(1,2, figsize=(10, 5))
coastal.plot(column='county', ax=ax[0])
inland.plot(column='county', ax=ax[1])
/Users/knaaptime/miniforge3/envs/urban_analysis/lib/python3.12/site-packages/geosnap/io/constructors.py:218: UserWarning: Currency columns unavailable at this resolution; not adjusting for inflation
  warn(

Coastal and Inland Southern California

Coastal and Inland Southern California

16.1 Residential Segregation Measures

The segregation package calculates dozens of segregation indices, each of which captures something different about the ways that population groups interact or remain separated in space. Most of the commonly-used statistics are global or aggregate measures, meaning they summarize the total level of segregation across all units in a study region. For an overview of segregation measure nomenclature, formulae, and canonical citations, see the tables at the end of Grannis (2002).

16.1.1 Single-Group Indices

Single-group indices measure the partitioning of one population group relative to everyone else. Early segregation work in the U.S. tends to focus on Black-white segregation, but it is also common to see work focused on a particular minority population versus the rest of the population (e.g. Black vs all other groups). To generate a single-group measure using the segregation package, you pass a dataframe holding population counts for each geographic unit, and the names of the columns for the focal population and the reference population (i.e. the minority the total population).

Code
dissim_hisp = Dissim(socal, "n_hispanic_persons", "n_total_pop")
dissim_black = Dissim(socal, "n_nonhisp_black_persons", "n_total_pop")

gini_hisp = Gini(socal, "n_hispanic_persons", "n_total_pop")
gini_black = Gini(socal, "n_nonhisp_black_persons", "n_total_pop")

entropy_hisp = Entropy(socal, "n_hispanic_persons", "n_total_pop")
entropy_black = Entropy(socal, "n_nonhisp_black_persons", "n_total_pop")

Here, we fit three segregation measures: Dissimilarity (\(D\)), Gini (\(G\)), and Entropy (also called the Information Theory index, sometimes denoted [Thiel’s] \(H\)) for the Black population and the Hispanic/Latino populations in the southern California region. Each class has a statistic attribute that holds the computed value for each segregation measure.

Code
dissim_hisp.statistic
np.float64(0.4995777695234679)
Code
dissim_black.statistic
np.float64(0.547197680270968)
Code
gini_hisp.statistic
0.6602166788700566
Code
gini_black.statistic
0.7234615852802052
Code
entropy_hisp.statistic
np.float64(0.2714618709533524)
Code
entropy_black.statistic
np.float64(0.2616509031724341)

According to the Dissimilarity and Gini indices, the black population in southern California is more segregated than the Latinx/Hispanic population, but the reverse is true according to the Entropy index.

16.1.1.1 Batch Computation

To examine several indices at once, segregation provides a set of “batch_compute” functions. Instead of a fitted Class, the batch_compute_singlegroup function returns a table of segregation indices and is a convenient way of collecting many statistics simultaneously.

Code
socal_all_singlegroup = batch_compute_singlegroup(socal, "n_hispanic_persons", "n_total_pop")
Code
socal_all_singlegroup
Statistic
Name
AbsoluteCentralization 0.7737
AbsoluteClustering 0.2704
AbsoluteConcentration 0.6548
Atkinson 0.3726
BiasCorrectedDissim 0.4993
BoundarySpatialDissim NaN
ConProf 0.4794
CorrelationR 0.3272
Delta 0.8843
DensityCorrectedDissim 0.3856
Dissim 0.4996
DistanceDecayInteraction 0.4788
DistanceDecayIsolation 0.5741
Entropy 0.2715
Gini 0.6602
Interaction 0.3723
Isolation 0.6277
MinMax 0.6663
ModifiedDissim 0.4901
ModifiedGini 0.6509
PARDissim 0.4808
RelativeCentralization 0.0739
RelativeClustering 0.0667
RelativeConcentration 0.3460
SpatialDissim 0.3593
SpatialProxProf 0.8412
SpatialProximity 1.1710

16.1.2 Multi-group Indices

Multi-group measures capture the partitioning of several population groups simultaneously. Most multi-group measures are extensions of single-group measures and have a more recent history in the literature (Reardon & Firebaugh, 2002).

Code
pop_groups = ['n_asian_persons', 'n_hispanic_persons', 'n_nonhisp_black_persons', 'n_nonhisp_white_persons']

multi_div_coast = MultiDiversity(coastal, pop_groups)
multi_div_inland = MultiDiversity(inland, pop_groups)


multi_info_coast = MultiInfoTheory(coastal, pop_groups)
multi_info_inland = MultiInfoTheory(inland, pop_groups)

For multigroup diversity:

Code
print(f"coast: {multi_div_coast.statistic}")
print(f"inland: {multi_div_inland.statistic}")
coast: 1.1839984141407496
inland: 1.06835444467478

for multigroup information theory:

Code
print(f"coast: {multi_info_coast.statistic}")
print(f"inland: {multi_info_inland.statistic}")
coast: 0.3180256187077314
inland: 0.2040707291185395

Regardless which index is used, multigroup segregation is higher in the coastal region than the inland one

16.1.2.1 Batch Computation

Again, the measures can be “batch computed”

Code
socal_all_multigroup
Statistic
Name
GlobalDistortion 335.5186
MultiDissim 0.5131
MultiDivergence 0.3498
MultiDiversity 1.1664
MultiGini 0.6766
MultiInfoTheory 0.2999
MultiNormExposure 0.3048
MultiRelativeDiversity 0.2955
MultiSquaredCoefVar 0.2508
SimpsonsConcentration 0.3517
SimpsonsInteraction 0.6483

16.1.3 Local Segregation Measures

Unlike global measures, local segregation statistics measure segregation in each geographic unit rather than summarizing segregation across the region. For example the recently proposed Distortion index is designed to visualize how segregation changes over a region (Bézenac et al., 2022; Olteanu et al., 2019).

The use of trajectory convergence analysis provides a flexible way for capturing change across all scales from small spatial units and how the rate of convergence to the citywide average modifies over space. Thus, the method provides an analysis of how far, in spatial terms, any individual or neighborhood is from the citywide multigroup distribution.

Code
d = LocalDistortion(socal, groups=pop_groups)
ax = d.data.plot('distortion',  scheme='quantiles', cmap='RdBu_r', alpha=0.6, )
ctx.add_basemap(ax=ax, crs=socal.crs)

Code
d.data.assign(geometry=d.data.geometry.simplify(100)).explore(
    "distortion",
    cmap="RdBu_r",
    style_kwds={"weight": 0.5},
    scheme="quantiles",
    tiles="CartoDB Positron",
)
Make this Notebook Trusted to load map: File -> Trust Notebook