Skip to content

Balancing

Balancing sub-package (balancer, base weights, importance).

Orchestrates the core balancing loop:

  1. Base weights (base_weights) -- compute initial expansion factors per zone: target_hh_pop / n_responses.
  2. Importance (importance) -- derive MOE-based per-control importance from PUMS replicate weights, with explicit YAML overrides.
  3. Balancer (balancer) -- maximum-entropy list balancing via np_balancer_numba, vendored from PopulationSim. Runs independently per geography zone.

Household weights leave here for processing.weighting.core.propagation, which carries them down the canonical hierarchy.

base_weights

Initial (base) expansion weights for the balancer.

Before balancing, each survey household needs a starting weight that reflects its basic expansion factor — the number of real-world households it represents. Without meaningful initial weights, the Newton-Raphson solver starts from 1.0 and must bridge the gap to expansion factors of hundreds or thousands, quickly slamming into the expansion-factor constraints.

This module provides two paths:

  1. Response inversion (default) — target_hh_pop / n_responses per zone. Works whenever PUMS-derived control totals are available (always, in our pipeline).

  2. Sample plan — a SamplePlan object mapping Census block groups to sampling strata (segments). Block-group populations are sourced from the crosswalk (Census block population summed to BG level). Each household is assigned a block group (via spatial join), mapped to a segment via the plan, and receives a segment-level initial weight: segment_pop / segment_responses.

The public entry point is compute_base_weights, which adds a base_weight column to the seed table.

importance

MOE-based importance weight calculation.

Uses PUMS successive-difference replicate weights (WGTP1-80 / PWGTP1-80) to estimate the Margin of Error (MOE) for each weighted control total, then converts to importance weights inversely proportional to the coefficient of variation (CV).

Controls with higher sampling uncertainty (larger CV) receive lower importance so the balancer doesn't chase noisy targets. Controls absent from the returned dict (e.g. structural totals whose MOE is meaningless) fall back to the balancer's default importance.

balancer

Maximum-entropy list balancer.

Thin Polars→numpy bridge around np_balancer_numba, vendored from PopulationSim. Runs independently per geography zone.

Algorithm

Find weight vector w closest to seed weights w₀ (KL-divergence) subject to marginal constraints:

$$ \min \sum_i w_i \ln(w_i / w_{0i}) \quad \text{s.t.} \quad Aw = t,\; w_i \ge 0 $$

where A is the incidence matrix and t is the target totals vector.

Implementation

Calls np_balancer_numba from _np_balancer, vendored from PopulationSim v0.10.0 -- a pure @njit function taking numpy arrays, so PopulationSim is not a dependency. Zones are independent and run in parallel via ThreadPoolExecutor.

Configuration (YAML)

The balancing: block of compute_weights (see BalancingConfig):

balancing:
  max_expansion_factor: 5.0     # upper bound = initial_weight x factor
  min_expansion_factor: 0.01    # lower bound = initial_weight x factor
  min_weight: 1                 # absolute floor
  # max_weight: null            # absolute ceiling (optional)
  max_iterations: 10000
  n_workers: 8                  # zones balanced in parallel
  default_importance: 100
  moe_based_importance: true

Code: processing.weighting.balancing