processing.weighting.validation

Validation sub-package.

checksums — recode null detection and incidence-sum overcount checks (pre-balancing).
control_validation — control configuration and cross-tab marginal consistency (pre-balancing).
coverage — how much of a profile's universe the control geography can place.
weight_checks — post-balancing sanity checks: weighted totals against the PUMS-derived control targets, and the hierarchy identities.

processing.weighting.validation.checksums

Incidence-sum checksums for the weighting pipeline.

Two complementary checks share a common reporting backend:

check_recode_nulls Runs on a recoded DataFrame before aggregation. Detects records where a control column evaluated to null (a gap in the recode mapping). Logs a warning by default; raises with strict=True.

check_incidence_sums Runs on the household-level incidence table after aggregation (and after fractional imputation for survey data). For each non-structural control it verifies that member-column sums match the expected structural total (p_total for person controls, h_total for household controls). Mismatches always raise ValueError.

check_recode_nulls

check_recode_nulls(
    df: pl.DataFrame,
    targets: list[str],
    *,
    level: ControlLevel,
    id_col: str,
    source_label: str,
    strict: bool = False
) -> None

Check for records whose control recode evaluated to null.

Parameters:

Name	Type	Description	Default
`df`	`pl.DataFrame`	Recoded table (households or persons).	required
`targets`	`list[str]`	Control registry names to check.	required
`level`	`ControlLevel`	Which level of controls to check (`HOUSEHOLD` or `PERSON`).	required
`id_col`	`str`	Record identifier column (e.g. `"hh_id"`, `"person_id"`).	required
`source_label`	`str`	Human label for log messages (e.g. `"PUMS"` or `"survey"`).	required
`strict`	`bool`	If `True`, raise `ValueError` on null recodes instead of just logging a warning. PUMS recodes should always be strict (every person must land in exactly one category); survey recodes may tolerate nulls until imputation is implemented.	`False`

check_incidence_sums

check_incidence_sums(
    seed: pl.DataFrame,
    targets: list[str],
    *,
    source_label: str,
    tolerance: float = 0.0
) -> None

Raise on incidence-column mismatches vs structural totals.

For person-level controls, sum({ctrl}__*) must equal p_total. For household-level controls, sum({ctrl}__*) must equal h_total (always 1). Both overcounts and undercounts are checked.

After fractional imputation, values may be non-integer — set tolerance to a small float (e.g. 0.01) to allow for floating-point drift.

Parameters:

Name	Type	Description	Default
`seed`	`pl.DataFrame`	Household-level incidence table with structural columns (`p_total`, `h_total`) and `{ctrl}__{member}` columns.	required
`targets`	`list[str]`	Control registry names.	required
`source_label`	`str`	Label for log messages.	required
`tolerance`	`float`	Maximum acceptable absolute deviation from the expected sum. Use `0.0` for exact integer match (PUMS), `0.01` for post-imputation survey data.	`0.0`

Raises:

Type	Description
`ValueError`	If any household has an incidence-sum mismatch.

processing.weighting.validation.control_validation

Control configuration and data validation.

Validates control configurations and cross-tab marginal consistency before the balancer runs. These checks catch configuration errors early (e.g., too many categories, inconsistent marginals) rather than failing deep in the balancing loop.

Functions:

validate_total_control_categories Validates total category count across all controls. Logs info at 100 categories, raises error at 200. Prevents balancer performance issues from excessive categories.

validate_crosstab_margins Validates that cross-tab marginals sum to match 1D dimension control targets. Ensures joint distributions are consistent with marginals.

validate_total_control_categories

validate_total_control_categories(
    controls: list[ControlTarget],
) -> None

Validate total category count across all controls.

Counts the total number of valid (non-sentinel) categories across all controls and logs a breakdown. Logs a warning if count exceeds INFO_TOTAL_CATEGORIES (100) or raises an error if count exceeds MAX_TOTAL_CATEGORIES (200).

Parameters

controls : list[ControlTarget] List of control instances to validate.

Raises:

ValueError If total category count exceeds MAX_TOTAL_CATEGORIES.

validate_crosstab_margins

validate_crosstab_margins(
    totals_df: pl.DataFrame,
    ctrl_instances: list[ControlTarget],
    specs: list[ControlSpec],
) -> None

Validate that cross-tab marginals match 1D dimension control targets.

For each cross-tab control, checks if any of its dimension controls are also present in the control set. If so, computes marginals by summing cross-tab cells for each dimension value and compares to 1D targets.

Parameters

totals_df : pl.DataFrame Combined totals from all controls (geo_id, control_name, category, target_total). ctrl_instances : list[ControlTarget] List of ControlTarget instances. specs : list[ControlSpec] List of control specifications (for names).

Raises:

ValueError If cross-tab marginals don't match 1D dimension control targets within tolerance.

warn_crosstab_sparsity

warn_crosstab_sparsity(
    seed_incidence: pl.DataFrame,
    ctrl_instances: list[ControlTarget],
    *,
    geo_col: str = "ctrl_geoid",
    threshold: int = _SPARSE_CELL_THRESHOLD
) -> None

Log warnings for cross-tab cells with low unweighted sample counts.

For each cross-tab control, sums the binary incidence columns per zone and reports any cells below threshold. This catches data sparsity that may cause the balancer to produce extreme weights.

Only cross-tab controls are checked — 1-D controls are already covered by the diagnostics sparsity table.

Parameters

seed_incidence : pl.DataFrame Household-level incidence matrix with ctrl_geoid and binary {control}__{category} columns. ctrl_instances : list[ControlTarget] Resolved control instances (may include both 1-D and cross-tab). geo_col : str Geography column in the incidence table. threshold : int Minimum unweighted count per cell. Cells below this trigger a warning log message.

processing.weighting.validation.coverage

How much of a profile's universe the control geography can place.

Two independent tests decide whether a household is "in the region", and they are allowed to disagree:

the model geography, assigned by add_zone_ids with a nearest-zone snap tolerance, which the usability profiles read through their zone axis;
the control geography, a strict point-in-polygon of the home coordinate against the target-zone file, which is what gives a household a balancing zone.

A household that passes the first and fails the second is admitted by the profile and cannot be weighted: it belongs to no zone, so no zone's controls describe it. That is a bound on what the weighting can answer. This module counts it, so the bound is stated rather than left to be inferred from a column of nulls.

check_control_geography_coverage

check_control_geography_coverage(
    seed: pl.DataFrame,
    *,
    profile: str | None,
    geo_col: str = "ctrl_geoid",
    max_unplaceable_share: float
) -> GeographyCoverage

Count the profile's households the control geography cannot place.

No new geography test is applied: geo_col is the point-in-polygon result the crosswalk already assigned, and a null there is the definition of outside.

Parameters:

Name	Type	Description	Default
`seed`	`pl.DataFrame`	The profile's seed, one row per household, carrying geo_col.	required
`profile`	`str \| None`	Usability profile being fitted, or None.	required
`geo_col`	`str`	Balancing-zone column assigned by the crosswalk.	`'ctrl_geoid'`
`max_unplaceable_share`	`float`	Largest share of the universe that may be unplaceable before this is treated as a misconfigured geography rather than a boundary effect.	required

Returns:

Type	Description
`GeographyCoverage`	The coverage counts, for the log and the diagnostics report.

Raises:

Type	Description
`ValueError`	If the unplaceable share exceeds max_unplaceable_share.

processing.weighting.validation.weight_checks

Post-balancing weight sanity checks.

Compares survey weighted totals against PUMS-derived control targets and verifies weight consistency across the table hierarchy.

weight_sanity_checks

weight_sanity_checks(
    tables: dict[str, pl.DataFrame],
    control_totals: ControlTotals,
    specs: list[ControlSpec],
    usability_flag_col: str,
    *,
    geo_col: str = "ctrl_geoid",
    profile: str | None = None
) -> None

Run weight sanity checks: log the fit to the controls, raise on a broken identity.

Parameters:

Name	Type	Description	Default
`tables`	`dict[str, pl.DataFrame]`	Weighted canonical tables.	required
`control_totals`	`ControlTotals`	The targets weights were balanced to.	required
`specs`	`list[ControlSpec]`	Control specifications, for picking a household- and a person-level control to compare against.	required
`usability_flag_col`	`str`	The flag the weighting was run under. The hierarchy identities only hold over the records that carried weight, so the checks have to read the same column the propagation did -- checking a different universe than was weighted would fail on correct output.	required
`geo_col`	`str`	Geography column the control totals are keyed on.	`'ctrl_geoid'`
`profile`	`str \| None`	Usability profile whose columns to check, or None for the un-suffixed set. Names the columns only; usability_flag_col is still what decides the universe.	`None`

check_weight_placement

check_weight_placement(
    tables: dict[str, pl.DataFrame],
    usability_flag_col: str,
    profile: str | None = None,
    *,
    supplied_totals: dict[str, float] | None = None
) -> None

Verify every weight sits on a record its consumer will receive, or raise.

Three things, each a way weight has gone missing before:

No weight on a record the profile does not admit. The propagation zeroes those, so a non-zero here means a table the usability pass never reached.
Each supplied table still sums to what was joined, so redistributing onto the usable records moved the vendor's total rather than shrinking it. Checked only where the caller supplies totals (the vendor path).
What the formatter will keep carries the whole total. [keep_records] [processing.completeness.kept_records.keep_records] is the formatter's own answer, so running it here and comparing totals proves the weighting and the formatter agree before any file is written. Skipped only when a table present carries no verdict, which is a partial call rather than a run.

Parameters:

Name	Type	Description	Default
`tables`	`dict[str, pl.DataFrame]`	Weighted canonical tables.	required
`usability_flag_col`	`str`	The flag the weighting was gated on.	required
`profile`	`str \| None`	Usability profile whose weight columns to read, or None for the un-suffixed set.	`None`
`supplied_totals`	`dict[str, float] \| None`	table -> the total that was joined from a weight file, for the tables that had one.	`None`

Raises:

Type	Description
`ValueError`	On the first table that fails any of the three.

check_persons_carry_their_household

check_persons_carry_their_household(
    tables: dict[str, pl.DataFrame], profile: str | None = None
) -> None

Raise unless household weight x number of persons = the sum of its persons.

A household stands for every one of its persons, so its persons together carry its weight once each. The supplied-weights path keeps a dropped person's share inside the household, so the identity is exact; a dropped household carries 0, and so must its persons.

Parameters:

Name	Type	Description	Default
`tables`	`dict[str, pl.DataFrame]`	Weighted canonical tables.	required
`profile`	`str \| None`	Usability profile whose weight columns to read, or None for the un-suffixed set.	`None`

Raises:

Type	Description
`ValueError`	Naming the households whose persons do not sum to them.

check_days_carry_their_person

check_days_carry_their_person(
    tables: dict[str, pl.DataFrame], profile: str | None = None
) -> None

Raise unless every person's days sum to that person's weight.

The supplied-weights path gives each day its person's final weight, in proportion to the supplier's day weights, so the identity is exact and has no exemptions: a person carrying weight that no day carries -- no usable day the supplier weighted, or no day at all -- is weight the day table has lost. Holding it person by person also holds the day total to the person total.

Parameters:

Name	Type	Description	Default
`tables`	`dict[str, pl.DataFrame]`	Weighted canonical tables.	required
`profile`	`str \| None`	Usability profile whose weight columns to read, or None for the un-suffixed set.	`None`

Raises:

Type	Description
`ValueError`	Naming the persons whose days do not sum to them.