Skip to content

processing.formatting.ctramp

CT-RAMP Formatting Module.

Transforms canonical survey data into CT-RAMP model format.

processing.formatting.ctramp.ctramp_config

Configuration model for CT-RAMP formatting.

CTRAMPConfig pydantic-model

Configuration for CT-RAMP formatting.

Attributes:

Name Type Description
income_low_threshold int

Income threshold for low income bracket
income_med_threshold int

Income threshold for medium income bracket
income_high_threshold int

Income threshold for high income bracket
age_adult int

Age threshold for adult vs child in joint tour composition
income_survey_year_to_ctramp_year float

Factor converting survey-year dollars to CT-RAMP-year dollars
gender_default_for_missing str

Default gender for missing/non-binary

Config:

  • extra: forbid

Fields:

Validators:

income_low_threshold pydantic-field 

income_low_threshold: int

Income threshold for low income bracket (in dollars). Income below this is classified as 'work_low'. Example: 60000 for $60k annual income

income_med_threshold pydantic-field 

income_med_threshold: int

Income threshold for medium income bracket (in dollars). Household income below this threshold (but above low) is classified as 'work_med'. Example: 150000 for $150k annual income

income_high_threshold pydantic-field 

income_high_threshold: int

Income threshold for high income bracket (in dollars). Household income below this threshold (but above med) is classified as 'work_high'. Above this is 'work_very high'. Example: 240000 for $240k annual income

taz_field pydantic-field 

taz_field: str

The field name in the household data that contains the TAZ ID for CTRAMP formatting.

usability_profile pydantic-field 

usability_profile: str

Which usability profile stamped by cascade_completeness decides the record universe. Required, and must name a profile that run stamped: a looser profile admits more, at the cost of no longer matching whichever profile the weighting and the other formatters were given.

age_adult pydantic-field 

age_adult: int

Age category threshold for adult vs child in joint tours. Use AgeCategory enum value (e.g., 4 = AGE_18_TO_24 and higher are adults).

income_survey_year_to_ctramp_year pydantic-field 

income_survey_year_to_ctramp_year: float

Factor to convert survey-year dollars to CT-RAMP-year dollars. Survey income is multiplied by this factor. For BATS 2023 -> CT-RAMP year 2000 the factor is 1 / 1.88 ~= 0.532 (2000 -> 2023 inflation is 1.88) per https://github.com/BayAreaMetro/modeling-website/wiki/InflationAssumptions

gender_default_for_missing pydantic-field 

gender_default_for_missing: str

Gender ('m' or 'f') given to missing/non-binary values, since CT-RAMP requires binary gender.

fixed_location_buffer_meters pydantic-field 

fixed_location_buffer_meters: float = 200.0

Spatial buffer in meters for matching fixed work/school locations to trip destinations when calculating distances.

distance_unit pydantic-field 

distance_unit: str = 'miles'

Distance unit for CT-RAMP formatting ('miles', 'kilometers', or 'meters').

model_config class-attribute instance-attribute 

model_config = ConfigDict(extra='forbid')

validate_income_ordering pydantic-validator 

validate_income_ordering() -> CTRAMPConfig

Validate that income thresholds are in proper order.

processing.formatting.ctramp.day_ids

CT-RAMP ids that carry the survey day.

CT-RAMP has no day dimension, so a multi-day respondent becomes one household and person record per day, with the day encoded into hh_id and person_id.

incorporate_day_into_ids 

incorporate_day_into_ids(
    households: pl.DataFrame,
    persons: pl.DataFrame,
    tours: pl.DataFrame,
    linked_trips: pl.DataFrame,
    joint_trips: pl.DataFrame,
    days: pl.DataFrame,
) -> tuple[
    pl.DataFrame,
    pl.DataFrame,
    pl.DataFrame,
    pl.DataFrame,
    pl.DataFrame,
]

Expand records to person-day level and encode the survey day into CTRAMP IDs.

CTRAMP treats each model run as a single day. Since BATS respondents have multiple survey days, each person-day must appear as a distinct "household" and "person" in the CTRAMP input files so tours and trips can be attributed to the correct day without double-counting.

ID encoding (preserves the canonical hierarchical structure): - CTRAMP hh_id = hh_id * 100 + day_num (unique per household-day) - CTRAMP person_id = day_id (unique per person-day)

Both formulas are invertible: the original IDs can always be recovered from the CTRAMP IDs.

Parameters:

Name Type Description Default
households pl.DataFrame

Canonical households (one row per household).

 required
persons pl.DataFrame

Canonical persons (one row per person).

 required
tours pl.DataFrame

Canonical tours (one row per tour, already has day_id).

 required
linked_trips pl.DataFrame

Canonical linked trips (one row per trip, already has day_id).

 required
joint_trips pl.DataFrame

Canonical joint trips (one row per joint trip, with the household's travel_date).

 required
days pl.DataFrame

Canonical person-days table with person_id, hh_id, day_id columns.

 required

Returns:

Type Description
pl.DataFrame

Tuple of (households, persons, tours, linked_trips, joint_trips) where
pl.DataFrame

each table is expanded to the person-day level and hh_id / person_id
pl.DataFrame

reflect the day-encoded IDs.

processing.formatting.ctramp.format_ao

Auto Ownership Results formatting for CT-RAMP.

Produces an ao_results_ctramp table matching the schema of the aoResults.csv file written by the CT-RAMP Java model::

HHID, AO

AO is the number of vehicles owned by the household (0-4+).

format_ao_results 

format_ao_results(households_ctramp: pl.DataFrame) -> pl.DataFrame

Derive aoResults from the already-formatted households table.

Parameters:

Name Type Description Default
households_ctramp pl.DataFrame

Formatted CT-RAMP households table, expected to have hh_id and autos.

 required

Returns:

Type Description
pl.DataFrame

DataFrame with columns HHID, AO.

processing.formatting.ctramp.format_cdap

CDAP Results formatting for CT-RAMP.

Produces a cdap_results_ctramp table matching the schema of the cdapResults.csv file written by the CT-RAMP Java model::

HHID, PersonID, PersonNum, PersonType, ActivityString

PersonType is an integer (1-8) and ActivityString is H/M/N.

format_cdap_results 

format_cdap_results(persons_ctramp: pl.DataFrame) -> pl.DataFrame

Derive cdapResults from the already-formatted persons table.

Parameters:

Name Type Description Default
persons_ctramp pl.DataFrame

Formatted CT-RAMP persons table, expected to have hh_id, person_id, person_num, person_type (int), and activity_pattern (H/M/N).

 required

Returns:

Type Description
pl.DataFrame

DataFrame with columns HHID, PersonID, PersonNum,
pl.DataFrame

PersonType (int), ActivityString (str).

processing.formatting.ctramp.format_ctramp

CT-RAMP Formatting Step.

Transforms canonical survey data (persons, households, tours, trips) into CT-RAMP (Coordinated Travel - Regional Activity Modeling Platform) format for use with activity-based travel demand models. See CTRAMP Data Models.

This module orchestrates the transformation of households, persons, tours, and trips from canonical format into eleven CT-RAMP tables, handling missing data and providing configurable income thresholds. Which records it writes is the usability_profile's choice, not this step's.

Components

Data Flow

Key Dependencies

  1. households_ctramp must be created first (needed by all downstream formatters)
  2. individual_tours_ctramp must be created before persons_ctramp (provides tour statistics)
  3. persons_ctramp and trips use already-formatted tour/household data
  4. Joint tables can be processed independently from individual tables

Mermaid Diagram

flowchart TD
    %% Canonical inputs
    subgraph inputs["Canonical Inputs"]
        direction TB
        hh_canon[("households")]
        per_canon[("persons")]
        tours_canon[("tours")]
        trips_canon[("linked_trips")]
        joint_trips_canon[("joint_trips")]
    end

    %% Stage 1
    subgraph s1["Stage 1: Households"]
        direction LR
        fmt_hh["format_households"]
        hh_ctramp{{"households_ctramp"}}
        fmt_hh --> hh_ctramp
    end

    %% Stage 2
    subgraph s2["Stage 2: Tours"]
        direction TB
        fmt_ind_tour["format_individual_tour"]
        ind_tour_ctramp{{"individual_tours_ctramp"}}
        fmt_ind_tour --> ind_tour_ctramp

        fmt_joint_tour["format_joint_tour"]
        joint_tour_ctramp{{"joint_tours_ctramp"}}
        fmt_joint_tour --> joint_tour_ctramp
    end

    %% Stage 3
    subgraph s3["Stage 3: Persons"]
        direction LR
        fmt_persons["format_persons"]
        per_ctramp{{"persons_ctramp"}}
        fmt_persons --> per_ctramp
    end

    %% Stage 4
    subgraph s4["Stage 4: Locations & Trips"]
        direction TB
        fmt_mand_loc["format_mandatory_location"]
        mand_loc_ctramp{{"mandatory_locations_ctramp"}}
        fmt_mand_loc --> mand_loc_ctramp

        fmt_ind_trip["format_individual_trip"]
        ind_trip_ctramp{{"individual_trips_ctramp"}}
        fmt_ind_trip --> ind_trip_ctramp

        fmt_joint_trip["format_joint_trip"]
        joint_trip_ctramp{{"joint_trips_ctramp"}}
        fmt_joint_trip --> joint_trip_ctramp
    end

    %% Canonical inputs to stages (consolidated)
    inputs -.-> s1
    inputs -.-> s2
    inputs -.-> s3
    inputs -.-> s4

    %% Stage ordering (vertical layout)
    s1 ~~~ s2
    s2 ~~~ s3
    s3 ~~~ s4

    %% Formatted table dependencies (thick solid)
    hh_ctramp ==> fmt_ind_tour
    hh_ctramp ==> fmt_joint_tour
    ind_tour_ctramp ==> fmt_persons
    hh_ctramp ==> fmt_mand_loc
    hh_ctramp ==> fmt_ind_trip
    ind_tour_ctramp ==> fmt_ind_trip
    hh_ctramp ==> fmt_joint_trip

    %% Styling
    classDef canonClass fill:#e1f5ff,stroke:#0277bd,stroke-width:2px
    classDef formatterClass fill:#fff3e0,stroke:#ef6c00,stroke-width:2px
    classDef outputClass fill:#e8f5e9,stroke:#2e7d32,stroke-width:3px

    class hh_canon,per_canon,tours_canon,trips_canon,joint_trips_canon canonClass
    class fmt_hh,fmt_ind_tour,fmt_persons,fmt_mand_loc,fmt_ind_trip formatterClass
    class fmt_joint_tour,fmt_joint_trip formatterClass
    class hh_ctramp,per_ctramp,ind_tour_ctramp,ind_trip_ctramp outputClass
    class mand_loc_ctramp,joint_tour_ctramp,joint_trip_ctramp outputClass

Legend:

  • 🔵 Blue cylinders: Canonical input tables
  • 🟠 Orange boxes: Formatter functions
  • 🟢 Green hexagons: CT-RAMP output tables (all formatted)
  • Dashed arrows (⋯→): Canonical inputs available to formatters (see component details above for specifics)
  • Thick solid arrows (⟹): Formatted table dependencies (execution order)

Note: Canonical input details for each formatter are listed in the Components section above.

Execution Stages

  1. Record selection: keeps the records the usability_profile admits (usable_<profile>); a profile's zone_coverage leaves out records with no TAZ
  2. Households: Formatted first to establish income brackets needed downstream
  3. Tours: Formatted before persons to provide tour frequency statistics
  4. Persons: Combines tour-derived fields (activity patterns, frequencies) with demographic characteristics
  5. Mandatory Locations: Uses both canonical (TAZ) and formatted (income) household data
  6. Trips: Formatted last, linking to already-formatted tours

Configuration

All thresholds and defaults are managed via CTRAMPConfig

Implementation Notes

Excluded Fields
  • Random Number Fields: Excluded because they are simulation-specific and not derivable from survey data (household: ao_rn, fp_rn, cdap_rn; tour: imtf_rn, imtod_rn, immc_rn, jtf_rn, jtl_rn, jtod_rn, etc.)
  • Model Output Fields: auto_suff (auto sufficiency), walk_subzone (walk-to-transit accessibility), wait times and logsums
Placeholder Values

When tour data is unavailable or incomplete, person-level tour frequency fields use these defaults:

  • activity_pattern: 'H' (home all day)
  • imf_choice: 0 (no mandatory tours)
  • inmf_choice: 0 (no non-mandatory tours)
  • wfh_choice: 0 (no work from home)
  • jtf_choice: Derived from joint tour data if available; otherwise NONE_NONE (-4)
Empty Data Handling

The module gracefully handles missing data:

  • Empty tour DataFrames result in placeholder person fields
  • Missing joint tour IDs default to individual tour treatment
  • Missing or null values use sensible defaults per field type

MODEL_MAP module-attribute 

MODEL_MAP = {
    "households_ctramp": HouseholdCTRAMPModel,
    "persons_ctramp": PersonCTRAMPModel,
    "mandatory_locations_ctramp": MandatoryLocationCTRAMPModel,
    "individual_trips_ctramp": IndividualTripCTRAMPModel,
    "individual_tours_ctramp": IndividualTourCTRAMPModel,
    "joint_trips_ctramp": JointTripCTRAMPModel,
    "joint_tours_ctramp": JointTourCTRAMPModel,
    "cdap_results_ctramp": CDAPResultsCTRAMPModel,
    "ao_results_ctramp": AOResultsCTRAMPModel,
    "all_tours_ctramp": AllTourCTRAMPModel,
    "all_trips_ctramp": AllTripCTRAMPModel,
}

format_ctramp 

format_ctramp(
    persons: pl.DataFrame,
    households: pl.DataFrame,
    unlinked_trips: pl.DataFrame,
    linked_trips: pl.DataFrame,
    tours: pl.DataFrame,
    joint_trips: pl.DataFrame,
    joint_tours: pl.DataFrame,
    days: pl.DataFrame,
    ctramp: CTRAMPConfig,
    habitual_locations: pl.DataFrame | None = None,
    habitual_location_days: pl.DataFrame | None = None,
) -> dict[str, pl.DataFrame]

Format canonical survey data to CT-RAMP model specification.

Transforms person, household, tour, and trip data from canonical format to CT-RAMP format required by the activity-based travel demand model. See module docstring for complete component descriptions and data flow.

Parameters:

Name Type Description Default
persons pl.DataFrame

Canonical person data with demographic fields. Required columns: person_id, hh_id, person_num, age, gender, employment, student, school_type, commute subsidies.

 required
households pl.DataFrame

Canonical household data with income and dwelling fields. Required columns: hh_id, home_lat, home_lon, income_bin, num_vehicles.

 required
linked_trips pl.DataFrame

Canonical linked trip data. Required columns: linked_trip_id, tour_id, o_purpose_category, d_purpose_category, mode_type, o_taz, d_taz, tour_direction, times, person_id, hh_id.

 required
tours pl.DataFrame

Canonical tour data. Required columns: tour_id, hh_id, subtour_num, parent_tour_id, joint_tour_id, tour_purpose, tour_category, tour_mode, origin_depart_time, origin_arrive_time, and the zone columns named by taz_field.

 required
joint_tours pl.DataFrame

Aggregated joint tour data carrying joint_tour_weight. May be empty but must be provided.

 required
unlinked_trips pl.DataFrame

Canonical unlinked trips used to derive detailed transit submodes for formatted tours and trips. May be empty but must be provided.

 required
joint_trips pl.DataFrame

Aggregated joint trip data. Required columns: joint_trip_id, hh_id, num_joint_travelers.

 required
days pl.DataFrame

Canonical person-days data. Required columns: person_id, hh_id, day_id. Used to expand persons and households to the person-day level and encode the survey day into CTRAMP hh_id / person_id so that multi-day respondents produce one distinct CTRAMP record per day rather than one record that conflates all survey days.

 required
ctramp CTRAMPConfig

Formatting settings: the usability profile deciding the record universe, the income brackets and dollar-year factor, the TAZ field, the adult age and the gender for missing values. See CTRAMPConfig.

 required
habitual_locations pl.DataFrame | None

Habitual locations, zoned by add_zone_ids. An employed person who reported no workplace takes their most-visited observed workplace as WorkLocation (see usual_workplace). None leaves the reported workplaces alone.

 None
habitual_location_days pl.DataFrame | None

Time spent at each habitual location per day, which picks the most-visited workplace.

 None

Returns:

Type Description
dict[str, pl.DataFrame]

Dictionary with keys: - households_ctramp: Formatted household data (7 core fields including income, TAZ, size) - persons_ctramp: Formatted person data (20+ fields including person type, activity patterns, tour frequencies) - mandatory_locations_ctramp: Mandatory location data (work/school location records) - individual_tours_ctramp: Individual tour data (tour-level attributes: purpose, mode, time, stops) - individual_trips_ctramp: Individual trip data (trip-level attributes for individual tours) - joint_tours_ctramp: Joint tour data (joint tours with composition and participants) - joint_trips_ctramp: Joint trip data (trip-level attributes for joint tours) - all_tours_ctramp, all_trips_ctramp: individual and joint, together - cdap_results_ctramp, ao_results_ctramp: CDAP and auto ownership results

Example

from processing.formatting.ctramp import format_ctramp

result = format_ctramp(
    persons=canonical_persons,
    households=canonical_households,
    unlinked_trips=canonical_unlinked_trips,
    linked_trips=canonical_linked_trips,
    tours=canonical_tours,
    joint_trips=canonical_joint_trips,
    joint_tours=canonical_joint_tours,
    days=canonical_days,
    ctramp=CTRAMPConfig(
        usability_profile="ctramp",  # the profile deciding the record universe
        income_low_threshold=30000,  # $30k divides low from medium ($2000)
        income_med_threshold=60000,  # $60k divides medium from high ($2000)
        income_high_threshold=100000,  # $100k divides high from very high ($2000)
        income_survey_year_to_ctramp_year=0.5319148936,  # 1/1.88: 2023 to $2000
        taz_field="taz",
        age_adult=4,
        gender_default_for_missing="f",
    ),
)

# Access formatted tables
households_ctramp = result["households_ctramp"]
persons_ctramp = result["persons_ctramp"]
mandatory_locations_ctramp = result["mandatory_locations_ctramp"]
individual_tours_ctramp = result["individual_tours_ctramp"]
joint_tours_ctramp = result["joint_tours_ctramp"]
individual_trips_ctramp = result["individual_trips_ctramp"]
joint_trips_ctramp = result["joint_trips_ctramp"]

processing.formatting.ctramp.format_households

Household formatting for CT-RAMP.

Transforms canonical household data into CT-RAMP model format, including:

  • Income Conversion: Converts categorical income values to midpoint dollars in configurable base year
  • Income Bracketing: Classifies households into low/med/high/very high income brackets using user-defined thresholds
  • Spatial Fields: Maps home TAZ (optionally filters households without valid TAZ)
  • Joint Tour Frequency: Derives jtf_choice field from joint tour counts by purpose category
  • Excludes: Random number fields (ao_rn, fp_rn, cdap_rn) and auto_suff (model simulation fields)

Note: Model-output fields (walk_subzone, humanVehicles, autonomousVehicles, random number fields, auto_suff) are excluded as they are not derivable from survey data.

format_households 

format_households(
    households_canonical: pl.DataFrame,
    persons_canonical: pl.DataFrame,
    tours_canonical: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format household data to CT-RAMP specification.

Transforms household data from canonical format to HouseholdCTRAMPModel Key transformations:

  • Rename fields to CT-RAMP conventions
  • Convert income categories to midpoint values
  • Compute household aggregates (size, workers, vehicles)
  • Map TAZ using configuration

Parameters:

Name Type Description Default
households_canonical pl.DataFrame

Canonical households DataFrame with hh_id, home_taz, income_detailed, income_followup, num_vehicles

 required
persons_canonical pl.DataFrame

Canonical persons DataFrame with hh_id, employment (for computing household size and worker count)

 required
tours_canonical pl.DataFrame

Canonical tours DataFrame with hh_id, joint_tour_id, tour_purpose (for computing joint tour frequency)

 required
config CTRAMPConfig

CTRAMPConfig instance with configuration parameters

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP household fields:

  • hh_id: Household ID
  • taz: Home TAZ
  • income: Annual household income (midpoint value)
  • autos: Number of automobiles
  • size: Number of persons
  • workers: Number of workers
  • jtf_choice: Joint tour frequency (count of unique joint tours)
Notes
  • Model-output fields (walk_subzone, humanVehicles, autonomousVehicles, random number fields, auto_suff) are excluded as they are not derivable from survey data

processing.formatting.ctramp.format_joint_tours

Format canonical joint tours for CT-RAMP.

format_joint_tour 

format_joint_tour(
    tours_canonical: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    joint_tours_canonical: pl.DataFrame,
    persons_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format canonical joint tours to CT-RAMP's household-level schema.

ctramp_joint_tour_numbers 

ctramp_joint_tour_numbers(
    tours_canonical: pl.DataFrame,
) -> pl.DataFrame

Number each household's joint tours from zero, as CT-RAMP identifies them.

Both joint files carry this number, so it is derived once from the tours -- the only table holding every admitted joint tour. Ranking it separately over each file's own rows agrees only while the two files cover the same joint tours; let one lose a tour and the rest silently shift down onto their neighbours' numbers, which no referential check can see because the number landed on does exist.

Parameters:

Name Type Description Default
tours_canonical pl.DataFrame

Canonical tours.

 required

Returns:

Type Description
pl.DataFrame

One row per joint tour: joint_tour_id and its 0-based
pl.DataFrame

_ctramp_joint_tour_id within the household.

joint_weight_columns 

joint_weight_columns(
    available: Collection[str],
    weight_col: str,
    member_count: pl.Expr,
) -> list[pl.Expr]

Carry a joint weight through, expressed as a per-member sample rate.

A joint weight is the SUM over its members, so its inverse is not a sample rate: CT-RAMP re-applies the party multiplier itself (num_participants/sampleRate), and inverting the sum would count the party twice. The per-member rate is the inverse of the member mean, i.e. n_members / weight.

The count comes from the member table the caller already grouped, not from the joint record. It matches what the weighting summed over because the CT-RAMP filters drop an unusable member's tour and its trips upstream, then demote any group left with fewer than two participants -- not because of the joint record's own flag, and not because of the zero-weight filter, which only ever removes whole households.

Parameters:

Name Type Description Default
available Collection[str]

Columns present on the frame the expressions will run against.

 required
weight_col str

The joint weight column to carry through.

 required
member_count pl.Expr

Expression giving the number of members behind the weight.

 required

Returns:

Type Description
list[pl.Expr]

The weight and sampleRate expressions, or empty when there is no
list[pl.Expr]

weight to carry.

processing.formatting.ctramp.format_joint_trips

Joint-trip formatting for CT-RAMP.

format_joint_trip 

format_joint_trip(
    joint_trips_canonical: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    tours_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format joint trips to CT-RAMP specification.

Transforms joint trip data using mean coordinates from joint_trips table. Links trips to joint tours using aggregated joint trip data. Maintains participant information for each trip leg.

Parameters:

Name Type Description Default
joint_trips_canonical pl.DataFrame

Aggregated joint trip DataFrame with joint_trip_id, hh_id, num_joint_travelers (mean locations)

 required
linked_trips_canonical pl.DataFrame

Canonical linked trips DataFrame to bridge joint_trip_id to tour_id (contains joint_trip_id, tour_id, o_purpose_category, d_purpose_category, mode_type, depart_time, arrive_time, num_travelers, o_taz, d_taz)

 required
tours_canonical pl.DataFrame

Canonical tours DataFrame with tour_id, joint_tour_id, tour_purpose, tour_category, tour_mode

 required
households_ctramp pl.DataFrame

Formatted CT-RAMP households DataFrame with hh_id, income

 required
config CTRAMPConfig

CT-RAMP configuration with income thresholds

 required
unlinked_trips_canonical pl.DataFrame

Canonical unlinked trips DataFrame with linked_trip_id, mode_type, access_mode, egress_mode, transit_submode, tnc_type; may be empty but must be provided

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP joint trip fields
Notes
  • Uses mean coordinates from joint_trips aggregation table
  • Deduplicates to one row per joint_trip_id

processing.formatting.ctramp.format_mandatory_location

Format mandatory locations for CT-RAMP specification.

calc_fixed_location_distances 

calc_fixed_location_distances(
    mandatory_locations: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    fixed_type: str,
    config: CTRAMPConfig,
) -> pl.DataFrame

Calculate fixed work/school location distances for validation.

Parameters:

Name Type Description Default
mandatory_locations pl.DataFrame

DataFrame with person fixed locations and home locations

 required
linked_trips_canonical pl.DataFrame

Canonical linked trips DataFrame with trip origins/destinations

 required
fixed_type str

Type of fixed location to calculate distances for ("work" or "school")

 required
config CTRAMPConfig

CT-RAMP configuration with fixed location buffer and distance unit

 required

Returns:

Type Description
pl.DataFrame

DataFrame with person_id and calculated work/school distances

format_mandatory_location 

format_mandatory_location(
    persons_ctramp: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format mandatory locations (work/school) to CT-RAMP specification.

Transforms person and household data to create mandatory location records for persons with work or school locations.

Parameters:

Name Type Description Default
persons_ctramp pl.DataFrame

Formatted CT-RAMP persons DataFrame with person_id, hh_id, person_num, person_type (integer code), age (non-binned continuous), employment_category, student_category, work_taz, school_taz

 required
households_ctramp pl.DataFrame

Formatted CT-RAMP households DataFrame with hh_id, income, home_taz (for HomeTAZ field)

 required
linked_trips_canonical pl.DataFrame

Canonical linked trips DataFrame for distance calculations

 required
config CTRAMPConfig

CT-RAMP configuration

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP mandatory location fields: - HHID, PersonID, PersonNum - HomeTAZ, Income - PersonType, PersonAge - EmploymentCategory, StudentCategory - WorkLocation, SchoolLocation
Notes
  • Excludes model-only fields (walk subzones)
  • Includes ALL persons; WorkLocation and SchoolLocation default to 0 when not available
  • Uses pre-computed employment_category and student_category from persons_ctramp

processing.formatting.ctramp.format_persons

Person formatting for CT-RAMP.

Transforms canonical person data into CT-RAMP model format, including:

  • Person Type Classification: Derives CT-RAMP person type from age category, employment status, and student status
  • Gender Mapping: Converts to binary m/f format (configurable default for non-binary/missing)
  • Free Parking: Determines eligibility based on commute subsidies
  • Tour Frequency Fields: Derives imf_choice (mandatory tour frequency), inmf_choice (non-mandatory tour frequency), and activity_pattern from tour data when available
  • Work-from-Home: Sets wfh_choice based on work tours vs work days
  • Value of Time: Left null. It is a model output, not survey data, and is kept only if the input already carries it
  • Defaults: A person with no tours gets activity_pattern 'H' and zero tour frequencies

activity_pattern, imf_choice and inmf_choice are derived from the person's formatted tours.

enrich_persons_with_person_type 

enrich_persons_with_person_type(
    persons_canonical: pl.DataFrame,
) -> pl.DataFrame

Enrich persons DataFrame with person_type and type fields.

Derives person_type (integer code) and type (string label) based on age, employment, and student status using the ctramp_person_type_expression.

If student_category and employment_category columns are present, they will be used as inputs to person_type classification for consistency. Otherwise, classification falls back to raw employment/student/school_type columns.

Parameters:

Name Type Description Default
persons_canonical pl.DataFrame

Canonical persons DataFrame

 required

Returns:

Type Description
pl.DataFrame

DataFrame with added person_type and type fields

format_persons 

format_persons(
    persons_canonical: pl.DataFrame,
    tours_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format person data to CT-RAMP specification.

Transforms person data from canonical format to CT-RAMP format. Key transformations:

  • Classify person type based on age, employment, student status
  • Map gender to m/f format
  • Determine free parking eligibility
  • Aggregate activity patterns and tour frequencies from tour data

Parameters:

Name Type Description Default
persons_canonical pl.DataFrame

Canonical persons DataFrame with derived person_type field commute_subsidy_provide_free_parking (employer provides free parking), commute_subsidy_provide_discounted_parking (employer provides discounted parking), commute_subsidy_use_free_parking (respondent uses free parking), commute_subsidy_use_discounted_parking (respondent uses discounted parking), value_of_time

 required
tours_ctramp pl.DataFrame

Formatted CT-RAMP tours DataFrame with person_id and tour_purpose (CTRAMP-formatted purpose strings like 'work_low', 'school_grade', etc.)

 required
config CTRAMPConfig

CT-RAMP configuration with age thresholds

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP person fields:

  • hh_id: Household ID
  • person_id: Person ID
  • person_num: Person number
  • age: Person age
  • gender: Gender (m/f)
  • type: Person type (1-8)
  • value_of_time: Value of time ($/hour); null unless supplied
  • fp_choice: Free parking choice (1/2)
  • activity_pattern: Daily activity pattern (M/N/H)
  • imf_choice: Individual mandatory tour frequency
  • inmf_choice: Individual non-mandatory tour frequency
  • wfh_choice: Work from home choice (0/1)
Notes
  • activity_pattern: M=mandatory tours, N=non-mandatory only, H=no tours
  • imf_choice: Count of mandatory tours (work/school)
  • inmf_choice: Count of non-mandatory tours
  • wfh_choice: 1 if employed, job_type=WFH, and no work tours (binary: WFH or commute, not both)
  • employment_category: derived from the BATS employment field via EMPLOYMENT_TO_CTRAMP:
BATS employment value CT-RAMP EmploymentCategory
EMPLOYED_FULLTIME Full-time worker
EMPLOYED_SELF Full-time worker (self-employed treated as full-time)
EMPLOYED_PARTTIME Part-time worker
EMPLOYED_UNPAID Part-time worker (unpaid work treated as part-time)
all others (not employed, retired, student, etc.) Not employed

Children classified as CHILD_UNDER_5 or STUDENT_NON_DRIVING_AGE by person type are overridden to Under age 16 regardless of reported employment.

debug_ptype 

debug_ptype(persons_ctramp: pl.DataFrame) -> None

Log person type distributions and detect bad attribute combinations.

Prints a frequency table of person_type by student_category by employment_category by age_bin for debugging. Then checks for known bad combinations and logs WARNING-level messages with counts.

processing.formatting.ctramp.format_tours

Tour formatting for CT-RAMP.

Transforms canonical tour data into CT-RAMP model format, including:

  • Individual tours (non-joint tours)
  • Joint tours (household member group tours)

TWO_SUBTOURS module-attribute 

TWO_SUBTOURS = 2

format_joint_tour 

format_joint_tour(
    tours_canonical: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    joint_tours_canonical: pl.DataFrame,
    persons_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format canonical joint tours to CT-RAMP's household-level schema.

format_individual_tour 

format_individual_tour(
    tours_canonical: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    persons_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
    *,
    include_joint: bool = False
) -> pl.DataFrame

Format individual tours to CT-RAMP specification.

Transforms tour data (excluding joint tours) to CT-RAMP format. Key Transformations:

  • Filtering: Excludes joint tours (processes only tours with joint_tour_id IS NULL)
  • Purpose Mapping: Converts canonical purpose categories to CT-RAMP tour purposes
  • Mode Translation: Maps canonical mode types to CT-RAMP mode codes
  • Time-of-Day: Extracts start/end hours from departure/arrival times
  • Stop Counts: Computes outbound and inbound stop counts from linked trips
    • Subtour Frequency: Encodes atWork_freq using CT-RAMP categorical values based on work subtour pattern
  • Excludes: Model simulation fields (random numbers, wait times, logsums)

Parameters:

Name Type Description Default
tours_canonical pl.DataFrame

Canonical tours DataFrame with tour_id, hh_id, person_id, person_num, tour_category, tour_purpose, o_taz, d_taz, origin_depart_time, origin_arrive_time, tour_mode, joint_tour_id (for filtering), parent_tour_id (for subtour counting)

 required
linked_trips_canonical pl.DataFrame

Canonical trips DataFrame with tour_id, tour_direction (1=outbound, 2=inbound, 3=subtour)

 required
unlinked_trips_canonical pl.DataFrame

Unlinked canonical trips used to derive transit submode; may be empty but must be provided

 required
persons_canonical pl.DataFrame

Canonical persons DataFrame with person_id, person_num, person_type (for mode mapping), school_type (for purpose mapping) are optional but re-derived if missing or invalid

 required
households_ctramp pl.DataFrame

Formatted CT-RAMP households DataFrame with hh_id, income

 required
config CTRAMPConfig

CT-RAMP configuration with income thresholds

 required
include_joint bool

Keep joint-tour members as their own per-person tours, producing the unified All* set instead of the individual-only one. Each participant already holds a tour row carrying its own tour_weight, so nothing is exploded or reweighted.

 False

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP individual tour fields:

  • hh_id, person_id, person_num, person_type
  • tour_id, tour_category, tour_purpose
  • orig_taz, dest_taz
  • start_hour, end_hour
  • tour_mode
  • atWork_freq (subtour frequency category)
  • num_ob_stops, num_ib_stops
Notes
  • Excludes joint tours unless include_joint
  • Excludes all model-only fields (random numbers, wait times, logsums)

processing.formatting.ctramp.format_trips

Trip formatting for CT-RAMP.

Transforms canonical trip data into CT-RAMP model format, including:

  • Individual trips (trips on individual tours)
  • Joint trips (trips on joint tours)

format_joint_trip 

format_joint_trip(
    joint_trips_canonical: pl.DataFrame,
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    tours_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format joint trips to CT-RAMP specification.

Transforms joint trip data using mean coordinates from joint_trips table. Links trips to joint tours using aggregated joint trip data. Maintains participant information for each trip leg.

Parameters:

Name Type Description Default
joint_trips_canonical pl.DataFrame

Aggregated joint trip DataFrame with joint_trip_id, hh_id, num_joint_travelers (mean locations)

 required
linked_trips_canonical pl.DataFrame

Canonical linked trips DataFrame to bridge joint_trip_id to tour_id (contains joint_trip_id, tour_id, o_purpose_category, d_purpose_category, mode_type, depart_time, arrive_time, num_travelers, o_taz, d_taz)

 required
tours_canonical pl.DataFrame

Canonical tours DataFrame with tour_id, joint_tour_id, tour_purpose, tour_category, tour_mode

 required
households_ctramp pl.DataFrame

Formatted CT-RAMP households DataFrame with hh_id, income

 required
config CTRAMPConfig

CT-RAMP configuration with income thresholds

 required
unlinked_trips_canonical pl.DataFrame

Canonical unlinked trips DataFrame with linked_trip_id, mode_type, access_mode, egress_mode, transit_submode, tnc_type; may be empty but must be provided

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP joint trip fields
Notes
  • Uses mean coordinates from joint_trips aggregation table
  • Deduplicates to one row per joint_trip_id

format_individual_trip 

format_individual_trip(
    linked_trips_canonical: pl.DataFrame,
    unlinked_trips_canonical: pl.DataFrame,
    tours_ctramp: pl.DataFrame,
    persons_canonical: pl.DataFrame,
    households_ctramp: pl.DataFrame,
    config: CTRAMPConfig,
) -> pl.DataFrame

Format individual trips to CT-RAMP specification.

Transforms linked trip data (for individual tours only) to CT-RAMP format. Links trips to individual tours via tour_id. Includes stop purpose, mode, location, and sequence within tour. Distinguishes outbound, inbound, and subtour trips.

Parameters:

Name Type Description Default
linked_trips_canonical pl.DataFrame

Canonical DataFrame with linked trip fields (tour_id, o_purpose_category, d_purpose_category, mode_type, o_taz, d_taz, tour_direction, depart_time, arrive_time, person_id, hh_id)

 required
tours_ctramp pl.DataFrame

Formatted CT-RAMP individual tours DataFrame (already filtered to individual tours only, without joint_tour_id). Contains tour_id, tour_purpose, tour_mode, tour_category

 required
persons_canonical pl.DataFrame

Canonical persons DataFrame with person_id, person_num, school_type

 required
households_ctramp pl.DataFrame

Formatted CT-RAMP households DataFrame with hh_id, income

 required
config CTRAMPConfig

CT-RAMP configuration with income thresholds

 required
unlinked_trips_canonical pl.DataFrame

Canonical unlinked trips DataFrame with linked_trip_id, mode_type, access_mode, egress_mode, transit_submode, tnc_type; may be empty but must be provided

 required

Returns:

Type Description
pl.DataFrame

DataFrame with CT-RAMP individual trip fields
Notes
  • Excludes trips on joint tours
  • Creates stop_id sequence starting at 1 per half-tour
  • Excludes model-only fields (parking costs, value of time, etc.)

processing.formatting.ctramp.joint_representative

Pick one member to stand for a joint group's location.

CT-RAMP writes a joint tour and its joint trips as single rows, so each needs one origin and one destination. The canonical tables deliberately do not supply them: members of a joint trip were in the same place, and collapsing their coordinates means choosing a rule, which belongs to the consumer rather than the survey.

The rule here is to use a member's own reported location rather than an average of them. An average names a point none of them visited, which then has to be placed in a zone system from scratch -- and near a boundary it can land on the wrong side, or (where zones do not tile the world) nowhere at all. A member's location is already zoned, in every zone system at once, and is somewhere a person actually went.

Which member is settled once per joint tour, not per leg. Choosing per leg lets a household take its outbound leg from one member and its inbound from another; where those two reported home a few metres apart across a boundary, the tour then departs one zone and returns to a different one -- a tour that never gets home. One member for the whole tour cannot produce that.

This lives under the CT-RAMP formatter because it is one consumer's convention, not a fact about the survey: another output may reasonably collapse a group a different way. Should a second consumer want the same rule, move the selection to a shared location rather than importing it from here -- and design it for that at the time, since the part worth sharing is the choice of member (the medoid, with its tie-break), not the columns each format happens to need.

COORD_COLUMNS module-attribute 

COORD_COLUMNS = ('o_lat', 'o_lon', 'd_lat', 'd_lon')

select_representative_members 

select_representative_members(
    joint_members: pl.DataFrame,
) -> pl.DataFrame

Reduce each joint trip to the member trip that stands for it.

The representative is the member sitting closest to the middle of the group, totalled over every leg of the joint tour and over both ends of each leg, so that one person supplies the whole itinerary. Two members are always equidistant from their own midpoint, so a pair -- the majority of joint trips -- is settled entirely by the tie-break, person_id. Members of a joint tour share a household and person_id embeds the person number, so that is the lowest person number in the group.

The member's whole row is returned rather than just its coordinates, so the caller can take the zone that was assigned to that real location instead of re-deriving one.

Parameters:

Name Type Description Default
joint_members pl.DataFrame

Member linked trips carrying joint_trip_id, joint_tour_id, person_id and both ends' coordinates.

 required

Returns:

Type Description
pl.DataFrame

One row per joint_trip_id: the representative member's trip.

representative_person_per_tour 

representative_person_per_tour(
    joint_members: pl.DataFrame,
) -> pl.DataFrame

The person standing for each joint tour.

The joint tour and its joint trips are two views of one outing, so both take their location and clock from the same member. Without this the tour file would describe whichever member happened to sort first, and could place the tour in a different zone from its own outbound trip.

Parameters:

Name Type Description Default
joint_members pl.DataFrame

Member linked trips, as for :func:select_representative_members.

 required

Returns:

Type Description
pl.DataFrame

One row per joint_tour_id with the representative's person_id.

log_members_spanning_zones 

log_members_spanning_zones(
    joint_members: pl.DataFrame, taz_field: str
) -> None

Report joint trips whose members were not all in the same zone.

Two members standing a few metres apart across a boundary really were in different zones, so this is the survey being accurate rather than a fault to repair. It is logged because the joint record reports only its representative's zone, and an analyst comparing that against the member trips should be able to see how often the two can differ instead of discovering it.

processing.formatting.ctramp.mode_mappings

CT-RAMP mode and transit-submode mappings.

TRANSIT_SUBMODE_NONE module-attribute 

TRANSIT_SUBMODE_NONE = 0

TRANSIT_SUBMODE_LOCAL module-attribute 

TRANSIT_SUBMODE_LOCAL = 1

TRANSIT_SUBMODE_LR module-attribute 

TRANSIT_SUBMODE_LR = 2

TRANSIT_SUBMODE_FERRY module-attribute 

TRANSIT_SUBMODE_FERRY = 3

TRANSIT_SUBMODE_EXPRESS module-attribute 

TRANSIT_SUBMODE_EXPRESS = 4

TRANSIT_SUBMODE_HEAVY module-attribute 

TRANSIT_SUBMODE_HEAVY = 5

TRANSIT_SUBMODE_COMMUTER module-attribute 

TRANSIT_SUBMODE_COMMUTER = 6

MODE_TO_TRANSIT_SUBMODE module-attribute 

MODE_TO_TRANSIT_SUBMODE = {
    Mode.BUS_LOCAL.value: TRANSIT_SUBMODE_LOCAL,
    Mode.BUS_LOCAL_PUBLIC.value: TRANSIT_SUBMODE_LOCAL,
    Mode.BUS_OTHER.value: TRANSIT_SUBMODE_LOCAL,
    Mode.PARATRANSIT.value: TRANSIT_SUBMODE_LOCAL,
    Mode.LIGHT_RAIL.value: TRANSIT_SUBMODE_LR,
    Mode.MUNI_METRO.value: TRANSIT_SUBMODE_LR,
    Mode.STREETCAR.value: TRANSIT_SUBMODE_LR,
    Mode.FERRY.value: TRANSIT_SUBMODE_FERRY,
    Mode.WATER.value: TRANSIT_SUBMODE_FERRY,
    Mode.BOAT.value: TRANSIT_SUBMODE_FERRY,
    Mode.BUS_EXPRESS.value: TRANSIT_SUBMODE_EXPRESS,
    Mode.BUS_BRT.value: TRANSIT_SUBMODE_EXPRESS,
    Mode.BART.value: TRANSIT_SUBMODE_HEAVY,
    Mode.RAIL.value: TRANSIT_SUBMODE_COMMUTER,
    Mode.RAIL_OTHER.value: TRANSIT_SUBMODE_COMMUTER,
    Mode.RAIL_INTERCITY.value: TRANSIT_SUBMODE_COMMUTER,
}

aggregate_transit_submode 

aggregate_transit_submode(
    unlinked_trips: pl.DataFrame, group_col: str
) -> pl.DataFrame

Aggregate the highest transit submode and the TNC type within each group.

Uses the detailed mode_1-mode_4 columns on unlinked trips to detect the transit submode (local bus, light rail, ferry, express bus, heavy rail, commuter rail) and returns the highest-ranked submode present per group. Also aggregates tnc_type across the group's TNC segments: POOLED wins if any TNC segment is pooled, otherwise the lowest tnc_type present (null if the group has no TNC segments).

Parameters:

Name Type Description Default
unlinked_trips pl.DataFrame

Canonical unlinked trips DataFrame containing group_col, the detailed mode_1-mode_4 columns, and optionally mode_type and tnc_type.

 required
group_col str

Column to group by (e.g. tour_id or joint_tour_id).

 required

Returns:

Type Description
pl.DataFrame

DataFrame with columns [group_col, transit_submode, tnc_type] where
pl.DataFrame

transit_submode is the highest submode rank (0 if none) and
pl.DataFrame

tnc_type is the aggregated TNC service type (null if no TNC segments).

ctramp_mode_expression 

ctramp_mode_expression(
    mode_type: pl.Expr,
    num_travelers: pl.Expr,
    access_mode: pl.Expr | None = None,
    egress_mode: pl.Expr | None = None,
    transit_submode: pl.Expr | None = None,
    tnc_type: pl.Expr | None = None,
) -> pl.Expr

Map canonical mode_type to CTRAMP mode integer code.

Parameters:

Name Type Description Default
mode_type pl.Expr

Polars expression for canonical mode_type (from ModeType enum)

 required
num_travelers pl.Expr

Polars expression for number of travelers in vehicle

 required
access_mode pl.Expr | None

Optional polars expression for access mode (AccessEgressMode enum)

 None
egress_mode pl.Expr | None

Optional polars expression for egress mode (AccessEgressMode enum)

 None
transit_submode pl.Expr | None

Optional polars expression for the transit submode rank (see TRANSIT_SUBMODE_* and :func:aggregate_transit_submode). When provided, transit trips are mapped to the matching CT-RAMP submode code (local/express/light rail/ferry/heavy rail/commuter rail) instead of always defaulting to local bus.

 None
tnc_type pl.Expr | None

Optional polars expression for determining TNC type

 None

Returns:

Type Description
pl.Expr

Polars expression resolving to CTRAMPModeType integer code
Notes
  • Walk=7, Bike=8
  • Transit walk-access codes: LOC=9, LRF=10, EXP=11, HVY=12, COM=13
  • Transit drive-access codes: LOC=14, LRF=15, EXP=16, HVY=17, COM=18
  • Ferry (calibration only, not standard CT-RAMP): WLK=22, DRV=23 Uses access_mode/egress_mode to detect drive-to-transit and transit_submode to pick the submode; defaults to local bus.
  • Personal vehicle by occupancy: DA=1, SR2=3, SR3=5 (non-toll)
  • TNC by service type: Regualr/Premium =20, Shared=21
  • Taxi=19
  • School bus treated as SR3=5
  • Unknown modes default to DA=1

processing.formatting.ctramp.person_mappings

CT-RAMP person-type, employment, gender, and industry mappings.

GENDER_MAP module-attribute 

GENDER_MAP = {
    Gender.MALE.value: CTRAMPGender.MALE.value,
    Gender.FEMALE.value: CTRAMPGender.FEMALE.value,
}

EMPLOYMENT_TO_CTRAMP module-attribute 

EMPLOYMENT_TO_CTRAMP = {
    Employment.EMPLOYED_FULLTIME.value: CTRAMPEmploymentCategory.FULL_TIME_EMPLOYED.value,
    Employment.EMPLOYED_PARTTIME.value: CTRAMPEmploymentCategory.PART_TIME_EMPLOYED.value,
    Employment.EMPLOYED_SELF.value: CTRAMPEmploymentCategory.FULL_TIME_EMPLOYED.value,
    Employment.EMPLOYED_UNPAID.value: CTRAMPEmploymentCategory.PART_TIME_EMPLOYED.value,
}

INDUSTRY_TO_EMPSIX module-attribute 

INDUSTRY_TO_EMPSIX = {
    Industry.AGRICULTURE.value: CTRAMPIndustry.AGREMPN.value,
    Industry.MINING.value: CTRAMPIndustry.AGREMPN.value,
    Industry.UTILITIES.value: CTRAMPIndustry.MWTEMPN.value,
    Industry.CONSTRUCTION.value: CTRAMPIndustry.OTHEMPN.value,
    Industry.MANUFACTURING.value: CTRAMPIndustry.MWTEMPN.value,
    Industry.WHOLESALE_TRADE.value: CTRAMPIndustry.MWTEMPN.value,
    Industry.RETAIL_TRADE.value: CTRAMPIndustry.RETEMPN.value,
    Industry.TRANSPORTATION.value: CTRAMPIndustry.MWTEMPN.value,
    Industry.INFORMATION.value: CTRAMPIndustry.OTHEMPN.value,
    Industry.FINANCE_AND_INSURANCE.value: CTRAMPIndustry.FPSEMPN.value,
    Industry.REALESTATE.value: CTRAMPIndustry.FPSEMPN.value,
    Industry.PROFESSIONAL.value: CTRAMPIndustry.FPSEMPN.value,
    Industry.MANAGEMENT.value: CTRAMPIndustry.FPSEMPN.value,
    Industry.ADMINISTRATIVE.value: CTRAMPIndustry.FPSEMPN.value,
    Industry.EDUCATIONAL.value: CTRAMPIndustry.HEREMPN.value,
    Industry.HEALTH_AND_SOCIAL.value: CTRAMPIndustry.HEREMPN.value,
    Industry.ARTS_AND_RECREATION.value: CTRAMPIndustry.HEREMPN.value,
    Industry.ACCOMMODATION.value: CTRAMPIndustry.HEREMPN.value,
    Industry.OTHER.value: CTRAMPIndustry.OTHEMPN.value,
    Industry.PUBLIC_ADMINISTRATION.value: CTRAMPIndustry.OTHEMPN.value,
}

INDUSTRY_OTHER_KEYWORD_TO_EMPSIX module-attribute 

INDUSTRY_OTHER_KEYWORD_TO_EMPSIX = {
    "technology": CTRAMPIndustry.FPSEMPN.value,
    "biotechnology": CTRAMPIndustry.FPSEMPN.value,
    "biotech": CTRAMPIndustry.FPSEMPN.value,
    "biomedical": CTRAMPIndustry.FPSEMPN.value,
    "tech": CTRAMPIndustry.FPSEMPN.value,
    "software": CTRAMPIndustry.FPSEMPN.value,
    "security": CTRAMPIndustry.FPSEMPN.value,
    "legal": CTRAMPIndustry.FPSEMPN.value,
    "law": CTRAMPIndustry.FPSEMPN.value,
    "attorney": CTRAMPIndustry.FPSEMPN.value,
    "marketing": CTRAMPIndustry.FPSEMPN.value,
    "government": CTRAMPIndustry.OTHEMPN.value,
    "judicial": CTRAMPIndustry.OTHEMPN.value,
    "national park service": CTRAMPIndustry.OTHEMPN.value,
    "law enforcement": CTRAMPIndustry.OTHEMPN.value,
    "military": CTRAMPIndustry.OTHEMPN.value,
    "library": CTRAMPIndustry.OTHEMPN.value,
    "automotive": CTRAMPIndustry.MWTEMPN.value,
    "nonprofit": CTRAMPIndustry.HEREMPN.value,
    "non-profit": CTRAMPIndustry.HEREMPN.value,
    "non profit": CTRAMPIndustry.HEREMPN.value,
    "philanthropy": CTRAMPIndustry.HEREMPN.value,
    "childcare": CTRAMPIndustry.HEREMPN.value,
    "health": CTRAMPIndustry.HEREMPN.value,
    "fitness": CTRAMPIndustry.HEREMPN.value,
    "school": CTRAMPIndustry.HEREMPN.value,
    "hospitality": CTRAMPIndustry.HEREMPN.value,
    "hotel": CTRAMPIndustry.HEREMPN.value,
    "e-commerce": CTRAMPIndustry.RETEMPN.value,
    "ecommerce": CTRAMPIndustry.RETEMPN.value,
}

log_person_type_warnings 

log_person_type_warnings(df: pl.DataFrame) -> dict[str, int]

Log warnings for logically impossible person attribute combinations.

Checks for combinations that are logically impossible in the real world (e.g., 10-year-old full-time workers, 4-year-old college students) and logs warnings to help identify data quality issues.

Parameters:

Name Type Description Default
df pl.DataFrame

DataFrame with age, employment, student, school_type columns

 required

Returns:

Type Description
dict[str, int]

Dictionary with warning counts by category
Example

warnings = log_person_type_warnings(persons_df) if sum(warnings.values()) > 0: ... logger.warning(f"Found {sum(warnings.values())} impossible combinations")

ctramp_person_type_expression 

ctramp_person_type_expression(
    employment_category_col: str = "employment_category",
    student_category_col: str = "student_category",
    age_col: str = "age",
) -> pl.Expr

Create expression to derive person category from person attributes.

This replicates the pptyp logic from the old pipeline's 02a-reformat step, converting employment/student/age data into person type categories.

Classification Precedence Rules (highest to lowest): 1. Age < 5 → CHILD_UNDER_5 (Type 8) - overrides all other attributes 2. Age 5-15 → STUDENT_NON_DRIVING_AGE (Type 7) - cannot be workers 3. Grade/high school student (16-17) → STUDENT_DRIVING_AGE (Type 6) - Children stay children regardless of employment 4. Full-time employment → FULL_TIME_WORKER (Type 1) - Beats student status even for full-time college students 5. Student status + school type → determines child/university types 6. Part-time employment → PART_TIME_WORKER (Type 2) - Unless also a college student → UNIVERSITY_STUDENT (Type 3) 7. Age 65+ without employment → RETIRED (Type 5) 8. Default by age group → NON_WORKER (Type 4) or child types

Edge Case Handling
  • Children with impossible employment (e.g., 10-year-old FT worker) are classified by age rules
  • Young adults (16-24) who are neither students nor employed are classified as NON_WORKER (Type 4), except 16-17 → STUDENT_DRIVING_AGE
  • Seniors 65+ with employment are classified as workers, not RETIRED
  • Impossible combinations (e.g., 4-year-old college student) should be logged using log_person_type_warnings() before classification

Parameters:

Name Type Description Default
employment_category_col str

Name of the pre-derived employment category column (CTRAMPEmploymentCategory values).

 'employment_category'
student_category_col str

Name of the pre-derived student category column (CTRAMPStudentCategory values).

 'student_category'
age_col str

Name of age column (categorical AgeCategory)

 'age'

Returns:

Type Description
pl.Expr

Polars expression that evaluates to CTRAMPPersonType enum value
See Also

log_person_type_warnings: Function to detect and log impossible combinations

Note

Age is a categorical variable (see AgeCategory enum): 1=under 5, 2=5-15, 3=16-17, 4=18-24, 5=25-34, etc.

Both category columns are required. Every caller derives them first -- student_category via ctramp_student_category_expression, employment category via EMPLOYMENT_TO_CTRAMP -- so that the rules live in one place. EMPLOYED_UNPAID is treated as part-time by that mapping, and a student 18 or over whose school_type is missing is resolved to COLLEGE_OR_HIGHER by ctramp_student_category_expression, not here.

add_industry_empsix 

add_industry_empsix(persons: pl.DataFrame) -> pl.DataFrame

Add an industry_empsix column derived from canonical industry.

Uses the structured NAICS industry code first, then fills any remaining nulls from keyword matches in the free-text industry_other when that column is present. Both inputs are read from the canonical persons frame; nothing is required upstream.

Parameters:

Name Type Description Default
persons pl.DataFrame

Canonical persons frame (must have industry; industry_other is used when present).

 required

Returns:

Type Description
pl.DataFrame

persons with an added industry_empsix column.

bad_person_type_combinations 

bad_person_type_combinations() -> pl.DataFrame

Return a lookup table of known bad person_type / attribute combinations.

Each row describes a (person_type, student_category, employment_category, age_bin) combo that should not occur, along with the expected correct person_type classification. Used by debug_ptype to flag rows in the frequency distribution table.

Bad combos (from BATS 2023 analysis): │ Full-time worker ┆ College or higher ┆ Part-time employed ┆ 18-64 │ → university student │ Full-time worker ┆ Grade or high school ┆ Full-time employed ┆ 5-17 │ → child of non-driving age │ Full-time worker ┆ Grade or high school ┆ Part-time employed ┆ 5-17 │ → child type by age │ Full-time worker ┆ Not a student ┆ Part-time employed ┆ 18-64 │ → part-time worker │ Nonworker ┆ College or higher ┆ Not employed ┆ 18-64 │ → university student │ Part-time worker ┆ College or higher ┆ Part-time employed ┆ 18-64 │ → university student │ Retired ┆ College or higher ┆ Full-time employed ┆ 65+ │ → full-time worker │ Retired ┆ College or higher ┆ Part-time employed ┆ 65+ │ → part-time worker │ Retired ┆ Not a student ┆ Full-time employed ┆ 65+ │ → full-time worker │ Retired ┆ Not a student ┆ Part-time employed ┆ 65+ │ → part-time worker │ University student ┆ College or higher ┆ Full-time employed ┆ 18-64 │ → full-time worker │ University student ┆ Not a student ┆ Not employed ┆ 18-64 │ → nonworker │ University student ┆ Not a student ┆ Part-time employed ┆ 18-64 │ → part-time worker │ Child of driving age ┆ Not a student ┆ Not employed ┆ 18-64 │ → nonworker

Note: rows 2-3 share the same join key (driving vs non-driving depends on exact age within 5-17); consolidated as "child type by age" in the rules.

Returns:

Type Description
pl.DataFrame

DataFrame with columns (person_type, student_category,
pl.DataFrame

employment_category, age_bin, expected).

processing.formatting.ctramp.purpose_mappings

CT-RAMP tour and trip purpose mappings.

NO_EMPLOYMENT module-attribute 

NO_EMPLOYMENT = pl.lit(None, dtype=pl.Int64)

PURPOSECATEGORY_TO_JTF_GROUP module-attribute 

PURPOSECATEGORY_TO_JTF_GROUP = {
    PurposeCategory.SHOP.value: "S",
    PurposeCategory.ERRAND.value: "M",
    PurposeCategory.MEAL.value: "E",
    PurposeCategory.SOCIALREC.value: "V",
    PurposeCategory.WORK.value: "D",
    PurposeCategory.WORK_RELATED.value: "D",
    PurposeCategory.SCHOOL.value: "D",
    PurposeCategory.SCHOOL_RELATED.value: "D",
    PurposeCategory.ESCORT.value: "D",
    PurposeCategory.OTHER.value: "D",
    PurposeCategory.HOME.value: "D",
    PurposeCategory.OVERNIGHT.value: "D",
    PurposeCategory.CHANGE_MODE.value: "D",
    PurposeCategory.MISSING.value: "D",
    PurposeCategory.PNTA.value: "D",
    PurposeCategory.NOT_IMPUTABLE.value: "D",
}

msg module-attribute 

msg = f"Missing PurposeCategory mappings in PURPOSECATEGORY_TO_JTF_GROUP: {_missing_categories}"

is_employed 

is_employed(employment: pl.Expr) -> pl.Expr

Whether the person has a job (see EMPLOYED).

is_work_purpose 

is_work_purpose(
    purpose_category: pl.Expr, employment: pl.Expr
) -> pl.Expr

Whether CT-RAMP takes a tour of this purpose, by this person, as work.

An employed person's work-related tour is work: the survey separates the workplace from a meeting, jobsite or volunteering, and the model does not. Employment decides, not person type, so a student with a job counts too. Anyone else's work-related tour stays non-mandatory -- a retiree's is volunteering, and a child's is a parent's purpose copied onto a shared trip. A reported workplace visit is work for everyone, as it always was.

ctramp_purpose_category_expression 

ctramp_purpose_category_expression(
    purpose_category: pl.Expr,
    income: pl.Expr,
    school_type: pl.Expr,
    employment: pl.Expr,
    income_low_threshold: int,
    income_med_threshold: int,
    income_high_threshold: int,
    parent_tour_purpose: pl.Expr | None = None,
    is_subtour: pl.Expr | None = None,
    purpose_kind: Literal["tour", "trip"] = "tour",
) -> pl.Expr

Map canonical PurposeCategory to CTRAMP purpose string.

CTRAMP requires detailed purpose strings that distinguish work income levels (low/med/high/very high) and school types (grade/high/university).

Parameters:

Name Type Description Default
purpose_category pl.Expr

Polars expression for canonical purpose category (from trips.PurposeCategory enum)

 required
income pl.Expr

Polars expression for household income (absolute dollars)

 required
school_type pl.Expr

Polars expression for school type (from persons.SchoolType enum)

 required
employment pl.Expr

Polars expression for the canonical employment code. Whether the person has a job decides whether a work-related purpose is work (see :func:is_work_purpose).

 required
income_low_threshold int

Income threshold for low bracket

 required
income_med_threshold int

Income threshold for med bracket

 required
income_high_threshold int

Income threshold for high bracket

 required
parent_tour_purpose pl.Expr | None

Polars expression for parent tour purpose (if available)

 None
is_subtour pl.Expr | None

Polars expression for identifying subtours

 None
purpose_kind Literal['tour', 'trip']

Emit detailed tour purposes (CTRAMPTourPurpose) or the simplified trip purposes (CTRAMPTripPurpose), which collapse work income levels, school types, and at-work/escort sub-purposes.

 'tour'

Returns:

Type Description
pl.Expr

Polars expression resolving to CTRAMP purpose string

processing.formatting.ctramp.student_mappings

CT-RAMP student-category mappings and validation diagnostics.

ctramp_student_category_expression 

ctramp_student_category_expression(
    school_taz_col: str = "school_taz",
    age_col: str = "age",
    student_col: str = "student",
    school_type_col: str = "school_type",
) -> pl.Expr

Create expression to derive student category from person attributes.

Derives CT-RAMP student category using a three-tier classification system: 1. Valid student status + valid school type → map by school level 2. Missing student OR missing school type → age-based fallback 3. Non-student with any school type → NOT_STUDENT

Classification Rules

Tier 1 - Valid Data (student status present + school type present): - ELEMENTARY/MIDDLE_SCHOOL/HIGH_SCHOOL/HOME_SCHOOL → GRADE_OR_HIGH_SCHOOL - COLLEGE_2YEAR/COLLEGE_4YEAR/GRADUATE_SCHOOL/VOCATIONAL → COLLEGE_OR_HIGHER - DAYCARE/PRESCHOOL/ATHOME → NOT_STUDENT (early childhood)

Tier 2 - Age-Based Fallback (student=MISSING OR school_type=MISSING/PNTA/OTHER): - Age 5-15 (AGE_5_TO_15) → GRADE_OR_HIGH_SCHOOL (compulsory education age) - Age 16-17 (AGE_16_TO_17) → GRADE_OR_HIGH_SCHOOL (still in school likely) - All other ages → NOT_STUDENT (under 5 or adult)

Tier 3 - Catch-all: - NONSTUDENT with any school type → NOT_STUDENT

Edge Case Handling
  • Children age 5-17 with missing student/school data are assumed to be in school, preventing incorrect NOT_STUDENT classification
  • Adults 18+ with missing data are assumed NOT_STUDENT (most adults)
  • Early childhood programs (daycare/preschool) are classified as NOT_STUDENT since CT-RAMP doesn't model school travel for ages <5

Parameters:

Name Type Description Default
school_taz_col str

Name of school location TAZ column (used to detect if person has a school location)

 'school_taz'
age_col str

Name of age column (AgeCategory enum values)

 'age'
student_col str

Name of student column (Student enum)

 'student'
school_type_col str

Name of school_type column (SchoolType enum)

 'school_type'

Returns:

Type Description
pl.Expr

Polars expression that evaluates to CTRAMPStudentCategory enum value
See Also

log_student_category_warnings: Function to detect and log data quality issues

Note

Age should be AgeCategory enum values (e.g., AGE_5_TO_15=2, AGE_16_TO_17=3). Applied before age category conversion to continuous in format_persons.py.

Examples:

>>> df = df.with_columns(
...     ctramp_student_category_expression().alias("student_category")
... )

log_student_category_warnings 

log_student_category_warnings(
    df: pl.DataFrame, config: CTRAMPConfig
) -> dict[str, int]

Detect and log problematic student/school type combinations.

Identifies data quality issues where student status, school type, and age are inconsistent or missing. These combinations may result in incorrect student_category classification or reliance on age-based fallbacks.

Warning Categories
  • missing_data_used_fallback: Students age 5-17 with missing student status or school type who received GRADE_OR_HIGH_SCHOOL via age fallback
  • preschool_students: Children under 5 with active student status (impossible)
  • age_inappropriate_school_types: Mismatched age/school_type combinations (e.g., teens in elementary, children in college, adults in HOME_SCHOOL)
  • nonstudents_with_school_location: Persons classified as NOT_STUDENT but with a school location TAZ (contradictory data)
  • fulltime_workers_no_work_location: Full-time workers without work location, including those with school location (college students misclassified as workers)

Parameters:

Name Type Description Default
df pl.DataFrame

DataFrame with person_id, age, student, school_type, student_category, person_type, work_taz, and school_taz columns

 required
config CTRAMPConfig

CTRAMPConfig object containing taz_field name for school/work locations

 required

Returns:

Type Description
dict[str, int]

Dictionary mapping warning category names to counts of affected persons
See Also

ctramp_student_category_expression: Expression that derives student_category log_person_type_warnings: Similar validation for person_type

Note

This function logs warnings but does not modify the DataFrame. It should be called after student_category and person_type have been derived to audit data quality.

processing.formatting.ctramp.tour_frequency_mappings

CT-RAMP mandatory and non-mandatory tour-frequency mappings.

get_imf_choice_from_counts 

get_imf_choice_from_counts(
    work_count: int, school_count: int
) -> int | None

Map mandatory tour counts to IMFChoice enum value.

Maps work and school tour counts to the appropriate IMFChoice enum value using ceiling semantics: counts >= 2 are capped at 2.

Parameters:

Name Type Description Default
work_count int

Number of work tours (0, 1, 2+)

 required
school_count int

Number of school tours (0, 1, 2+)

 required

Returns:

Type Description
int | None

IMFChoice enum value (1-5), or None if both counts are 0
Mapping logic
  • 1 work, 0 school -> ONE_WORK (1)
  • 2+ work, 0 school -> TWO_WORK (2)
  • 0 work, 1 school -> ONE_SCHOOL (3)
  • 0 work, 2+ school -> TWO_SCHOOL (4)
  • 1+ work, 1+ school -> ONE_WORK_ONE_SCHOOL (5)
  • 0 work, 0 school -> None

get_inmf_code_from_counts 

get_inmf_code_from_counts(
    escort: int,
    shopping: int,
    othmaint: int,
    othdiscr: int,
    eatout: int,
    social: int,
) -> int

Map per-purpose non-mandatory tour counts to INMF alternative code.

Maps individual non-mandatory tour counts to the CT-RAMP alternative code (1-96) using ceiling semantics: counts exceeding the codebook maximum are capped to the maximum before lookup.

Parameters:

Name Type Description Default
escort int

Number of escort tours (0..2+, capped to 2)

 required
shopping int

Number of shopping tours (0..1+, capped to 1)

 required
othmaint int

Number of other maintenance tours (0..1+, capped to 1)

 required
othdiscr int

Number of other discretionary tours (0..1+, capped to 1)

 required
eatout int

Number of eating out tours (0..1+, capped to 1)

 required
social int

Number of social tours (0..1+, capped to 1)

 required

Returns:

Type Description
int

Alternative code (1-96), or 0 if all counts are 0

aggregate_tour_statistics 

aggregate_tour_statistics(tours: pl.DataFrame) -> pl.DataFrame

Aggregate tour statistics by person for activity patterns.

Computes: - activity_pattern: M (mandatory), N (non-mandatory), H (no tours) - imf_choice: IMF alternative code (1-5) or 0 if no mandatory tours - inmf_choice: INMF alternative code (1-96) or 0 if no non-mandatory tours

Parameters:

Name Type Description Default
tours pl.DataFrame

DataFrame with tour_purpose field (CTRAMP formatted)

 required

Returns:

Type Description
pl.DataFrame

DataFrame with person_id and aggregated statistics

log_patterns_not_offered 

log_patterns_not_offered(persons_ctramp: pl.DataFrame) -> None

Warn about person-days whose mandatory pattern the model never offers their type.

The pattern is kept as the survey shows it; the model simply has no alternative to reproduce it with, so the log says how many there are.

Parameters:

Name Type Description Default
persons_ctramp pl.DataFrame

Person-days with person_type and imf_choice.

 required