Standard Operating Procedures for Conducting Surveys

This page provides information about MTC’s Standard Operating Procedures (SOP) for conducting MTC-funded transit passenger surveys within the Bay Area. Please use the Contents links below to find appropriate guidance by section. Please refer any questions or comments to Shimon Israel at sisrael@bayareametro.gov or 415-778-5239.

CONTENTS

1. SOP Goals
2. Procurement and Contracting
3. Data Collection
4. Survey Weighting
5. Survey Scheduling
6. Survey Funding
7. Data Sharing

SOP Goals

The SOPs here describe how the Bay Area Transit Passenger Survey (TPS) program will be administered in cooperation between MTC and operators. Through this process, it is our current goal to cooperatively survey the entire region’s Transportation Development Act (TDA) funded fixed-route transit ridership over a five- to seven-year period. Survey goals include supporting operator Title VI reporting requirements; refinement of analytical planning tools, such as MTC’s travel demand model; and providing data for regional and operator-specific ridership market and equity analyses.

Procurement and Contracting

MTC will procure consulting resources to carry out the TPS program, and oversee consultant performance to ensure delivery of high quality data and survey summary products. MTC’s current contracting environment includes a “bench” of three consultants to spread the work among. All three were selected in a competitive recruitment and all have a lot of experience in transit passenger surveying. The three bench consultants from MTC’s most recent procurement are Corey, Canapary, & Galanis of San Francisco, California; ETC Institute of Olathe, Kansas; and, Redhill Group of Irvine, California.

MTC staff works closely with operator liaisons in all matters related to survey scheduling (see Survey Scheduling, below), contracting, and administration. Work is assigned to contractors via MTC-issued task orders, which include work scope, requested deliverables, budget, anticipated schedule, and other relevant items such as disadvantaged business enterprise (DBE) subcontracting goals, etc. A recent example executed task order is available here.

Data Collection

It is MTC’s goal to collect data from somewhere between 5 and 7.5 percent of passenger boardings, attempting to strike a balance between survey cost and sample representativeness of transit ridership. Sampling plans are developed to select passengers randomly, spread over the day-long operating hours of transit routes, for both weekday and weekend ridership, as relevant. Survey quota goals are segmented by time-of-day periods for weekday ridership (either MTC’s time periods, the time periods used by operators, or custom time periods), and by Saturday and Sunday for weekends. The battery of questions asked passengers has become mostly standardized over the many surveys conducted by MTC. An exception to this is the approximately 3-5 questions that can be added by operators to produce a more customized survey. Example custom questions include customer service-type questions such as frequency of ridership and rider experience on transit vehicles. The types of questions asked in a standard MTC survey are described below, and include:

1. Geocoded location data

a. Trip origin	
b. First transit boarding 	
c. Last transit alighting	
d. Trip destination	
e. Bus/rail transfer location points	
f. Home location	
g. Work location (if appropriate)	
h. School location (if appropriate)

2. Access/egress modes

a. Access mode from trip origin to first transit boarding	
b. Egress mode from last transit alighting to trip destination

3. Type of trip taken

a. Origin type (e.g., home, work, shopping, etc.)	
b. Destination type (same types as origin)

4. Time leaving home in the A.M. and returning home in the P.M.

5. Fare payment

a. Payment method (e.g., cash, Clipper Card, paper transfer, etc.)	
b. Payment category (e.g., adult, senior, youth, etc.)

6. Household and person demographics

a. Hispanic/Latino status
b. Race	
c. Age	
d. Gender	
e. Worker status	
f. Student status	
g. Ability to speak English	
h. Language spoken at home other than English	
i. Disability status
j. Number of persons in household	
k. Number of workers in household	
l. Number of household working vehicles	
m. Household income

7. Custom questions by operators. Examples include:

a. Frequency of riding the transit operator
b. Means of getting schedule information (paper schedules, internet site, etc.)
c. Overall satisfaction with service

Due to lower response rates and incompleteness of many surveys, MTC has migrated away from the use of paper survey instruments for transit passenger data collection. Instead, MTC has collected passenger data using two different methods. The first method, initially deployed in 2012, involved capturing passenger names and contact phone numbers, and then administering a follow-up computer-assisted telephone interview (CATI). The CATI survey efficiently tailors questions to match the travel patterns and demographic characteristics of each passenger. More recently, beginning in 2014, MTC has been utilizing the method advocated by the Federal Transit Administration (FTA) for transit passenger surveys, that of an in-person interview performed with a tablet computer. The tablet computer approach can combine higher response rates with more complete and more accurate surveying if administered by experienced, well-trained interviewers. Similar to the CATI approach, the tablet computer aids interviewers in asking questions via a surveying “script,” skipping questions that aren’t relevant to individual passengers. To help people understand the types of questions asked by MTC, a draft mock-up paper version of the Muni survey (which has also been translated into an in-person tablet computer “script”) is available here. The tablet computer “script” of the in-person Napa VINE survey is provided here to understand what is captured in the fully expanded in-person questionnaire. It is important to remember that the tablet computer used by interviewers skips through the survey, so not every question is asked of all passengers.

It should be noted that, while MTC staff believe that an in-person interview utilizing a tablet computer is currently the best available survey technology, this method is more complicated and expensive to administer. MTC Staff are open to other methods of data collection and they seek the continual refinement of survey methodology.

Survey Weighting

Because we don’t collect data from every passenger riding every route, the survey data must be carefully weighted and expanded to represent the full system ridership. After data collection and editing are complete, the survey records are expanded to observed passenger counts. The expanded survey records are intended to represent actual ridership for given service and time of day. Data are typically expanded by transit route, time of data, and direction of travel, though MTC often weights some higher ridership routes using a different, FTA-advocated method - by stop/station boarding-to-alighting combination. This latter method requires more data inputs, specifically information about the boarding and alighting patterns of passengers, data not readily available for all operators. In a closed fare payment system such as BART - where data exist summarizing all passenger totals by boarding and alighting station pairs - the survey data can readily be expanded by time of day and by station-to-station combination. For transit modes where the passenger boarding-to-alighting combination is not generally collected by the operator, directly collecting passenger boarding/alighting data can facilitate this weighting approach. Representative boarding-to-alighting data can be collected by time of day in a “pre-survey” effort separate from the main passenger survey (FTA recommends surveying 20% of riders for this purpose). The pre-survey data are then used to establish sampling plan goals for collection of representative passenger data, and are later used to weight the data. This data expansion approach is especially useful in understanding passenger movements on longer routes serving many travel markets (e.g., crosstown, to and from downtown, etc.), particularly when combined with a proper geographical segmentation of a given route. For example, the San Francisco Muni K/T line travels from Balboa Park BART Station toward Downtown San Francisco and then south to Sunnydale Station. A passenger survey of the route will find many different trip origin/destination combinations, as well as differences in rider characteristics based on what portion of the route they are riding. Segmenting the route (e.g., K/T inbound from Balboa Park BART to Van Ness, Van Ness to Embarcadero, outbound to Sunnydale, and the reverse direction of those segments) and performing boarding-to-alighting data collection provides the framework for a more nuanced portrait of the rider population traveling within and between each segment. Additionally, this approach ensures that the proper proportion of transit riders taking short trips within each segment is surveyed representatively.

Survey Scheduling

It is anticipated that it will take approximately five to seven years to survey all TDA-claiming operators within the Bay Area. In order to achieve “representativeness” of passenger selection for sampling, the surveys will be conducted in fall and spring seasons, when ridership is presumed to be less impacted by vacations, holidays, and tourism. In terms of scheduling operator surveys by year and season, this will be done in cooperation with the steering group and with individual operators. Variables to consider in developing the schedule include:

1. Timing related to capital projects, such as new light rail construction
2. Timing related to federal requirements, such as Title VI reporting
3. Changes to vehicle operating schedules
4. Balancing the resource and budgetary needs of surveying larger and smaller operators
5. Clustering sampling of operators by geography to increase efficiency in deploying consultant resources

The tentative schedule for surveying operators is available here.

Survey Funding

As described in Attachment B, Appendix B-5 of Resolution 3866 related to transit coordination, available here, both MTC and the operators have respective funding responsibilities. As per the Resolution 3866 amendment, MTC will pay 80 percent of the cost to survey service of the top seven operators by ridership, and these operators will pay a 20-percent share. These larger operators are listed as follows:

1. Alameda-Contra Costa Transit District (AC Transit)
2. Bay Area Rapid Transit District (BART)
3. Caltrain
4. Golden Gate Bridge, Highway, and Transportation District (Golden Gate Transit)
5. San Francisco Municipal Transportation Agency (SF Muni)
6. San Mateo County Transit District (SamTrans)
7. Santa Clara Valley Transportation Authority (VTA)

Federally-funded Bay Area operators not listed above will pay no cost to survey their service. MTC recognizes that operators might sometimes have surveying goals beyond satisfying baseline Title VI requirements, and that they might wish to leverage MTC’s procurement to fulfill them. For example, an operator may want to implement a survey of special populations (such as attendees to sporting events) or to better understand which modes passengers use to get to/from certain rail stations, bus stops, or ferry terminals. Such augmented efforts can typically cost the operator less - and be more efficient than a separate procurement - when administered in combination with existing MTC surveying. MTC will make every reasonable effort to accommodate such requests, and to work with operators to develop the necessary inter-agency agreements (or to modify existing agreements) for funding them. Additional scope of this kind will be added at the operator’s sole expense.

Data Sharing

All deliverables produced by the Survey Program are shared equally between MTC and operators. Both operators and MTC own any reporting materials and the raw and processed data arising from data collection. Operators are free to decide how to use these work products and with whom to share them - including internal staff, outside consultants, other governmental entities, academic researchers, and other interested parties. Because the surveys ask respondents about some personal and household characteristics regarded as personally-identifiable information (PII), it is in the interests of both MTC and operators to safeguard these data from general public release. MTC has worked closely with its General Counsel to develop two non-disclosure agreements (NDA) for the purpose of sharing data containing PII: (1) an NDA for data with passenger location information (e.g., passenger home, work, school, etc. locations) geocoded to the discrete point level (latitude, longitude); and (2) an NDA for data with location information geocoded to more aggregate geographies such as census tracts, transportation analysis zones (TAZ), or something equivalent or larger in size and/or population.

Based on Steering Committee member feedback, MTC will obtain permission to share discrete-level (latitude, longitude) data collected for the following operators:

1. SF Muni
2. BART
3. SamTrans
4. Caltrain

The NDAs MTC uses are available here for download. Transit operators are encouraged to develop similar instruments for safeguarding shared data, and/or to include protective language within their consultant contracts for this purpose.