Detailed Information on the
Transportation Security Administration: Passenger Screening Technology Assessment

Purchase sufficient explosive trace portals to achieve one unit per four lanes of screener checkpoints at the Category X and I airports by FY 2007

Complete a comprehensive capital plan that addresses long term system performance needs by the third quarter of FY 2006.

Develop and implement performance targets for the new performance measures by the second quarter of FY 2006.

Measure: Level of machine effectiveness (this measure is classified)

Explanation:This measure will indicate the performance of systems operating in the field that are tested at startup to determine ongoing ability to detect threat objects at acceptable levels.

Measure: Level of machine efficiency

Explanation:This measure is the screening checkpoint capital cost per person screened with respect to walk-through metal detectors, carry on item x-ray machines, and explosives trace detection devices.

Is the program purpose clear?

Explanation: The purpose of the Passenger Screening Technology Program is provide the technology necessary to effectively carry out a statutory mandate to prevent the entry of firearms, explosives, and other dangerous weapons on aircraft through inspection of passengers as well as carry-on baggage.

Evidence: Section 110(a) of the Aviation and Transportation Security Act of 2001 requires that TSA: ''shall provide for the screening of all passengers and property, including United States mail, cargo, carry-on and checked baggage, and other articles, that will by an air carrier or foreign air carrier in air transportation or intrastate air transportation..."

Does the program address a specific and existing problem, interest or need?

Explanation: Aviation remains one of the primary focuses of Middle East and other terrorist organizations for actions against U.S. citizens, and the airport passenger screening function constitutes the front lines of preventing aircraft contraband that can assist in terrorist and other criminal acts intended to harm passengers, aircraft and other persons and property.

Evidence: Transportation Security Administration Transportation risk assessments and audits, classified intelligence/threat data collections and reports, and security oversight inspections, checkpoint arrests, dangerous item confiscation levels at airports.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: This program is the only effort that electronically screens passengers and carry on baggage before boarding commercial passenger aircraft.

Evidence: Section 110(a) of the Aviation and Transportation Security Act of 2001 requires that TSA: ''shall provide for the screening of all passengers and property, including United States mail, cargo, carry-on and checked baggage, and other articles, that will be carried aboard a passenger aircraft operated by an air carrier or foreign air carrier in air transportation or intrastate air transportation...'

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Current passenger screening technology deployed may not be sufficient to address all security needs, such as adequate screening for explosives. Management infrastructure of TSA's capital programs needs to be strengthened.

Evidence: No evidence necessary.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Screening checkpoint technology can only directly serve the screening purpose and the intended beneficiary -- air carriers and the flying public. Therefore, this question is not relevant to this program.

Evidence: No evidence necessary.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: TSA has specific long term performance measures under development related to efficiency and reliability outcomes.

Evidence: Primary measures under development include the level of machine efficiency and reliability.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Most measure targets are under development.

Evidence: No evidence necessary.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: TSA has specific annual performance measures under development related to effectiveness, efficiency, and reliability outcomes.

Evidence: Primary measures under development include the level of machine effectiveness, efficiency, and reliability.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Most measure targets are under development.

Evidence: No evidence necessary.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The annual and long-term goals of the program are aligned with partners such as maintenance contractors, support contractors, general contractors etc. The statements of work, task and delivery orders, and schedules of the contracts that support the program have direct input into achievement of programmatic goals.

Evidence: Chief Technology Officer (CTO) Programmatic Documentation: Contractor Statements of Work, Scopes of Work, Schedules, CTO Program Plans, CTO Acquisition Plan. Each contain sections that require work towards the long-term and/or annual goals of the CTO.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Passenger Screening Technology Program has not yet received independent evaluations of sufficient scope and quality. TSA should establish and implement an evaluation agenda to assess and validate key aspects of its program such as the checkpoint equipment architecture, equipment maintenance, lifecycle management plans and strategies, and acquisition management/contractor oversight. TSA will complete an evaluation plan by November, 2004.

Evidence: No evidence necessary.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Passenge technology for airport checkpoints is uniquely idenfied in the TSA Budget requests and performance goals and targets are tied to the funding level.

Evidence: All checkpoint technology funding is differentiated in the Budget justifications and the justifications are organized in a performance based structure.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: TSA's primary strategic planning deficiencies include the lack of clearly defined performance outcome goals and targets, as well as a comprehensive screening technology capital plan supporting technology investment decisions. Outcome goals and targets are under development. TSA has not yet developed a capital plan.

Evidence: No evidence necessary.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals and used the results to guide the resulting activity?

Explanation: TSA is in the process of performing an alternatives analysis and cost benefit analysis in accordance with OMB Circular A-94.

Evidence: No evidence necessary.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: TSA does not currently collect and utilize adequate performance information from its primary equipment contractors that supply, install, and maintain passenger screening equipment. However, TSA is in the process of implemeting improved management information systems.

Evidence: No evidence necessary.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Key program partners such as maintenance contractors, support contractors, general contractors, etc. are responsible for achieving results in accordance with TSA performance goals. The DHS IG has found that TSA did not in the past hold its primary equipment service provider accountable for performance. In response, TSA is in the process of improving its future contracts in this area.

Evidence: Current and planned contract documentation such as statements of work, scopes of work, schedules, etc., currently or will include performance targets which tie to program goals. Contractors are required to have earned value management systems.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Program funds are obligated consistently with the overall program plan. The schedule for obligations is established and meets the resource needs of the program. Procedures exist for reporting actual expenditures.

Evidence: Monthly obligations reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While TSA is developing efficiency performance measures and a comprehensive capital plan to help guide efficiency/effectiveness technology decisions, these efforts are not yet sufficiently mature.

Evidence: No evidence necessary.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates with the Baggage Screening Technology, Screener Workforce, and Training programs, as well as commercial airports, in allocating passenger screening equipment, and for decisions on checkpoint reconfigurations. The program also coordinates with Aviation Operations in areas such as workforce performance.

Evidence: In 2003, TSA completed an internal "Passenger Screener Performance Improvement Study" which developed a coordinated action plan in each programmatic area affecting screener performance. Action plans included technology improvements. Screener workforce allocations by screening checkpoint are coordinated with the specific technologies deployed.

Does the program use strong financial management practices?

Explanation: While TSA received a clean audit opinion, it received material weaknesses in internal control, including property management. Property management is substantially related to the Passenger Screening Technology Program.

Evidence: No evidence necessary.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Notable management deficiencies currently include the lack of a detailed screening technology capital plan guiding management decisions, the lack of an effective and cost-effective plan and contract strategy for managing equipment maintenance, inadequate program management training of TSA staff, and the lack of adequate program management information systems. All of these areas are being actively addressed.

Evidence: TSA will produce a detailed capital plan by the end of 2004; current equipment maintenance strategies are being revised and a new contract approach developed; all equipment program managers are receiving program management training and will become Level III certified; a new management information system is being implemented.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: TSA has prepared an Acquisition Plan that has not yet been approved. Current services contracts were awarded under FAA, prior to TSA, and are not performance-based. Equipment contracts are firm-fixed price, with schedules for deliverables. FAA contracts are expiring during FY '04 and will be upgraded in early FY '05.

Evidence: No evidence necessary.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: TSA has not yet established targets and timeframes for its long term performance goals.

Evidence: No evidence necessary.

Does the program (including program partners) achieve its annual performance goals?

Explanation: TSA has not yet established targets for its annual performance goals.

Evidence: No evidence necessary.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: While TSA is currently re-competing all services contracts to increase program efficiency and lower costs, in this case primarily with respect to technology maintenance, it cannot yet demonstrate efficiency and cost effectiveness improvements. Efficiency performance targets are still under development.

Evidence: No evidence necessary.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While some checkpoint screening technology is used across a wide swath of public and private organizations in the United States, and TSA uses the same or better technological approach, nothing adequately compares in the overall size and scope of TSA's operation for the purposes of a comparative analysis. TSA should determine how it can compare checkpoint technological enterprise performance with overseas aviation security operators, both public and private. TSA should include such an analysis in its future evaluation plans.

Evidence: Magnetometers, hand wands, and x-ray systems form the basis of checkpoint screening for most public and private organizations with security screening operations, including TSA. The same systems are offered by vendors to each entity.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While the Checkpoint Technology Program has not been subject to an independent evaluation of sufficient quality and scope, broad evaluations of screening system performance by GAO and the Inspector General have raised few significant problems with the checkpoint screening technology architecture, or the specific equipment. The September 11th Commission highlighted a security performance concern with respect to the inadequacy of checkpoint technology in the area of screening for explosives.

Evidence: GAO and the IG have centered concerns to date on the issue of accelerating the deployment of Threat Image Projection (TIP) technology for testing screeners at operating checkpoint x-rays. TIP has now been fully deployed.

Were program goals achieved within budgeted costs and established schedules?

Explanation: Budgeted cost and schedule targets changed several times in 2003 for passenger screening technology primarily because of policy changes with respect to capital deployments and ongoing planning negotiations with airports. Consequently, adhering to original schedules proved difficult. It is expected that long term capital planning efforts started in FY 2004 will foster improvements in execution of both acquisition and sustainment activities.

Evidence: Different cost and schedule deployment plans indicate intent and ability to abide by originally-proposed cost and schedule goals.