Detailed Information on the
Workforce Investment Act - Youth Activities Assessment

Propose giving the Secretary of Labor and States increased authority to reallocate resources to areas of need.

Strengthen accountability for employment outcomes and skill attainment by adopting common performance measures and targets to allow for comparisons with other federal job training programs.

Conducting an evaluation to determine WIA services' impact on employment and earnings outcomes for participants. An evaluation using existing administrative data will be completed by December 2008.

Consolidate the program with three other State grant programs, to increase the number of workers trained, improve services, and eliminate unnecessary overhead.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Measure: Placement.

Explanation:Percent of youth who enter employment or enroll in post-secondary education and/or advanced training/occupational skills training or the military in the first quarter after exit. 2002 data are for previous, similar placement measure for youth aged 14-18.

Measure: Credential Attainment.

Explanation:Percent of students who attain a GED, high school diploma, or certificate by the end of the third quarter after exit. New measure implemented in 2005. Data prior to 2005 are for a previous, similar measure.

Measure: Literacy or Numeracy Gains.

Explanation:Percent of basic-skills-deficient students who achieve literacy or numeracy gains of one or more Adult Basic Education level. New in PY 2006.

Measure: Cost per participant

Explanation:2004 show performance prior to the proposed program consolidation.

Is the program purpose clear?

Explanation: Supports formula-funded grants to States to fund education and training for disadvantaged youth aged 14-21. Provides a variety of services for improving educational and employment outcomes for youth.

Evidence: Sec. 129 of the Workforce Investment Act (29 U.S.C. 2854/Public Law 105-220). "Prepared Workforce" Goal 1.2A in DOL's FYs 2003-2008 Strategic Plan. Strategic plans are accessible at www.dol.gov/aboutdol/main.htm.

Does the program address a specific interest, problem or need?

Explanation: There are 35 million youth aged 16-24 in the US. 4 million of those are high school dropouts, 6.8 million are high school graduates with no college education. The unemployment rate for dropouts aged 16-24 is 24%, the rate for high school graduates without college is 15%. The unemployment rate for African American youth who are dropouts is 43%. Youth Activities is aimed at improving the educational and employment outcomes of these youth.

Evidence: DOL Report on Youth Workforce Development: What We've Learned, 2002.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: Population served by program is a very small portion of those eligible to receive services. There are 10.6 million youth age 16-24 who are high school dropouts (4 million) or have completed high school but have no college education (6.8 million). Youth Activities serves fewer than 500,000 youth.

Evidence: DOL FY 2004 Congressional Justification. DOL Report on Youth Workforce Development: What We've Learned, 2002.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: Youth Activities provides education and training to at-risk youth. There are a number of Federal programs that provide similar education and training services including Job Corps, Youth Opportunity Grants, Department of Education's Vocational Education, TRIO and Gear Up programs and HUD's Youthbuild. These programs all have similar goals and objectives. However, the Administration's WIA reauthorization proposal includes a substantial change in the population to be served, proposing to serve primarily out-of-school youth--a large and underserved population facing multiple barriers to success in the workforce. In addition, DOL is an active participant in the White House Task Force for Disadvantaged Youth, which is examining--and looking at ways to address--the duplication of services.

Evidence:  

Is the program optimally designed to address the interest, problem or need?

Explanation: Currently, funds for the WIA youth program are spread too thinly nationwide due to the statutory formula and broad population to be served. The Administration's WIA reauthorization proposal includes substantial changes to the program, including a shift from formula resources to an increased emphasis on resources targeted to problem areas and an increased focus on out-of-school youth.

Evidence: General Accounting Office, "Workforce Investment Act: Issues Related to Allocation Formulas for Youth, Adults, and Dislocated Workers" (April 2003).

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Youth Activities is part of the Job Training Common Measures exercise. Accordingly, the program has adopted four specific long-term goals that will better measure the impacts of the program and allow comparisons across similar programs. WIA requires that DOL negotiate performance levels with each State. The numerical annual targets set by DOL in the FY04 submission will be used as the basis for those negotiations and the basis for setting long-term targets. OMB, DOL and other partner agencies are currently working together to finalize the methodology for applying the common measures to the WIA youth program and guidance on the common measures will be issued within the next few months.

Evidence: Section 136 of the Workforce Investment Act (29 U.S.C. 2871) (P.L. 105-220). DOL's FYs 2003-2008 Strategic Plan. Strategic plans are accessible at www.dol.gov/aboutdol/main.htm.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: Youth Activities is part of the Job Training Common Measures exercise. Accordingly, the program has adopted four specific annual goals that will better measure the impacts of the program and allow comparisons across similar programs. DOL has begun to implement the common measures format, establishing numerical annual targets for some of the common measures in its FY 2004 Budget. WIA requires that DOL negotiate performance levels with each State. The numerical targets set by DOL in the FY04 submission will be used as the basis for those negotiations.

Evidence: DOL FY04 Annual Performance Plan. Performance plans are accessible at www.dol.gov/aboutdol/main.htm.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: States are mandated to adopt specific goals established in statute, negotiate with DOL to establish individual State performance levels and report on their progress on achieving these levels. States comply with this requirement. The statute also provides an incentive for participation in which States that meet or exceed negotiated levels are eligible to receive incentive grants. To be eligible to receive the award States must provide complete performance reports that meet DOL's data validation standards. In the future, DOL expects to establish a more rigorous data validation process. These new validation requirements will provide an additional basis for incentives or sanctions (in addition to performance levels).

Evidence: Sections 136 and 503 of the Workforce Investment Act (29 U.S.C. 2871 and 29 U.S.C 9273) (P.L. 105-220). Training and Employment Guidance Letter (TEGL) 14-00, Change 1: Guidance on the WIA Management Information and Reporting System, October 1, 2001, Attachments E, F, and G; Field Memo No. 30-00 WIA Financial Reporting; June 23, 2000; TEGL 8-99: Negotiating Performance Goals; and Incentives and Sanctions for the Negotiation and Goal Setting Process; March 3, 2000, Sec. 4-5; TEGL 7-99; Core and Customer Satisfaction Performance Measures for the WIA System; March 3, 2000; FYs 2003 and 2004 Annual Performance Plans. TEGLs are accessible at www.doleta.gov/usworkforce/documents/, and performance plans are accessible at www.dol.gov/aboutdol/main.htm.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: Historically there has been a significant disconnect between the youth job training and education communities -- at the Federal, State and local level. WIA established local Youth Councils with responsibility for steering local WIA youth policy. Educational entities are not a mandated partner on these boards and anecdotal information indicates that, by and large, they are not participating. OMB's recent job training common measures exercise is one of many attempts to bring DOL and ED programs together.

Evidence: Section 117(h) of the Workforce Investment Act (29 U.S.C. 2832/Public Law 105-220).

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: No impact or outcome evaluations of the WIA program have been conducted nor are any financed under the WIA long-term research plan. An impact analysis was conducted of the predecessor program in 1992. Process evaluations have been conducted of the WIA implementation. DOL revised its evaluation plan, which will establish a regular cycle of evaluation for all major job training and employment programs. The evaluation of Youth Activities will begin in FY 2006.

Evidence: DOL FY 2004 Budget request.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: Funding changes are roughly linked to participation numbers but not to performance outcomes. DOL can produce only very rough estimates of the amounts of funding needed to maintain participation levels from prior years. Even in that regard, it appears that estimates have far exceeded what is needed due, in part, to the poor reporting by States and localities. In preparing its FY 2004 budget submission, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels on performance outcomes.

Evidence: DOL's FY 2003 and FY 2004 Budget Requests. Budget requests are accessible at www.dol.gov/aboutdol/main.htm.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: WIA specifies the performance measures on which States will report and requires that DOL negotiate performance levels with each State. As specified in statute, these negotiations occurred at the beginning of WIA implementation and established goals for the first 3 years of implementation and then again in year three for the final two years of authorization. This statutory requirement hampers DOL's ability to develop long-term goals. DOL is taking two steps to address this obstacle. First, the WIA reauthorization proposal includes changes to the State negotiation process that, among other things, will allow for longer-term strategic planning. Second, as part of the Job Training Common Measures exercise, DOL has adopted 4 long-term and annual goals for Youth Activities and established numerical values for those goals wherever possible. WIA requires that DOL negotiate performance levels with each State.

Evidence: DOL's FY 2003-2008 Strategic Plan and FY 2004 Annual Performance Plan. Strategic and performance plans are posted at www.dol.gov/aboutdol/main.htm.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOL uses performance data to award incentive grants and identify areas needing technical assistance, which DOL provides to States that fail on one or more performance measures. However, this information is not reported uniformly to DOL in as timely a manner as would be useful for effective program management. For example, in the FY 2002 Annual Performance Report, the data for the younger youth goal (14-18 yr olds) were incomplete. Only 42 of the 53 States and jurisdictions reported final data and 7 of those submitted data that are questionable. States are supposed to report some information to DOL quarterly, but there may be a delay of several months from the end of a quarter to the time that DOL receives, compiles, validates, and publishes the information. In 2003, DOL issued policy guidance for a data validation project that will create more precise programming specifications and standards for use in validating grantee-reported data. ETA anticipates that states/grantees will finish validating 2002 data and begin reporting to ETA in March 2004.

Evidence: DOL's Annual Report on Performance and Accountability for FY 2002, which is accessible at www.dol.gov/_sec/media/reports/annual2002/; and DOL's IG's Report No. 06-02-006-03-390, "Workforce Investment Act Performance Outcomes Reporting Oversight," September 30, 2002. DOL's IG's reports are accessible at www.oig.dol.gov/public/reports/oa/main.htm. TEGL 3-03: Data Validation Policy for Employment and Training Programs. Information on the DOL's data validation initiative for ETA programs is accessible at www.uses.doleta.gov/dv/.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: WIA awards incentive grants to States that meet or exceed negotiated levels of performance. In addition, the statute authorizes the Secretary to apply sanctions to States that fail to meet agreed-upon performance levels. For PY 2001, four States received monetary sanctions for failing to meet performance levels for two consecutive years. It should be noted that in the past, a weaker sanction authority under the predecessor program was not utilized. DOL ties performance ratings for managers and supervisors to the achievement of Departmental and program-specific goals and outcomes; employees are aligned on a fiscal year cycle to help cascade standards to non-supervisory personnel, and a system accountability review is planned for 2004.

Evidence: Sections 503 and 136 (g) of the Workforce Investment Act (29 U.S.C. 9273 and 2871, P.L. 105-220). DOL's revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001). States' Annual Reports are accessible at www.doleta.gov/usworkforce/documents/. TEGL 19-02 regarding Sanctions Policy for Failure to Meet State Negotiated Performance Levels under Title I of the Workforce Investment Act (WIA) was issued on February 10, 2003.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Although the Federal program fully obligates the funds of approximately $1.1 billion annually, States and localities carry over approximately $0.5 billion of unspent funds annually. Further, even though States and localities claim to have committed most of these funds, the nature of the commitments is unknown, except case by case, so it is difficult to say confidently how available the funds are to provide services. Tracking of spending for intended purposes is insufficient. The WIA reauthorization proposal would allow DOL to recapture unexpended funds in each program in states and local areas in excess of 30% of the total funds available during a program year. WIA currently allows recapture of funds not obligated by a State or local area which exceed 20% of total funds allotted for the previous year. The removal of any ambiguity by usage of the commonly defined and understood term 'expend' will improve the ability to track funds and ensure that they are spent in a timely manner.

Evidence: Current estimates of unspent carry-in for Program Years 2000-2003, as compared to expenditures (and annual appropriations of $1.1 B): PY 2000: Carry-in $220 M, Expenditures $792 M; PY 2001: Carry-in $438 M, Expenditures $1.1 B; PY 2002: Carry-in $479 M , Expenditures $1.2 B; PY 2003: Carry-in $479 M, Expenditures $1.1 B, Carry-out $346 M (Please note: these figures are estimates.) GAO Report GAO-02-1074, "Workforce Investment Act: Interim Report on Status of Spending and States' Available Funds," September 2002; and DOL's IG's Report No. 04-02-004-03-390 (September 20, 2002) on the State of Ohio's application of agreed-upon procedures for WIA grant obligations, expenditures, and payments. DOL's IG reports are accessible at www.oig.dol.gov/public/reports/oa/main.htm.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: As part of its FY04 submission, DOL submitted unit cost data requested as part of an efficiency measure agreed to as one of the interagency Job Training Common Measures. Over the next year, OMB and the agencies involved in the Job Training Common Measures will work together to craft a methodology for utilizing this data to assess cost effectiveness and efficiency.

Evidence: DOL's FY 2004 Budget Request and FY 2004 Annual Performance Plan. Budget requests and performance plans are accessible at www.dol.gov/aboutdol/main.htm.

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: Like the rest of DOL, ETA does not have an integrated accounting and performance management system to identify the full cost of achieving this program's performance goals and support day-to-day operations. In preparing its FY 2004 budget submission, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels on performance outcomes.

Evidence: DOL's FY 2003 and FY 2004 Budget Requests. Budget requests are accessible at www.dol.gov/aboutdol/main.htm.

Does the program use strong financial management practices?

Explanation: The program is free of material internal control weaknesses as identified by auditors. However, DOL has not completed its evaluation of the level and nature of erroneous payments in its WIA programs, including this one, and how to minimize them. This evaluation is a high priority for ETA and is expected to be presented to OMB soon.

Evidence: DOL's FY 2002 and 2003 Annual Reports on Performance and Accountability, which are accessible at www.dol.gov/aboutdol/main.htm.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: DOL has tied its performance goals to performance ratings for managers; new appraisal system is to be cascaded through the Department. DOL's Assistant Secretary for Employment and Training is working to improve performance accountability of Federal managers, management of grantees and sub-recipients, and erroneous payments. DOL is developing a strategy for program reporting that will substantially address issues of timely, accurate and valid program information. It will address local, state and federal levels on performance and financial reporting, validation of data and improved compliance tools. This will help DOL to manage the program more effectively and support compliance and technical assistance efforts.

Evidence: DOL's revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001).

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: DOL has improved its oversight but continues to have insufficient knowledge of grantees' activities, due in part to WIA financial requirements. Even though states and localities may claim to have obligated most funding, the extent of such commitments is unknown, except case by case. As a result, it is difficult for DOL to say confidently the amount of funds that are available to provide services and whether funds have been spent as intended. DOL has taken several steps to improve oversight. In 2002, the Assistant Secretary directed minimum standards and set goals for grants monitoring. DOL achieved these goals in every region, and, to date, 44 of the 52 states and territories have been monitored on site. In 2003, DOL issued an accounting procedures manual for regions and an order outlining grants administration oversight roles and responsibilities. In addition, the Administration has proposed in WIA reauthorization that financial reporting and any reallocations of unspent funding be based on when funds are spent, rather than when they are obligated or committed.

Evidence: Employment and Training Order (No 1-03, April 2003); ETA Regional Accounting Procedures and Policies Manual; ETA Regional Accounting Policies and Procedures Manual; Change 1 to TEGL 16-99, which provides clarifications to the Workforce Investment Act (WIA) financial reporting instructions issued in TEGL No. 16-99, is accessible at http://wdr.doleta.gov/directives/corr_doc.asp?DOCN=1433

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: States submit WIA performance information to DOL quarterly and annually. On a quarterly basis, DOL works with state grantees to review and analyze states' performance information and helps states translate their performance into policy decisions. On ETA's Web site, DOL provides an easy link to quarterly and annual performance results for the nation and individual states. This information includes summaries of WIA participant characteristics and services received and comparisons of states' performance. DOL will continue to enhance the ETA performance web site over the next year. ETA will make it easier for the public to find performance information from the main web site and to understand what ETA and its partners seek to achieve and whether those goals have been attained.

Evidence: Performance information for ETA programs, including Youth Activities, is accessible at www.doleta.gov/usworkforce/performance/. State and national WIA annual results for PY 2000 and 2001, definitions of the performance measures, the most recent quarterly review of results reported for the quarter ending June 30, 2003, and other performance information are posted at www.doleta.gov/usworkforce/performance/ Reports.cfm. The DOL FY 2002 and FY 2003 Annual Performance and Accountability Reports are posted at www.dol.gov/dol/aboutdol/main.htm.

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: Youth Activities has adopted four new long-term goals as part of the Job Training Common Measures exercise. DOL is working with States to establish annual and long-term targets in the common measures format. Partial credit is based on the fact that DOL has made progress toward achieving its old long-term goals.

Evidence: DOL's FY 2003-2008 Strategic Plan is accessible at www.dol.gov/_sec/stratplan/main.htm. The FY 2002 and FY 2003 Annual Reports are posted at www.dol.gov/dol/aboutdol/main.htm. National Summary of PY 2001 WIA Annual Performance Data posted at www.doleta.gov/usworkforce/documents/AnnualReports/annual_report_2001.cfm, and PY 2002 data are posted at www.doleta.gov/usworkforce/performance/Edition_5_6-30-03.pdf.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Youth Activities has adopted four new annual goals as part of the Job Training Common Measures exercise. As part of the FY04 budget, DOL established numerical annual targets for most of the common measures based on similar/proxy data they had been collecting under their old goals. As DOL is required, by statute, to negotiate performance goals with each State, these common measures targets will be used as the basis for State negotiations. One of the new goals (literacy/numeracy) is a completely new goal -- DOL does not currently collect any similar data. Accordingly, DOL was unable to develop a long-term numerical target for this goal. In addition to credit toward achieving their new goals, partial credit is based on the fact that DOL has also made progress in achieving its old annual goals.

Evidence: DOL's FYs 2003-2008 Strategic Plan; Annual Reports on Performance and Accountability for FYs 2000, 2001, 2002, and 2003; and FY 2004 Annual Performance Plan. Strategic and performance plans and annual reports are accessible at www.dol.gov/aboutdol/main.htm.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: Unit costs have remained roughly the same for the WIA and the predecessor program.

Evidence:  

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: While the placement rate of this program (66% for younger youth and 78% for older youth in 2001) is below that of Job Corps (89.9% in 2001), the relative cost of these programs means that the average cost per placement for Youth Activities is below $6,000 while the average cost for Job Corps, a residential program, is over $21,000.

Evidence: Note: the placement rate described in this question measures retention in the first quarter after exit (versus in the third quarter as currently reported by Youth Activities) in order to allow for comparison between the two programs.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: No impact or outcome evaluations of the WIA program have been conducted nor are any financed under the WIA long-term research plan. Last impact evaluation (conducted in 1992) did not have favorable results. However, DOL is developing a revised evaluation plan that will establish a regular cycle of evaluation for all major job training and employment programs. The evaluation of Youth Activities will begin in FY 2006.

Evidence: