Detailed Information on the
Teacher Quality Enhancement Assessment

Implement plan to ensure adequate grantee oversight.

Develop long-term and annual performance measures and efficiency measures, collect baseline data, and establish appropriate targets.

Consider program reforms to enable ED to use all available TQE funding to support the highest-quality grant applications.

Assess the results of the recently completed evaluation and utilize those findings to evaluate program performance and modify program management where appropriate.

Develop and implement a strategy for making program performance data accessible to the public in a transparent way.

Review and revise program managers' performance agreements to ensure that managers are held accountable for achieving key program results.

Measure: The percentage of program completers who are highly qualified teachers

Explanation:The percentage of program completers, from Institutions of Higher Education with Teacher Quality Enhancement Partnership grants, who are highly qualified teachers (according to the NCLB definition) upon program completion.

Measure: Percentage of preservice teachers passing subject matter competency tests.

Explanation:Pass rates for preservice teachers passing subject matter competency tests as part of State licensure requirements in the States that receive funds from the Teacher Quality Enhancement State Program to prepare teachers that are highly competent in the academic content areas in which they will be teaching.

Measure: Cost per highly qualified teacher

Explanation:The cost of a successful outcome where success is defined as a program completer from a teacher training program that has received a Partenrship grant, who is considered a highly qualified teacher.

Is the program purpose clear?

Explanation: The Teacher Quality Enhancement (TQE) program's purpose is to improve the quality of the Nation's teachers by improving teacher preparation and professional development programs for current and prospective teachers, with the ultimate goal of improving student achievement.

Evidence: The purpose is laid out in Title II, Part A of the Higher Education Act, which states that: "The purposes of this title are to (1) improve student achievement; (2) improve the quality of the current and future teaching force by improving teacher preparation of prospective teachers and enhancing professional development activities; (3) hold institutions of higher education accountable for preparing teachers who have the necessary teaching skills ....; and (4) recruit highly qualified individuals, including individuals from other occupations, into the teaching force."

Does the program address a specific and existing problem, interest, or need?

Explanation: Researchers and policymakers agree that teacher quality is key to improving student achievement. However, much of the research in this area says that teacher preparation programs are not adequately training new teachers, and that these new teachers do not receive enough support in their early years of teaching. As a result, many students are taught by underprepared teachers. In addition, research has found that one third of new teachers leave the profession within five years. High levels of attrition are most severe in the highest need areas, where one half of new teachers leave within their first five years.

Evidence: Scheerens/Bosker report 'The Foundations of Educational Effectiveness' (1997); Sanders/Rivers report, 'Cumulative and Residual Effects of Teachers on Future Student Academic Achievement' (1996); National Center for Educational Statistics report 'The Condition of Education' (2001); Office of Postsecondary Education report 'Meeting the Highly Qualified Teachers Challenge: The Secretary's Second Annual Report on Teacher Quality' (2003).

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: This program is one of a range of programs in the Department that address the teacher quality issue. The program focuses on the key role that Institutions of Higher Education (IHEs) play in preparing and supporting new teachers, and in improving the quality of current teachers through improved professional development. While Title II of the NCLB Act would allow the Teacher Quality State Grants program to fund similar reforms, in practice, funds from that program are focused on local educational agencies (LEAs) and only involve IHEs in a secondary role, if at all. Although the Preparing Tomorrow's Teachers to use Technology program is similar to TQE in that it focuses on the role of the IHEs in teacher preparation, its scope is limited to technology, and therefore would not be an appropriate vehicle for institution-wide reforms.

Evidence: No other program in the Federal government focuses exclusively on the role of IHEs in teacher preparation and on improving the level of collaboration between Schools of Education and Schools of Arts and Sciences as well as between teacher preparation programs and local school districts, especially high-need districts. Research suggests that strengthening these collaborations is associated with improvements in the quality of teacher preparations programs and the students that they graduate.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Early implementation of the program has not revealed any major flaws in the actual program model that would greatly limit the program's effectiveness or efficiency. However, the authorizing statute mandates that funds appropriated for the program are divided between the State, Partnership and Recruitment parts of the program according to a 45:45:10 ratio. The statutorily mandated ratio does not reflect the level of demand for program funds, and has compelled ED to lapse TQE funds in the last two fiscal years. Other minor flaws include: inadequate funding for evaluating the State and Recruitment grants, TQE's lack of support for alternative certification programs, and the redundancy of the Recruitment program to the State and Partnership programs.

Evidence: While every competition for the Partnership program has been oversubscribed, the program encountered difficulty recruiting sufficient quality applicants for State or Recruitment funds. As a result, in FY 2002 the program lapsed $655,000 and $1,416,000 under the State and Recruitment programs respectively. In a recent report, the General Accounting Office (GAO) suggested that Congress re-visit the issue of the statutory funding ratio in order to avoid future fund lapses, especially within the State program. It should also be noted that the authorizing statute allows all of the activities of the Recruitment program to be carried out under the State and Partnership programs, and identical entities are eligible to receive funds under either of the State and Partnership programs and the Recruitment program.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program is focused on the roles of IHEs and States in ensuring that both prospective and veteran teachers have the content knowledge and teaching skills they need to help all students reach high academic standards. This IHE focus is integral to improving teacher preparation, strengthening teacher quality and, ultimately, raising student achievement.

Evidence: By mandating that IHEs partner with high-need schools or school districts, the Partnership program ensures that the program resources are more effectively targeted to achieve the maximum benefit. States oversee the teacher certification process and establish student achievement standards. As a result, it is appropriate that the State program suports better alignment of teacher certification with student achievement standards.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ED has not yet developed multiple meaningful long-term measures for the TQE program. ED has developed a long-term performance measure that focuses on the quality of partnership grants participants. The Department is working with OMB on developing an additional long-term measure for the State grants and an appropriate efficiency measure for the entire TQE program.

Evidence: The program has developed one long-term performance measure, focusing on the percentage of program completers that are highly qualified teachers (according to the NCLB definition). The program is currently working to develop an additional long-term performance measure.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has developed ambitious targets and timeframes for its only long-term measure but does not yet have multiple measures.

Evidence: The target established for the program is that by 2008, 90 percent of program participants will be highly qualified teachers (according to the NCLB definition) upon program completion. The program is currently working to develop additional targets and timeframes for the long-term measures that are currently under development.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: ED has not yet developed multiple meaningful annual measures for the TQE program. ED has developed an annual performance measure that focuses on the quality of partnership grants participants. The Department is working with OMB on developing an additional annual measure for the State grants and an appropriate efficiency measure for the entire TQE program.

Evidence: The program has developed one annual performance measure, focusing on the percentage of program completers that are highly qualified teachers (according to the NCLB definition). The program is currently working to develop an additional long-term performance measure.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Baseline data is not yet available to set useful targets for the annual measure.

Evidence: In order to collect the necessary baseline data, the program's Annual Performance Report will have to be revised. ED plans to complete the revision and collect this data by the end of 2003.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: With the recent development new annual and long-term performance goals, partners have not yet been able to commit to these new goals. The program plans outreach to its grantees to communicate the new goals and integrate these performance goals into each grantee's work plan.The TQE program will revise its Annual Performance Reports to gather the neceassry data for the new indicators.

Evidence: Applicants are currently required to demonstrate that their project has clear, measurable project goals and performance objectives and that these will lead directly to improvements in teaching quality and student achievement as measured against rigorous academic standards. Once the grantees have been informed about the newly formulated goal and objectives, the program will utilize the annual outcomes-based work plans to ensure that grantees are incorporating them into their work.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A longitudinal study of the Partnership program is currently underway and it is expected that this evaluation will provide performance information to support program improvements and evaluate effectiveness. Because of limited evaluation funding, program evaluations are not being carried out on the State and Recruitment programs.

Evidence: The Partnership evaluation is looking at both implementation issues and program outcomes, in terms of student achievement. The evaluation is examining the association between collaborative activities associated with the Partnership grants among institutions of higher education and schools, and student achievement outcomes. Using student achievements at schools participating in Partnership grants, comparisons will be drawn with a control group of comparable, non-Partnership schools. The first impact data will be available in FY 2006.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable performance information to assess (whether directly or indirectly) the impact of the Federal investment. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). ED's 05 integrated budget and performance plan includes the program's annual and long-term goals.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has identified strategic planning deficiencies and taken meaningful steps to address these deficiencies. ED and OMB will continue to work to establish an additional meaningful annual, long-term, and efficiency measure.

Evidence: In addition, the program has also initiated a process to revise program materials, such as application packets and annual performance reports, to reflect its new long-term and annual performance measures.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Grant recipients are required to submit Annual Performance Reports, and a Final Report. Furthermore, student achievement data are collected by the Department annually and are being utilized within the Partnership evaluation to assess the effectiveness of the program. However, this data has not been used to manage the program in order to improve performance.

Evidence: The program has initiated a process to revise its Annual Performance Reports in order to collect more pertinent, outcomes oriented data--particluarly for the Partnerships grant program. It is expected that this data will be used in the future to enhance program management.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: ED's managers are subject to EDPAS which links employee performance to relevant Strategic Plan goals and action steps, and is designed to measure the degree to which a manager contributes to improving program performance. To receive a "Yes," the ED needs to: (1) identify for OMB the federal managers for this program; and (2) demonstrate the relationship between these managers' performance standards and the program's long-term and annual measures; and (3) demonstrate the relationship between program partner's performance standards and the program's long-term and annual measures.

Evidence:  

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: At the Federal level, all funds are obligated according to an annual spending schedule that is established at the beginning of the fiscal year. At the partner level, grantees are obligating funds at a reasonable rate.

Evidence: At the start of each fiscal year, the program establishes an Annual Spending Plan that governs the timing of all obligations and ensures that funds are spent for the intended purposes. To date, ED has only lapsed TQE funds due to a lack of quality applications, rather than from poor fiscal management. TQE grantees have obligated funds at approximately the same rate as grantees in other ED higher education programs.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This program has not yet instituted procedures to measure and improve cost efficiency in program execution. However, as part of the President's Management Agenda, the Department is implementing "One-ED" -- an agency-wide initiative to re-evaluate the efficiency of every significant business function, including the development of unit measures and the consideration of competitive sourcing and IT improvements. A "yes" answer is likely once the One-ED process is applied to this program's relevant business functions.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Department has implemented strategies to encourage collaboration and coordination between the Teacher Quality Enhancement program and other programs addressing teacher quality issues, both within the Office of Postsecondary Education and across the Department as a whole.

Evidence: ED has convened a high-level working group to develop common performance measures for its teacher quality programs. TQE's new performance indicator is a result of this effort. TQE is also part of a separate cross-cutting team at ED which is collaborating on other critical teacher quality issues. Within the Office of Postsecondary Education, the two programs that most directly address teacher quality (Teacher Quality Enhancement and Preparing Tomorrow's Teachers to use Technology) have been placed under the leadership of a single Federal manager, in order to further encourage collaboration and coordination.

Does the program use strong financial management practices?

Explanation: No internal control weaknesses have been reported by auditors. Plus, the Department has a system for identifying excessive draw downs, and can put individual grantees on probation which requires ED approval of all grantee draw downs.

Evidence: N/A

Has the program taken meaningful steps to address its management deficiencies?

Explanation: A recent GAO report on the program identified important management deficiencies and the program has taken a number of meaningful steps to address these deficiencies.

Evidence: The GAO report (GAO-03-6) found that the program did not have an effective system for communicating program information to grantees. In response, ED has improved communications efforts in a number of areas, including hosting two national grantee meetings for program participants.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Independent peer review panels are used to score and rank all applications.

Evidence: Program funds are used to pay for the peer review process. 100 percent of grants are subject to peer review.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: In reviewing program management, the program has concluded that current oversight practices do not provide staff with sufficient knowledge of grantee activities. The program office has developed a plan to ensure that there is sufficient oversight of grantee activities. However, implementation of this plan has not yet been completed.

Evidence:  

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: GPRA data are now reported in several formats, including on the Department's website. Basic award information on awardees and grant amounts is also available on the Department's web-site. However, this publicly available information is not performance related. As a result, ED has begun to revise TQE's Annual Performance Reports, in order to provide more useful performance data.

Evidence: This data will include information on the program's annual and long-term measures.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: The program has recently developed a long-term performance goal and is working on establishing a second long-term measure. However, data are not yet available for this new long-term PART measure.

Evidence: The revised TQE Annual Performance Reports should begin to provide baseline data on the long-term measure within the next year.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has recently developed an annual performance goal and is working on establishing a second annual measure. However, data are not yet available for the this new annual PART measure.

Evidence: The revised TQE Annual Performance Reports should begin to provide baseline data on the annual measure within the next year.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The Department is working with OMB on developing an appropriate efficiency measure for this program.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: There is no comparable data available to compare TQE with other Federal teacher quality programs.

Evidence: ED may be able to make some comparisons between teacher quality programs in future years as performance measures for these programs are implemented.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The Partnership program evaluation is currently under way and will not be completed until FY 2006. At that time an assessment of effectiveness will be available. However, an initial report of the evaluation does note that grantees find that partnerships and alliances formed through the grants have had positive impacts in improving the quality of teacher preparation in those cases.

Evidence: A recent GAO report also noted that TQE grantees have formed meaningful partnerships that appear to have had positive impact in improving the quality of teacher preparation amongst its partner IHEs.