Detailed Information on the
Training and Advisory Services Assessment

Implement follow-up surveys to obtain information on the quality, relevance, and usefulness of program services.

Implement the efficiency measure and continue work to establish and implement at least one additional efficiency measure.

Develop long-term performance goals to assess program's effectiveness.

Use the findings from the customer satisfaction survey annually to identify areas in need of improvement and incorporate these changes into the monitoring plan to improve grantee performance and accountability.

Measure: The percentage of customers of Equity Assistance Centers that develop, implement, or improve their policies and practices in eliminating, reducing, or preventing harassment, conflict, and school violence.

Explanation:This new measure, approved by OMB, was announced in the notice of the FY 2005 competition for new Equity Assistance Centers published March 1, 2005. Data for both measures published in the notice will be collected through annual performance reports and a customer satisfaction survey.

Measure: The percentage of customers of Equity Assistance Centers that develop, implement, or improve their policies and practices ensuring that students of different race, sex, and national origin have equitable opportunity for high-quality instruction.

Explanation:

Measure: The percentage of customers that report that products and services they received from the Equity Assistance Centers are of high quality.

Explanation:Measure of quality of recipient services and products. Data for the measure will be collected through annual performance reports and a customer satisfaction survey to be administered for the first time in 2006.

Measure: The percentage of customers that report that the products and services they received from the Equity Assistance Centers are of high usefulness to their policies and practices.

Explanation:Measure of usefulness of recipient services and products. The measure, approved by OMB, was announced in the notice of the FY 2005 competition for new Equity Assistance Centers published March 1, 2005. Data for the measure will be collected through annual performance reports and a customer satisfaction survey to be administered for the first time in 2006.

Measure: The percentage of grant funds carried over in each year of the project.

Explanation:

Is the program purpose clear?

Explanation: The program is designed to provide technical assistance and training to schools on issues related to desegregation and to ensure that all children, regardless of race, gender, or national origin, have equal access to a quality education.

Evidence: 34 CFR 272.1 "This program provides financial assistance to operate Desegregation Assistance Centers to enable them to provide technical assistance (including training) at the request of school boards and other responsible governmental agencies in the preparation, adoption, and implementation of plans for the desegregation of public schools, and in the development of effective methods of coping with special educational problems occasioned by desegregation."

Does the program address a specific and existing problem, interest or need?

Explanation: Data and numerous studies indicate that equal access to a quality education and discrimination remain problems.

Evidence: Studies document continuing gaps in access and educational attainment as evidenced by data on low performing schools, NAEP test scores, and statistics on harassment and hate crimes.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: This is the only Federal program that focuses on providing technical assistance and training to ensure that children, regarldess of race, gender or national origin have equal access to a quality education.

Evidence:  

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence indicating that the structure of the program is a flawed design that would limit the program's effectiveness.

Evidence: The design of the program, 10 Equity Assistance Centers serving different regions of the country, allows technical assistance to be provided nationwide while also being responsive to specific needs in different parts of the country.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The design of the program, 10 Equity Assistance Centers serving different regions of the country, allows technical assistance to be provided nationwide while also being responsive to specific needs in different parts of the country.

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Department is working on developing long-term performance measures across a number of technical assistance programs.

Evidence:  

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Department is working on developing targets and times frames for long-term measures across a number of technical assistance programs.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has adopted three annual measures (common to all Education technical assistance programs) for 2006 to measure the quality, relevance, and utility of program products and services.

Evidence:  

Does the program have baselines and ambitious targets for its annual measures?

Explanation: No baseline data is available. A new indicator was put in place for FY 2004 and additional indicators are being developed at this time.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All grantees are operating under a newly revised Cooperative Agreement and provide information on an annual basis.

Evidence: Cooperative Agreements were aligned to the No Child Left Behind legislation and were designed to ensure grantees focus on the program purpose and performance measures.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department does not plan to conduct formal evaluations of this program. In recent years the program has carried out two customer satisfaction surveys.

Evidence:  

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not matched with changes in funding levels. The program does not have valid and reliable performance information to assess the impact of Federal investment. However, ED has satisfied the second part of this question in that the Department's budget submissions show the full cost of the program (including S&E).

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Department has developed common performance measures across technical assistance programs that will be applied to the new Comprehensive Centers program.

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Department collects annual performance reports to oversee grantee performance, and staff talk with the EAC's on a regular basis. In addition, two customer satisfaction surveys examined if beneficiaries were satisfied with the services they received and if they reviewed or changed their policies as a result of EAC services.

Evidence: In the 2000 survey, administered by the Policy Studies Associates, three-fourths of respondents were very satisfied with the services they received, and 98 percent reported that they reviewed or changed policies as a result of the EAC service. Due to a variety of circumstances, the response rate for the 2001 survey was too low to use as a measure of the program.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Currently, ED cannot demonstrate how federal managers and program partners are held accountable for program goals. ??However, the Department has initiated several efforts to improve accountability in its programs. ??First, ED is in the process of ensuring that EDPAS plans -- which link employee performance to relevant Strategic Plan goals and action steps ' hold Department employees accountable for specific actions tied to improving program performance. ??ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance.?? Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results.

Evidence: The President's Management Agenda scorecard (Human Capital and Budget & Performance Integration initiatives) notes ED's efforts to improve accountability. ??The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated within the timeframes set out by Department schedules and used for the purposes intended.

Evidence: Program staff monitor to make sure that grantees are drawing down funds at an acceptable rate.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: To date, the Department has not established procedures for this program to measure and achieve efficiencies in program operations. ??However, ED is in the process of developing its competitive sourcing Green Plan, and is working to improve the efficiency of its grantmaking activities. ??The Department has also established a strengthened Investment Review Board to review and approve information technology purchases agency-wide. ??

Evidence: Department Investment Review Board materials. ??ED's Discretionary Grants Improvement Team (DiGIT) recommendations.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Equity Assistance Centers work with a number of federal entities as well as in collaboratives with other State and local entities.

Evidence: EAC's work with the Office of Civil Rights helping schools with desegregation and equity issues. The EAC's have also collaborated with the Department of Justice on school violence and harassment issues.

Does the program use strong financial management practices?

Explanation: Recent agency-wide audits have not identified deficiencies in the financial management of this program.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Major internal management deficiencies have not been identified for this program.

Evidence:  

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Independent peer review panels are used to score and rank all applications.

Evidence: While the same 10 centers received funding in the last two competitions there was a clear competitive process and a limited number of eligible entities from which to draw.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Department maintains information on grantee activities through annual reports, meeting with EACs, and technical assistance activities.

Evidence: The program's monitoring efforts focus on the review of annual performance reports as well as quarterly meetings, conference calls, and regular telephone contact with grantees.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: While the program collects grantee performance data on an annual basis, the information has not been made available to the public. Education is developing a department-wide approach to improve the way programs provide performance information to the public. In 2004, Education will conduct pilots with selected programs to assess effective and efficient strategies to share meaningful and transparent information.

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Department has not yet established long-term performance goals for this program.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: Annual performance goals are currently being developed.

Evidence:  

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department has not yet developed appropriate efficiency measures for this program.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: No data are available for comparable programs.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Although the program has been proactive in soliciting customer satisfaction information to guide management, no rigorous evaluations have investigated program performance.

Evidence: The absence of a program evaluation highlights a broader issue underlying the evaluation of very small competitive grant programs. In this case, several customer satisfaction surveys served as a good proxy for understanding elements of program efficacy. The benefits of a rigorous evaluation should be weighed against cost and the known utility of customer satisfaction survey data, especially in the context of a program with such modest administrative funds.