With some 700,000 civilian employees, the Department of Defense is the single largest employer of civil service workers in the U.S. government. At the request of the DoD Office of the Chancellor for Education and Professional Development, RAND undertook a study to examine the approaches used to evaluate academic quality and productivity in a variety of postsecondary education and training contexts, including corporations, state governments, and universities.
The study then considered which approaches might be most relevant to the Chancellor's office.
Recommendations are based on a broad review and synthesis of the related literature and documentation from organizations engaged in education assessment, as well as information gathered through expert interviews, conferences, and site visits to exemplary organizations.
The report identifies four general approaches to assessment and discusses the strength and weaknesses of each in different settings.
The study also considers the overall context for assessment and recommends that the DoD develop a clear link between education and professional development on the one hand and the overall mission of DoD on the other. In addition, it interviewed assessment experts and visited organizations that conduct such evaluations.
NDRI identified several ways that the Chancellor might structure such evaluations.
He or she can allow individual DoD institutions or programs to review the quality and productivity of their efforts, with no outside involvement.
Such a review process can be monitored by the Chancellor's office or a third party.
Alternatively, the Chancellor's office can take on the entire review process itself or turn it over to an outside organization. Or the Chancellor can focus not on the institutions or programs but on their outcomes and attempt to measure and certify student achievement.
Each approach has strengths and weaknesses; each works well in a different setting.
But given that the DoD system of education and professional development is highly complex and decentralized and that the Chancellor's office has little formal authority over the organizations providing courses, the most promising assessment approach would involve intermediaries, NDRI found. Such intermediaries would be responsible for evaluating the processes that individual departments and schools employ to assess the quality and productivity of their educational efforts.
These intermediaries could be other DoD entities or non-DoD organizations.
This practice would be similar to quality improvement efforts that have been used in the business world for the last 25 years and that were adopted by the International Organization for Standardization (ISO) in the 1980s to certify that manufacturing companies worldwide adhere to certain quality standards.
This practice also would be similar to academic audits that increasingly are being used in other education settings.
Such audits typically are conducted by intermediary organizations and focus on assuring that providers of education have effective processes in place to measure their own quality and engage in ongoing self-improvement.
The study also recommended that the Chancellor's office be responsible for more than just assessments of existing education and development efforts. Following the lead of many corporations and educational institutions, the Chancellor's office should develop a clear link between education and professional development on the one hand and the basic mission of the DoD on the other, the study suggests.
In practice, this would mean that the Chancellor's office should advocate for the development of a central learning organization within the DoD that would be modeled after a corporate learning organization or state higher education coordinating board.
Such a move would be challenging: it would require high-level DoD support and substantial collaboration among a range of stakeholders, including other organizations within the Defense Department responsible for workforce planning and personnel policies.
