DEVELOPMENT OF OUTCOMES ASSESSMENT PLANS
(DRAFT FOR DISCUSSION)

BACKGROUND
The current emphasis on outcomes assessment is the result of changing views over the past decade and a half about academic quality and effectiveness from an almost exclusive pre-occupation with "inputs" and "processes" to a more mission specific focus on outputs and "outcomes." In the past, quality was measured by inputs such as the academic preparation of incoming students, quality and reputation of faculty, the number of books and other learning resources available to students, and processes, such as programs offered, curricular requirements, and student support services. During the 1990's, however, colleges and universities have come under increasing pressure to demonstrate that they provide added ''value'' to their students, and, in the case of public institutions, contribute to the state economy. The national trend is now toward a results-oriented concern for educational outcomes. Assessment of student learning, in particular, has become the focus.

The emphasis on assessment is driven to a large extent by increased pressure from the federal government, the states, and by accrediting associations for institutions of higher education. In 1988, the U.S. Department of Education required accrediting agencies to ask institutions to (1) specify their educational goals and (2) conduct student assessment to determine whether they are achieving these goals.¹ During the 1990's, the accrediting agencies required outcomes assessment to be included in the self-study report of all institutions seeking accreditation. More recently, the New York State Education Department has developed mandates for student assessment.

Currently, our accrediting body, the Middle States Commission on Higher Education (MSCHE), requires that the self-study reports of institutions undergoing re-accreditation include a comprehensive plan for assessing effectiveness. The Commission specifies two partly overlapping types of assessment:

• Assessment of Institutional Effectiveness: evaluation of the institution's a) overall effectiveness in meeting its goals, b) effectiveness in assuring that students achieve the appropriate learning and other outcomes, and c) efficiency in the use of resources.
  
  
• Assessment of Student Learning: demonstration that a) students have knowledge, skills and competencies that are consistent with the institution's goals meet appropriate higher educational standards.
  
  
• MSCHE does not prescribe any particular assessment process. It gives institutions much latitude to develop plans that work for them. However, it requires that
1. assessment focus on whether the stated goals and objectives of the institution are met
2. that student learning is an integral component (even primary focus) of the assessment process, and
3. institutions demonstrate that they actually use the results of assessment to improve themselves.

OUTCOMES ASSESSMENT FOR INSTITUTIONAL EFFECTIVENESS
Assessment of outcomes, while driven by the need to meet accreditation requirements, can be a powerful tool in enhancing the effectiveness of New School University. If done thoughtfully, assessment will provide information on how effective we are in meeting our goals and point to areas in need of improvement. If it is built into strategic planning, which is underway, assessment can be used to enhance academic quality and to make decisions about future investments.

Outcomes assessment has met with resistance by faculty and others, who question its relevance (applicability to the classroom), validity (truly measuring learning outcomes), proportionality (institutional benefits compared to the effort devoted to it) and significance (answer the question: So what?)² However, the assessment movement has brought to the surface important questions about what students actually gain from their experiences at colleges and universities.

The diagram below illustrates the role of assessment in institutional effectiveness and improvement³:

click on graph for larger image

As the diagram shows, assessment within the institutional effectiveness model, consists of asking:

• What are we trying to accomplish?
  Mission and goals
  
• How will we get there?
  Plan/strategies--mechanisms for achieving the mission
  
• How do we know that we have accomplished what we set out to accomplish?
  Assessment--establish indicators of success, set benchmarks
  Evaluation--establish measures for the indicators of success, collect data
  
• What changes do we need to make, based on the findings of the evaluation?
  Adjustments (if necessary)-make appropriate changes in plan that will lead to improvements.

Proposal for Development of Assessment Plans at New School University

• Assessment will be incorporated into strategic planning. In effect, we will adopt the institutional effectiveness model, where the unit of analysis is the academic program.
  
  
• Define outcomes assessment as the process in which academic divisions/programs identify the most appropriate objectives for specific programs, e.g. general education, academic majors/concentrations, and then examine whether they are achieving the desired stated objectives.
  The purpose of assessment will be to produce feedback to the schools and academic programs allowing each unit to improve its programs and increase student learning. It is not an evaluation of individual students, faculty, or staff members.
  
  
• Different plans will be developed for academic divisions, since the mission and goals of each school are different. Eventually, where appropriate, we will develop program/department-specific plans as first step in instituting a systematic program review process.
  
  
• The first focus of assessment will be student outcomes. Eventually, assessment will include other dimensions, with the ultimate goal that the program review become the assessing mechanism.
  

Steps in Development of Assessment Plans

1. The process of outcomes assessment starts with clear statements regarding what we try to accomplish (mission statements). We have made progress in developing mission statements through the Middle States Steering Committee. In their final form, mission statements should include language about what students are expected to learn or what graduates are expected to know.
   
   
2. The next step is the definition of major goals for each division, a process already in progress. Mission and goals must include language on student learning. The Director of Institutional Research will work with school-based subcommittees in developing goals that address student learning. It is crucial that schools/departments and, especially, faculty have considerable input in defining important goals of their schools/programs and what students should know when they graduate.
   
   
3. Identification of indicators (measures) to assess progress toward achieving the goals.
   
   
4. Identification of methods for assessing student achievement at important stages of the program. Decisions need to be made as to when to assess (e.g. upon entrance, during senior year, etc.) and who will do the assessing. Questions that must be considered are: what staff, financial, IT support is there for carrying out an effective assessment plan?
   
   
5. Determine how the results will be used for program improvement.
   
   
6. Develop timetable for accomplishing the previous steps. Each unit will need to create a schedule for selecting, implementing, and using the results of assessment strategies. Please note: the plan does not need to be implemented in one or two years; we should aim for a multi-year plan.
   
   
7. Implement assessment plans and revise as needed.

Timetable for Development of Plans--NOT FOR THEIR IMPLEMENTATION

November	Develop of major academic program goals for schools.
December	Finalize definition of major goals.
January	Develop measures for achievement of goals.
February	Determine measuring instruments to use, strategies for data collection; decide how data will be used in informing future policy decisions.
March	Develop timeline for implementation of assessment plans. Complete assessment plan for inclusion in self-study report.
April-May	Revise, begin implementing plan.

Some Issues that Must Be Resolved

• Who will be the assessing what? What do schools and what does the university assess?
  
  
• What staff, IT, and financial resources will be required? Will there be sufficient support?
  
  
• What will be the pattern of authority and responsibility for administering the process?
  
  
• How will we assure that assessment information is used in educationally-related decisions?
  

Footnotes:

¹ Peterson, M. L., et. al. (1999) Designing Student Assessment to Strengthen Institutional Performance in Baccalaureate Institutions (National Center for Post-Secondary Improvement). Stanford, CA: Stanford University, NCPI.

² Strada, Michael. J. "Assessing the Assessment Decade: Why a Gap Between Theory and Practice Fuels Faculty Criticism." Paper presented at the Association for Institutional Research conference, Long Beach, CA, June 2001.

³ Adapted from Alan Sturz: "A User's Guide to Institutional Effectiveness." Paper presented at the North East Association for Institutional Research conference, Philadelphia, PA, November 1998.