OVERVIEW

In this report, the University Task Force on Assessment offers a number of recommendations for improving assessment activities at Oakland University. Before describing our recommendations, however, we have listed the goals and objectives that guided the work of the task force and have provided background information to support the recommendations we will be making. Included in the background information are: history of assessment activities, problems encountered, faculty perceptions of assessment, and results of the NCA accreditation visit to Oakland University in 1999. Recommendations include changes and additions to the current assessment structure and process, the establishment of an assessment center, suggestions for implementation, and additional recommendations. Financial considerations are addressed briefly at the end of the report.

GOALS AND OBJECTIVES OF THE TASK FORCE

The University Task Force on Assessment is charged with (1) addressing the concerns of the NCA (North Central Association of Colleges and Schools), which will be returning to Oakland University in 2004 for a focus visit on assessment, and (2) exploring ways in which the assessment process can be used to enhance the quality of student learning at Oakland University. The goal is to have a process that allows the academic units to ensure quality while providing timely information for program improvement. Streamlining the process so that it can be more easily integrated into the ongoing work of the department will be one focus of the task force.

Additional priorities from the Office of the Vice Provost for Undergraduate Education:

The task force needs to (1) study the uneven implementation of the assessment process across campus, one of the major concerns cited by the NCA; (2) make recommendations about how to address the situation; and (3) suggest a plan to be put forward to the Provost for funding. The task force needs to address the development of a process that actually works for the departments, so that the process is used on an ongoing basis to assess student learning outcomes, and the information gathered is used to inform curricular revisions.

Goals Identified by the Task Force

Primary Goal: The task force will explore ways that the assessment process can be better utilized to improve academic programs.

Strategic Goals: The task force will propose:

a) means for changing the current assessment process to improve implementation of unit and program assessments.

b) mechanisms for increasing university-wide administrative support for assessment.

c) ways to improve faculty interest in assessment activities and increase support for the goals of assessment and its importance for program development.

d) ways to improve knowledge of assessment techniques.

e) means for providing consultation resources for individual units and programs, including consultation on assessment methods, interpretation of results, making changes, and statistical analysis.

f) a mechanism for a centralized location and the staff necessary for providing assessment services and resources to units and programs.

BACKGROUND INFORMATION

In 1989, NCA introduced its Assessment Initiative by issuing its first "Statement on Assessment and Student Academic Achievement" (L�pez, 1999). The Assessment Initiative specified that, as a criterion for accreditation, each member institution would plan and implement assessment of student learning. NCA spelled out its belief that affiliated institutions must be "student-centered learning organizations committed to continuous improvement in the quality of the education achieved by their students (Lopez, 1999, p.4)." Therefore, following the 1989 comprehensive evaluation by NCA, Oakland University established a standing Senate committee, the University Committee on Assessment, and developed a University Assessment Plan, which was adopted in 1992. The plan was reviewed during a 1994 NCA focus visit, and annual reporting from participating units and departments went into effect in the Fall of 1995. Staff support for assessment activities would come from the Office of Institutional Research and Assessment (formerly the Office of Institutional Research), and the Assessment Committee would receive an annual allocation of $20,000 for activities to support assessment.

Under the University Assessment Plan, assessment was decentralized: each unit or department was asked to develop and implement its own assessment plan. It is important to note that along with decentralization, faculty and administration made the decision that assessment activities would be designed and implemented by department faculty, and administration took a "hands-off" position to facilitate faculty autonomy.

The University Committee on Assessment recommended that plans: (a) formulate program goals and learning objectives that are consistent with the Mission Statement of Oakland University, (b) specify how learning objectives would be measured, and (c) assign responsibility for implementation. Approximately 50 plans were developed by units and departments and approved by the University Committee on Assessment between 1992 and 1994. Annual assessment reports were to be sent to the Assessment Committee for review, and copies were sent to department chairs and deans of the relevant school or college (or the Assistant Vice President for Academic Affairs in the case of the BGS program [now the Vice Provost for Undergraduate Education]). The University Committee on Assessment would review annual reports and provide written feedback to each department or unit.

Problems During the First Decade

Over the course of the early years of mandated assessment, several problems surfaced. First, annual reports were not consistently received or received on time by the University Committee on Assessment. We believe that there were a number of reasons for this: lack of knowledge of the constructive goals of program assessment, a low degree of faculty commitment to assessment activities, inconsistent feedback from the University Committee on Assessment, lack of department resources, and lack of accountability. Many units put assessment on the back burner. When the first reports were received, it was clear that most departments and units were putting forth only minimal effort in their assessment activities. Despite extensive feedback from the University Committee on Assessment, few departments responded to committee feedback or improved their activities.

A second major reason for inconsistent reporting stemmed from the composition and duties of the University Committee on Assessment. Because of negative perceptions of assessment and the high workload involved in reviewing annual assessment reports, serving on the University Committee on Assessment has not been a welcome service activity. In addition, most committee members have come to the committee with little or no expertise in assessment, and reviews of annual reports have been inconsistent from year-to-year. To complicate this problem, formal training was not provided to committee members; instead, members simply used their own commonsense judgments to assess annual reports from departments and units. Annual reports were either reviewed by the whole committee or, in recent years, by one of two subcommittees. As a result, one year’s review might inform a department that they were doing a good job; whereas the next year’s review might conclude that the same department needed to completely revamp its assessment activities. Not surprisingly, this engendered a further slow-down in progress toward satisfactory program assessment. And, despite the fact that many departments and units did not provide annual reports to the committee as requested, the workload of the University Committee on Assessment has been extremely high. As a result, the committee frequently has been unable to provide feedback in a timely manner.

The University Committee on Assessment was further hampered in its efforts to provide consistent feedback because the criteria for what constituted satisfactory assessment activities changed over the years. As assessment became a topic of national conversation among educators and accrediting bodies, the standards were raised. Early assessment plans approved by the committee were largely based on surveys and interviews--self reports of student and alumni perceptions of their satisfaction with degree programs and their suggestions for change. Surveys and interviews can provide useful information, but they do not measure actual student outcomes. During the last half of the 1990’s, it became increasingly clear to the University Committee on Assessment that departments and programs needed to begin to anchor their assessment activities on direct measures of student outcomes: evaluations of knowledge, skills, and performance. Unfortunately, by that time, some departments that had originally included measures of student performance in their plans (e.g., standardized tests taken on a voluntary basis), had dropped them because of sample sizes that were not large enough to be meaningful. Except for programs whose students had to pass specific tests for outside accreditation purposes, there were almost no direct measures of students’ progress toward program goals.

Departments and units were being asked by the University Committee on Assessment to incorporate reliable and valid direct measures of student performance into their assessment plans, but no resources were provided directly to the units to enable this. Nationally, the most widely used direct measures are standardized achievement tests, preferably given pre- and post-program to gauge "value added" by the program to students’ knowledge, skills, and performance (L�pez, 1999; Palomba & Banta, 1999). Because these tests are nationally-normed, they also allow universities to compare the achievement of their students with students nationwide. Other direct measures include: locally constructed tests (objective or essay or a combination) and evaluations of students’ work skills (e.g. performance in internships or jobs) and work products (e.g. projects and papers from capstone experiences or portfolios of students’ work in the program). To produce reliable scores, many direct measures, such as essay examinations and evaluations of projects and papers, require evaluation by multiple raters according to specified criteria. Standardized tests are expensive; locally constructed tests are difficult and time-intensive for faculty to design and pretest. Departments and programs were not given budgets or faculty release time for assessment activities. And, despite attempts to advertise the availability of funds from the University Committee on Assessment, the criteria for funding were unclear and very few units requested funds. For example, the call for proposals for funding strongly implied that activities should have university-wide relevance and evidence of matching funds from the department; this provided a negative incentive for many departments. Most departments did not have funds available for matching, and because assessment activities need to be tied to specific program goals, most assessment activities reflect the unique needs of a particular department or program.

In addition, there was no incentive to increase the quality of assessment activities. Performing assessment activities is considered "service," and service activities are generally devalued in tenure and promotion decisions, and they fall low on the list of activities that qualify for merit pay. Finally, there was no accountability. Assessment activities were largely relegated to one or a few members of a department (most always personnel untrained in assessment), and department chairs usually did not share in the responsibilities for improving assessment. In most departments, assessment was rarely even discussed by the faculty as a whole. The decision to give faculty complete autonomy in designing and implementing their assessment plans meant that academic deans were also uninvolved. This created the illusion that the entire university supported a laissez-faire approach to assessment. As a result, feedback from the University Committee on Assessment was largely ignored.

According to a review in progress of assessment activities across the university, the number of departments and units that are engaged in high-quality program assessment at the present time probably falls somewhere between ten and fifteen percent of those with assessment plans. Most (but not all) of these are professional programs with outside accreditation requirements whose standards also have been raised to be consistent with those of accrediting bodies for institutions of higher education. Currently, the University Committee on Assessment has slightly over fifty plans on file; however, Oakland University has about seventy-five undergraduate programs and over fifty graduate programs. These numbers indicate that most of our programs have not yet developed assessment plans and, consequently, are not yet engaged in any official assessment activities. This level of progress suggests that we are behind the curve in our assessment activities (Lopez, 1999).

According to criteria argued by Lopez, (Associate Director, North Central Association of Colleges and Schools, Commission on Institutions of Higher Education), universities can be placed into one of three progressive levels of attainment in assessment. At level one, Beginning Implementation, institutions "have still not agreed on effective means to assess student learning in general education competencies and skills or to document learning across each of their majors or any other academic programs (p. 7)." At level two, Some Implementation, institutions "have a number of the essential components of an assessment program in place, demonstrating that assessment and improvement of student learning are becoming an institutional priority (p. 7)." Level three institutions, Ongoing Implementation, have mature assessment practices in place; assessment activities contain the important elements described by NCA evaluators; faculty regularly recommend and implement curricular changes based on assessment activities and track the effects of curricular changes on student learning (p. 7). Based on these criteria, it is difficult to place Oakland higher than level one. Most units and programs are not adequately measuring students’ learning outcomes; general education assessment is still a "work in progress." Almost no departments or programs have really used results from assessment of student learning, performance, or achievement to change and improve their programs--the major goal of outcomes assessment--and, to date, not a single one has assessed program changes or improvements resulting from assessment activities. A few departments and programs have made small changes in course availability or course content, but these changes have been based on faculty perceptions or results from student satisfaction surveys.

Assessment activities at Oakland University (assessment plans and implementation of activities) are getting better overall, but progress is very slow. Aiding departments in the creation and implementation of better assessment activities requires that someone trained in assessment techniques work one-on-one with assessment representatives from departments and programs to design procedures that capture students’ progress toward the goals and objectives of their specific program. Individualized attention produces results. A number of excellent new plans that include direct measures of student performance have recently been approved by the University Committee on Assessment, but these plans are just beginning to be implemented. It will be several years before they bear fruit. And, most units still need to create a satisfactory assessment plan. To allow for more attention to individual unit and program assessment activities, the University Committee on Assessment moved to a (temporary) staggered three-year reporting schedule in 1999-2000. The new schedule has allowed the committee to attempt to catch up with overdue reviews of annual reports.

Oakland Faculty Perceptions of Assessment

In 1999, the University Committee on Assessment designed a survey, which was sent to current and former department assessment representatives and current department chairs, asking for their perceptions of assessment activities. The survey included both structured and open-ended questions. The response rate was only 45% (n = 39). The full report is available from the Office of Institutional Research and Assessment, but we will summarize the results here because they were extremely helpful to the task force in proposing suggestions to improve assessment activities.

The first question asked how members of your department feel about assessment. The following responses were obtained:

5.1% Everyone or almost everyone has positive feelings about assessment.

17.9% The majority is positive about assessment

30.8% About half are positive and half are negative

20.5% The majority is negative about assessment

10.3% Everyone or almost everyone has negative feelings about assessment

15.4% No response

In summary, 31% said their department is split; 23% said that most of their department is positive; 31% said that most of their department is negative.

When asked to explain this answer, positive feelings were explained by responses such as:

"Feedback on our programs is considered important."

"Accountability is everywhere, the order of the day."

"Keeps us on our toes. We know where we are."

Negative feelings engendered explanations such as:

"We’re not convinced of its usefulness."

"Effectiveness of assessment is not clear to people."

"Most do not believe it is a valid indication of learning; forces ‘teaching to the test’ as in K-12."

"Just more paperwork"

"It is viewed as busy work, done because of conservative political agendas and as part of efforts at making universities ‘accountable’ in a business sense."

"View it as a potential invasion of their freedom to teach a course as they choose."

The survey also included a set of structured questions using a Likert format. Respondents were asked to rate the extent of their agreement/disagreement with a number of statements by choosing one of the following responses: strongly agree, somewhat agree, no opinion, somewhat disagree, or strongly disagree. The percentage of responses indicating some or strong agreement are in parentheses after each question.

1. Most members of my department are aware of the assessment process. (76.9%)

2. Most members of my department have no interest in assessment. (48.7%)

3. Most members of my department are not opposed to assessment as long as others do it. (51.3%)

4. Most members of my department perceive value in the assessment process. (41.0%)

5. Assessment activities are required by the university because they can contribute substantially to meeting university mission and goals. (48.7%)

6. Assessment activities are required by the university mainly to satisfy external accreditation demands. (82.1%)

7. Assessment activities are required by the university and external accreditors to conform to a current fad in higher education. (69.3%)

8. The university should value faculty work in assessment in terms of promotion/tenure/salary. (59.0%)

9. If the university valued assessment it would provide faculty grants or release time to support it. (66.7%)

The responses to the questions above indicate that for this sample of faculty, assessment is not perceived to be a valued or valuable activity. The responses to questions 4, 6, and 7 are particularly problematic because they suggest that chairs and assessment representatives (arguably those most invested in doing assessment) believe that neither faculty nor university administration truly values assessment and that Oakland University is just going through the motions to satisfy outside accreditors.

In addition, 61.5% reported great difficulty in implementing their assessment plan; 10.3% reported some difficulty; 17.9% reported no problems, and 10.3% gave no response. When asked to explain their difficulties, typical responses included:

"Coming up with good assessment tools and implementing them."

"Agreeing on the instrument."

"We have fallen behind on external [alumni] surveys…and have been disappointed in the rate of return."

"We have too many assessments: University, NCA, NCATE, MDE, our internal methods."

"Faculty have suspicions about motivation behind assessment."

"Assessment is time consuming for this extremely understaffed department."

"Money and time to carry out activities."

"We need proper funding to do the job right…"

Overall, the responses to the faculty assessment survey were entirely consistent with attitudes and opinions of university faculty nationwide (Ewell, 1997; Lopez, 1999; Palomba & Banta, 1999, among others), and they provided much useful information for the task force.

Results of the 1999 NCA Visit

To quote from the Executive Summary of the 1999 NCA report:

"In the abstract, the recommendations of the Assessment Committee were sound. The committee worked heroically with departments to assist them in developing their plans. Most departments cooperated, albeit some only grudgingly. In the concrete, however, the assessment of student academic achievement is not working as well as it should…and they should be farther along (p. 37)."

The evaluation team also cited an over-reliance on survey methodologies that "employ self-reports to measure student perception rather than actual achievement (p. 38)." They stressed a need for the inclusion of direct measures of learning outcomes in assessment plans, such as the use of standardized tests and employer surveys to measure "actual student accomplishment or ‘value added’ during the educational process…(p. 38)"

They also noted that some assessment techniques used by professional schools (Nursing and Health Sciences, in particular) could be used as models for the balance of the university. These schools use a blend of direct and indirect measures to assess student learning and achievement in their programs. NCA strongly urged Oakland University to "get on with it (p. 38)."

The NCA team also noted a disconnection between academic administrators and the assessment process. They cited a failure to allocate resources and prioritize activities as well as to modify budgets to address issues raised by assessment. They also made suggestions that the assessment process should be more formal and that chairs and deans should become more directly involved in accountability of departments and programs. Finally, they recommended that the academic administration find ways to assure faculty of the diagnostic and constructive nature of assessment and to allay faculty uneasiness and suspicion about the use of assessment results in a punitive way toward departments or particular individuals.

Oakland University received generally high marks from the NCA accreditation team, but assessment activities will be part of a focused visit in 2004-2005 to look at both general education and the assessment of student academic achievement. In terms of assessment, they will be looking for answers to the following two questions (p. 51):

1. Are student learning outcomes regularly and properly assessed within the program?

2. Are assessment findings informing university planning and budget processes and thus leading to improvements in student learning outcomes.

As a result of the problems identified by the NCA team, the Provost empowered the University Task Force on Assessment to investigate problems in the assessment process and to recommend solutions. The task force convened in a series of meeting from April, 2000 to February, 2001.

TASK FORCE RECOMMENDATIONS

The task force has organized its recommendations into four sections: Structure and Process, Resources, Implementation, and Further Recommendations.

Structure and Process

The purpose of conducting assessment of students’ academic achievement is to use the results to maintain and/or improve the quality of university programs. Assessment is a dynamic process in which results of evaluations of students’ knowledge, skills, and performance feed back into the program to inform curricular changes, and the changes are evaluated to see if they resulted in intended improvements. In order to facilitate this feedback loop, the task force recommends a more formal structure and set of procedures to improve communication and accountability. The present structure (described in the first section of the report) is problematic for several reasons:

1. The lines of accountability and the link to needed financial resources are absent or unclear.

2. The University Committee on Assessment cannot handle the volume of reports and other assessment-related activities (for example, reviewing assessment reports and providing feedback; working with individual units to improve their plans and procedures; reviewing and providing feedback for proposals for new activities; planning educational and other training opportunities).

3. A university-wide committee lacks knowledge of individual university programs and the outside accreditation requirements of professional programs, resulting in duplication of effort.

Therefore, we are suggesting a structure that contains the following elements:

 1. The formation of a Curriculum and Assessment Committee within each department, unit, or program and the designation of an individual to be the Department Assessment Coordinator.

•It would be this committee’s responsibility to design or modify the assessment plan of the academic unit and to conduct ongoing assessment activities after consultation with the entire department or program faculty. Assessment plans would be required for each undergraduate and graduate program awarding a degree.

•Assessment plans should contain each of the following elements:

a) Citation of appropriate goals from Oakland University’s mission statement.

b) Specification of academic unit goals flowing from each of the mission statement goals cited.

c) Operationalization of the unit’s goals into objectives for student learning that include specific knowledge, skills, and achievement expectations.

d) A full description of the methods by which progress toward the operationalized unit goals will be measured, along with copies of all assessment instruments: tests, surveys, criteria used for portfolio analysis, etc.

e) A list of individual(s) who have primary responsibility for implementing assessment activities (the Department Assessment Coordinator and any others).

f) A complete description of the procedures used in an academic unit for translating assessment results into program changes, including procedures for evaluating program changes.

•The Curriculum and Assessment Committee and department chairs or program directors would disseminate assessment results and feedback from College/School and University Committee on Assessment reviews to program faculty for discussion on a semi-annual (or more frequent) basis. (These committees and their activities are described in detail in part 3, below.)

•The Curriculum and Assessment Committee would have the responsibility for translating faculty discussions into modifications of assessment plans and proposals for curricular changes, as well as designing and implementing evaluation of these changes.

•The Curriculum and Assessment Committee would write up a formal assessment report for external review every two years using a standard outline format provided by the University Committee on Assessment. The report would include a report of expenditures and a proposed budget, including the rationale for funding requests, for the next two years. A brief status report would be forwarded to the College or School Assessment Committee in alternate years.

•The Curriculum and Assessment Committee would be responsible for responding to feedback from reviews of bi-annual reports.

2. Chair’s (or program director’s) responsibility for keeping assessment activities on track.

•The chair would meet with the Curriculum and Assessment Committee on a regular basis to ensure timely implementation and reporting.

•In conjunction with the committee, the chair would lead faculty discussions of (a) program goals and objectives, (b) assessment results and how they will be used to improve the curriculum, and (c) responses to feedback from reviews.

•The chair would write an executive summary of department activities (i.e., meetings with the committee, faculty discussions) to be attached to bi-annual assessment reports and alternate-year status reports.

•The chair’s office would manage and coordinate the assessment budget of each academic unit and provide information needed for the bi-annual report.

3. Another layer of external review would be established—School or College Assessment Committees. We recommend that an assessment committee should be formed within the School of Business Administration, the School of Education and Human Services, the School of Engineering and Computer Science, the School of Health Sciences, and the School of Nursing; and three committees should be formed within the College of Arts and Sciences (Sciences, Social Sciences, and Humanities). These committees would be responsible for initial review and approval of assessment plans and review of bi-annual reports and budget requests (also see Resources Section below).

•The College and School Assessment Committees would provide feedback to Department Assessment Coordinators and chairs of academic units about their assessment plans and their assessment reports (functions currently handled by the University Committee on Assessment). These committees would also follow-up on actions taken on their recommendations to the units. The committees would be provided with a budget and clerical support for these and other listed activities.

•Each School and College Committee would include an academic administrator, such as an Associate Dean, to (a) facilitate communication between faculty and their dean and (b) provide a system of accountability in which chairs are responsible to their deans for satisfactory assessment activities.

•The School and College Committees would forward summaries of budget recommendations and reviews of assessment reports to the University Committee on Assessment.

•The School and College Committees would be responsible for coordinating school or college-wide surveys and other activities and would be provided with a budget and clerical help for these activities.

•The School and College Committees would include an individual who acts as a liaison to the Committee on Instruction to ensure that curricular changes requiring committee approval are backed up by actual assessment data.

•The School and College Committees would include an individual who acts as a liaison to the Committee on General Education to coordinate assessment of general education courses.

4. The University Committee on Assessment would review forwarded summaries of budget recommendations and assessment report feedback letters from School or College Assessment Committees. They would provide feedback to the School or College Assessment Committees and the University Senate.

•Membership on the University Committee on Assessment would include the Vice Provost for Undergraduate Education, who will have administrative responsibility for university assessment as well as for the Institute for Teaching and Learning.

•The director of the University Assessment Center (see Resources Section below) also would serve as a member of the University Committee on Assessment and would provide a semi-annual report of center activities to the committee and the Provost.

•The University Committee on Assessment would be responsible for selecting recipients of assessment research fellowships (see Resources Section below) and would be provided with a budget and clerical support for these activities.

•The University Committee on Assessment would review and prioritize proposals for release time or stipends for assessment activities (see Resources Section below).

•Up to $3000 would be made available to the University Committee on Assessment for the purpose of funding faculty wanting to attend assessment conferences.

5. Assessment support and record keeping would be moved from the Office of Institutional Research and Assessment to the Institute for Teaching and Learning (see Resources Section below). The task force believes that relocating support activities makes sense for two very important reasons. First, assessment is an integral part of teaching and learning—not institutional research, which is focused on tracking enrollments and student demographics. It would not make sense to have an institute devoted to the improvement of teaching and learning that did not also include assessment of student learning. Second, the task force wanted to make a strong statement that assessment should continue to be a faculty enterprise (in the same way that faculty need to be responsible for designing and modifying university curricula). By moving assessment support to the Institute for Teaching and Learning, the task force also hopes to reinforce the idea that assessment is as much a part of faculty’s self-description as is teaching and learning. Finally, co-locating administrative oversight of the University Assessment Center and the Institute for Teaching and Learning with the Vice Provost for Undergraduate Education would strengthen the importance of assessment and provide continuity of administrative support at the university level.

Relocating assessment resources to this new location will also serve to allay faculty concerns about the uses of assessment information by the administration—a real concern of many faculty members. Even though the majority of assessment activities do not identify or single out particular courses and instructors, there is always the potential for assessment information to be used in a punitive way against faculty and departments. This fear was clear from the many negative responses to open-ended questions in the assessment survey and is a common source of faculty unease nationwide (Morse & Santiago, 2000; Palomba & Banta, 1999). Increasing faculty motivation to carry out high-quality assessment requires more positive attitudes toward assessment, and changing negative faculty perceptions requires convincing faculty that they—not the administration—make the decisions for their programs based on the results of evaluations of the outcomes of their students. Improving faculty motivation and attitudes would be much easier to accomplish if resources were located in the Institute for Teaching and Learning, which is faculty-centered, rather than in the Office of Institutional Research, which is viewed as an administrative adjunct.

Resources

One of the major problems reported by faculty is the lack of resources for assessment activities. Some important resources would include ongoing budgets for assessment activities, stipends or release time for individuals involved in assessment, clerical help, and increased access to alumni databases. A strong university commitment to assessment also should include resources for bringing assessment-related speakers, trainers, and other activities to campus and support for professional development and assessment-related research (Halpern, 1987; Lopez, 1999; Morse & Santiago, 2000; Palomba & Banta, 1999). In addition, we believe that results of assessment across the university should be widely communicated to the university community, and outstanding activities should be recognized and rewarded. Finally, we believe that units and programs are in dire need of professional help to design, implement, and interpret evaluations of student outcomes. Improving assessment activities requires access to assessment consultants and help with data collection, data analysis, and interpretation of statistical results, and we believe that these activities could best be managed within an assessment center. Thus, the task force recommends that allocation of the following resources to assessment activities should be considered:

1. Budget allocations. Budgets of units and programs should include funds needed for assessment-related activities. Many units have reported difficulties meeting the financial costs of assessment, both in terms of supplies and clerical support. Reasonable costs for these activities need to be funded on an ongoing basis by building them into unit and program budgets.

2. Release time and stipends. The task force recommends that assessment activities could be greatly facilitated by funding a limited number of course releases and stipends over the next three years for faculty who want to become heavily involved in assessment activities. Because we are so far behind where we should be, the task force believes that the process needs to be jump-started by providing release time or a stipend for faculty to develop and implement assessment activities.

Faculty could apply to obtain a course release in either the Fall or Winter term or a stipend of $5000.00 to be paid during the combined Spring/Summer terms. Individuals wishing to apply for release time or stipends would submit a detailed proposal to their dean with the approval of their chair. All plans approved by the deans of the schools and college would be forwarded to the University Committee on Assessment. This committee would prioritize the proposals and forward them to the Provost for final selection and funding. We would recommend that within each unit or program, only one faculty member could apply for a course release or stipend during the three years of the program. We also would recommend that for every thirty units and programs across the university, one course release could be made available in the Fall and Winter terms, and one stipend could be made available for the combined Spring/Summer term. This would result in funding 3/30ths (10%) of the total units and programs each year for the next 3 years.

3. Research fellowships. The task force believes that faculty have not yet become aware of the opportunities to conduct and publish the results of assessment-related research. Most areas of specialization have journals that publish the results of assessment; some have entire journals related to assessment activities. We believe that research in assessment could be facilitated by two additional faculty research fellowships each year. These $7500 awards could be made by the University Committee on Assessment using procedures similar to those currently used to award other contractually-designated research fellowships.

4. Professional development leaves. The task force recommends that proposals from faculty who wish to receive training in outcomes assessment and assessment administration should be considered for professional development leaves.

5. An assessment center. The task force strongly recommends the establishment of a University Assessment Center, located within the Institute for Teaching and Learning. The center would be headed by an academic administrator. At this time, we see a clear and strong need for a full-time person in this position. We envision the director of the assessment center to be an individual who also holds a position as a tenured faculty member. At least one additional staff person, perhaps a Research Specialist (C9) would be needed to support the work of the director.

An advisory board composed of chairs from each of the School or College Assessment Committees and the University Committee on Assessment (and perhaps others, such as students and individuals from the community) would provide guidance. The center director would report to the Vice Provost for Undergraduate Education. Responsibilities of the director of the University Assessment Center would include the following:

•Individual consultation with units and programs to establish and maintain assessment activities. Consultation activities would include development and operationalization of unit and program goals and objectives, development of assessment plans and plan revisions, development of measurement techniques and measuring instruments, and assistance with data collection, scoring, data analysis, and interpretation. Consultation for individuals wishing to apply to the University Committee on Assessment for research grants or fellowships or to the Provost’s office for faculty development leaves also would be available.

•Meetings with deans and chairs groups and faculty assemblies to discuss and promote program assessment and curriculum improvement.

•Working with Alumni Relations to facilitate development, implementation, and analysis of alumni surveys requested by many units and programs.

•Developing and implementing university-wide assessment activities, including speakers, workshops, recognition programs, and communication of assessment activities to the university community. Communication would include an annual report, a website, a regular column in the Teaching and Learning Newsletter, and dissemination of information to Oakland University publications and Communications and Marketing.

•Maintaining assessment records and administering the budget of the University Committee on Assessment. The center director would take over duties of the University Committee on Assessment, handled currently by OIRA (which would be able to return to its earlier designation of OIR).

•Maintaining a library of assessment resources.

Implementation

Oakland will be communicating its progress in assessment to North Central in 2003, and the NCA evaluation team will revisit the campus for a focused visit on assessment in 2004. Because Oakland’s progress toward satisfactory assessment has been slow, the task force recommends that the changes in structure and resources be implemented as soon as possible. It would be wonderful to see it begin to take shape in the 2001-2002 academic year. We feel, however, that it is extremely important to maintain the current momentum toward better assessment activities. Therefore, it is our strong recommendation that one or two Oakland faculty members with assessment expertise be recruited to consult with individual units on improving their assessment plans until a permanent director can be hired, and the current reporting and budget structure can be formally modified. This solution, however, should be viewed as only temporary, because there is simply too much to be done on a part-time basis, particularly by faculty with research, teaching, and departmental and other service responsibilities. We are hopeful that a qualified individual from within the Oakland community will step forward to take on the responsibilities of the center director (perhaps as a step toward a career in administration or perhaps as a temporary break from teaching), although we have not yet identified such a person from within our midst. We believe that this person is essential for improving Oakland’s assessment activities, and if no one steps up to the plate, we hope that the university will consider going for an outside hire.

Education and training. The task force recognizes a strong and immediate need for education and training of faculty and others involved in assessment. It is clear from the survey results that faculty share many misconceptions about the goals and purposes of assessment. We recommend that education and training should be made an immediate goal. First, we would like to see the development of a program within the office of the Vice Provost for Undergraduate Education to bring in academic assessment experts to talk to the university community about the benefits of assessment. Faculty misconceptions need to be changed, and widespread anxiety, distrust, and fear of assessment need to be allayed as soon as possible for real progress to be made. We know from past attempts of the University Committee on Assessment and OIRA to bring in speakers to talk about assessment that it is generally difficult to attract faculty to these talks, especially when they are scheduled during the faculty workweek. One idea might be to have a series of assessment breakfasts, lunches, or dinners at Meadowbrook Hall featuring a dynamic speaker.

Second, many units and programs need immediate assistance and training in the criteria for good assessment practices. The continued over-reliance on survey techniques suggests that the requirement for direct measures of student learning, performance, and achievement still needs to be explained to most units and programs. A significant number need immediate help in redesigning their assessment plans. At the most basic level, units and programs need a user-friendly but comprehensive set of guidelines for designing assessment plans. A search of assessment publications and websites has not yet uncovered a useful, generic set of guidelines, but the search could be extended or a set of guidelines could be written for Oakland.

Finally, we also need to remain aware that most faculty are not assessment specialists; although most have commonsense notions of how students should be assessed. Unfortunately, commonsense ideas rarely satisfy the criteria set down by North Central and assessment experts (as we have discovered from the 1999 NCA visit). Many units and programs urgently need professional help. Using North Central’s criteria, many units have not conducted any meaningful assessment of students’ progress toward the goals of their programs. Others have actually decreased their activities, often dropping the direct measures from their plans and activities because they were too difficult to implement or were not working. A small number have simply refused to do any assessment or have disregarded all attempts of the University Committee on Assessment to persuade them to begin. And, there are still many university degree programs without any assessment plans. We believe that these units need to be identified and jump started. Therefore, we would like the university to consider bringing in assessment consultants to help programs that have made the least (or no) progress.

We would suggest a format similar to that used by the research office to schedule time with their grant-writing consultant, David Bauer. A unit desiring help could prepare to meet with a consultant by first identifying their program goals and objectives for student learning. Then, their assessment committee or coordinator would schedule an appointment to consult with an expert about the best methods for assessing students' progress toward meeting these objectives. In consultation with faculty, experts can often identify existing assignments and projects that can be evaluated as direct measures of students’ progress toward program goals and help to design the tools to properly evaluate them. Experts can also suggest standardized tests and measures that might be used, and they can help to design or improve surveys to evaluate perceptions of progress toward specific program goals. Small group workshops might be an alternative for units and programs whose progress is more advanced but who still need to improve their activities.

Use of assessment results to inform university planning and budgets. In an earlier section, we have described a process for including the direct costs of assessment activities in unit and program budgets on an ongoing basis. But, there is more to discuss in attempting to address the full concerns of the NCA evaluation team on this issue. From high-quality assessment, we learn more about a unit or a program than the simple test results of our students. For example, if we learn that students’ knowledge, skills, or performance fall below the criteria we have set for students in a program, then the next step is to ascertain means for bringing up their performance. Sometimes, obvious solutions come to mind; other times they do not. Factors such as sections that are too large, over-reliance on part time instructors, or lack of adequate laboratory facilities are frequently viewed as causes of inadequate student achievement. Remedies for these sorts of problems can require a financial commitment from the administration based on the results of assessment activities, as well as changes in university planning. To date, there has been little, if any, action taken to address the needs identified by units and programs in their assessment reports, and NCA is looking for this to change. It is hoped that when an assessment center is in place and when chairs and deans are more directly involved in oversight of assessment activities, information coming out of assessment activities will result in changes in planning and budgets.

In addition, faculty need to take increased responsibility for what their students are (or are not) learning. It is incumbent upon faculty to do their part in controlling costs and finding ways to improve student learning that do not require large expenditures. After all, moving more resources into assessment means that resources will be reduced elsewhere—that’s life in higher education. Fortunately, the Institute for Teaching and Learning, soon to be implemented, can be an invaluable resource for educating faculty who want to become better teachers and improve the achievement of their students at little or no cost. Simply recognizing that not all students learn in the same way is an important beginning. For example, lecture courses can be changed without much difficulty to incorporate teaching methods that reach more of our students. For assessment to result in real improvements in student outcomes, however, both faculty and administration need to commit themselves to this endeavor.

Consideration of assessment activities in tenure, promotion, and merit pay decisions. The task force also discussed the lack of rewards for individuals involved in assessment. We recommend that work on assessment should be considered a valuable activity, earning consideration in tenure and promotion decisions and worthy of merit pay. In many units and programs across Oakland University, assessment (and "service" in general) is given short shrift and considered to be of little value in determination of rewards. Clearly, this perception, often explicitly codified in department guidelines, reduces the desire of faculty members to commit their time and energy to assessment activities. Assessment activities need more reward value. There is an annual award for outstanding teaching, and there are two for outstanding research. Perhaps this would be an advantageous time to acknowledge and reward university service with a similar, prestigious annual award.

Increasing the visibility of assessment. The task force discussed the need to make assessment activities an integral part of curricula and campus life. This could begin by enhancing the visibility and importance of assessment in the Mission Statement, in the Graduate and Undergraduate catalogs, on the Oakland University website, and in recruiting efforts. In addition, we would recommend that assessment be given greater emphasis in communications across the university and across the community. Assessment of our outcomes could be a selling point for the university—it is a concrete way of telling current and prospective students that we care about their learning and we are working hard to ensure that students graduating from our programs are well-prepared for future success. It is also a way of telling business and industry that we are working hard to provide them with high quality, well-trained employees.

Students. The task force also discussed the need to consider involving students in assessment of their own achievement. To date, students have been largely ignored in the process at Oakland. We feel that this is a mistake. Assessment experts agree that the most successful assessment activities are those in which students become invested and those which students perceive to be related to their own future success (see, for example, Lopez, 1999; Palomba & Banta, 1999). We believe that units and programs should consider bringing students (perhaps from clubs and organizations related to their program) into the discussion of assessment activities; these involved students can be expected to become advocates among their peers. In addition, the task force discussed the possibility of using student performance data at the individual level. Many students would appreciate knowing how they stack up and how well they have achieved the objectives of their major program. This is one of the hallmarks of a mature assessment program.

FINANCIAL CONSIDERATIONS

The task force recognizes that for assessment to be successful, an investment of institutional resources is necessary (Halpern, 1987; Morse & Santiago, 2000; Palomba & Banta, 2000). We have proposed the creation of an assessment center along with other initiatives, realizing that this will require allocation of substantial resources at a time when increased resources are being sought by many constituencies. We believe that there are probably less expensive alternatives, but we went with an assessment center for two reasons: it is a common solution to the assessment dilemma; it will provide the quickest and best long-term solution. We believe that over the long term, it is extremely important that Oakland University "get on with it." We do not have space here to go into the history of the pressures toward accountability in institutions of higher education (for a review, see Ewell, 1997), but we can point out a couple of important trends that provide evidence for the need to consider committing significant resources to assessment.

First, there are increasing public demands for accountability from legislators, taxpayers, employers, and others (Morse & Santiago, 2000). Evaluating and improving the quality of student learning through outcomes assessment is the solution that the higher education community has proposed, and this solution has been accepted by accrediting bodies. One result of pressures toward accountability is a new trend toward performance-based funding of universities (Ewell, 1998; Morse & Santiago, 2000). At least eight states currently reward institutions for high scores on outcomes assessment. In the state of Tennessee, the legislature now uses the results of outcome-based assessment as the basis for allocating funds to higher education. We can think of the possibility of performance-based funding as either a threat or an opportunity—either way it makes sense to encourage and fund quality assessment.

REFERENCES

Ewell, P. T. (1997). Accountability and Assessment in a second decade. Presentation from the 1997 AAHE Conference on Assessment and Quality. Washington, DC: AAHE.

Ewell, P. T. (1998). From the states—statewide testing: The sequel. Assessment Update, 10(5), 12-13.

Halpern, D. F. (1987). Student outcomes assessment: What do institutions stand to gain? San Francisco: Jossey-Bass, Inc.

L�pez, C. L. (1999). A decade of assessing student learning: What we have learned; what’s next? lopez@ncacihe.org.

Morse, J. A., & Santiago, Jr., G. (January/February 2000). Accreditation and faculty: Working together. Academe, 30-34.

Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass, Inc.

 Submitted by:

John Bohte, Political Science
Terri Orbuch, Sociology
Daniel Braunstein, Management 
Sandra Pelfry, Accounting
James Cipielewski, Reading & Lang. Arts
Christina Sieloff, Nursing
Christine Hansen, Psychology, Task Force Chair
Xhristine Stiller, Physical Therapy
Shawn Lombardo, Library
Ishwar Sethi, Comp. Science & Engr.
Kathleen Moore, Chemistry