Newsletter 4

Draft Self Study Released

A preliminary draft by CSUF’s Self Study Task Force has been received by Ralph Wolff, executive director of the Senior Schools and College Division of the Western Association of Schools and Colleges (WASC) and the head of CSUF’s site team, Don Farmer, Vice President for Academic Affairs at King’s College, Wilkes-Barre, Pennsylvania . As soon as their comments are returned, the complete draft will be released to the campus community on CSUF’s WASC web site.

The study is in two parts. "Phase I" (available on the Internet at http://wasc.fullerton.edu) was completed last year after a ten month study of existing data on the campus that related to the three themes of our study: student learning; faculty and staff learning; the campus environment for learning. The focus of Phase I was on information, materials, and reports that the University had already amassed. Sources of this data included the departmental Program Performance Reviews (PPRs), Annual Reports, Accreditation Reports, and reports generated by various entities or centers on campus.

The Task Force also reviewed many other campus-generated reports as part of Phase I. These included annual "campus climate" surveys, the semesterly reports from the Office of Analytical Studies, enrollment management data generated from Student and Academic Affairs, annual reports by the Academic Senate, and progress reports from programs like the Fullerton First Year experience and Employee Training and Development. The University Planning Committee’s efforts to define the "marks" of a Fullerton student helped the Task Force envision how the Mission and Goals statement can serve as a tool for program evaluation and planning.

For "Phase II" of our Self Study, the WASC Self Study Task Force collected new data designed to "fill in gaps" that were discovered after our previous analysis. To secure these data, among other efforts, we participated in two HERI surveys (Higher Education Research Institute at UCLA). The first HERI survey compared our incoming freshmen to a national sample and asked specific questions concerning expectations about their learning experience. The second HERI survey was conducted among full-time faculty and is also part of a national sample of faculty. A similar instrument was designed and distributed to full-time staff. Both surveys invited respondents to discuss their perceptions about our learning environment and the University’s mission of making "learning preeminent."

The Self Study Task Force also conducted a survey of campus classrooms as part of its effort to assess the contribution of the physical infrastructure to learning. The Task Force also issued an RFP to the campus to commission over a dozen studies of learning outcomes assessment. These findings, as well as analyses based on other new studies, are reported in Phase II.

Phase II includes an Appendix that provides a progress report on nine issues raised in our previous Self Study in 1990. This includes a brief overview of WASC’s concerns at that time and a summary of what has happened in regard to those issues since our 1994 "Midterm Report" to WASC.

After consultation and collaboration with the full WASC Self Study Task Force, the Steering Committee synthesized the findings from Phase I and Phase II into a set of recommendations. In writing the recommendations, the Steering Committee acknowledged changes occurring in the CSU system as well as developments on campus. Issues of technology, assessment, accountability and self-governance all represent opportunities for the University to further its mission of learning. Highlights of those recommendations are:

• Continue to develop a campus culture centered on learning. Our mission of making learning preeminent enables us to integrate our focus on student learning with our traditionally strong emphasis on faculty scholarly and creative activities and on staff training and development. However, across the campus, carrying out the University’s mission to make learning preeminent needs to assume a more prominent role in determining appropriate rewards for faculty and staff.
• Continue providing funds for technological improvement and innovation, particularly with support structures and services. Computers need to be replaced and updated on a regular basis, and technical systems need to be maintained. Just as importantly, those campus centers that facilitate the use of the technology—particularly the Faculty Development Center and the Employee Training and Development program—need sustained support.
• Broaden access to increase the use of expensive technology. Faculty and students become excited about the learning alternatives made available through technology and become frustrated when computers in the student laboratories are not available, when classrooms have not been outfitted to utilize the technology, or when training programs are not readily available. Specifically, we recommend that the classroom renovation program be extended to include smaller classrooms. Perhaps most importantly, techniques of assessment that measure effectively how technology improves student learning need to be incorporated throughout courses and programs that rely on instructional technology tools.
• Keep assessment firmly rooted in program improvement. Adopting assessment practices must be viewed as a positive approach to continual improvement and not seen as a management strategy designed to make faculty and staff accountable.
• Increase support for students in our University who are not native speakers of English (which is currently estimated at 48 percent of our student body, including 32 percent of those born in the United States and about 80 percent of those born outside the U.S.). We must be prepared to meet the challenge of our demographic environment, and this includes not only instruction in language competency but a renewed emphasis on culture, socialization, and civic values.
• Monitor progress in reaching our learning goals, including the general education goals and the goals of our degree programs. We have introduced a new approach to general education that incorporates learning goals and stresses writing and communication skills. We need to institute effective assessment strategies to ensure that our efforts to improve the effectiveness of general education and our degree programs are meeting our students’ needs.

As part of its effort to develop procedures that will enable re-accreditation to be a useful exercise for its member institutions, WASC is experimenting with a process referred to as an "academic audit." What is an "academic audit"? And how is it different from the usual accreditation process?

The academic audit process was developed in England and Hong Kong, and is now used there as well as in New Zealand, a number of European countries, and elsewhere. It utilizes several of the techniques and assumptions that also characterize accreditation processes. For example, external evaluators comprise the audit team. Prior to a site visit, the team prepares using materials supplied by the host institution. The institution clearly defines the areas of investigation in which it is interested—most commonly, areas centrally related to the institution’s mission—and prepares appropriate documentation. The team visits the campus, spends several days gathering information, and writes a report. This, of course, is much like the usual accreditation site team visit.

However, the audit differs in other ways. First, it does not attempt to look at every feature of the institution as defined by external standards, but rather focuses on the institution’s own identification of its purpose and goals. The audit relies on judicious sampling and on an examination of the processes the campus employs to ensure desired outcomes, rather than on specific products. In other words, those conducting the audit select a few representative programs and, somewhat like a financial audit, trace the processes that are used to provide feedback and ideas for improvement in the program. The goal of the audit is not to measure the quality of the program (although that is always a sub-text of every accreditation). Rather, the audit focuses on how a program establishes its own goals, how it goes about implementing them, and what strategies it follows to build in improvement and adapt to change.

At Fullerton, our WASC visiting team will employ one element of the academic audit. By means of in depth interviews, the team will explore the ways in which core academic programs have implemented the campus mission of making learning preeminent.

We have asked the accreditation team to focus its investigation on how programs support student learning. We are confident—and the data in our Self Study demonstrate—that CSUF programs provide students with knowledge and skills that are the hallmarks of an excellent education. We also know that some programs have been more self-conscious than others about articulating the processes they use to measure their success. We think we have constructed a set of questions that will enable all programs to respond effectively to queries that the visiting team will pose. They are:

1. How do you decide what you want your students to learn in your program? (Describe any processes that you use to reach agreement about the outcomes you desire your program to achieve.)
2. What do you expect your students to know and be able to do as a result of completing your program?
3. How do you communicate these expectations to students?
4. How is the curriculum of your program structured to reflect your program’s goals for student learning and to facilitate student progress toward achieving them?
5. How do you know that students are learning what you expect them to learn at various points in the curriculum?
6. How do you know that your graduates have accomplished the goals of your program?
7. How do you use this information (the answers to questions 5 and 6) to improve your program?

Approximately ten days to two weeks before the WASC site team arrives on campus (March 21-24), the team will tell us which programs it has selected for its audit. We expect that the site team will select six or seven degree programs (departments or interdisciplinary programs, plus the General Education Committee, for the audit interviews. We expect that these will be academic programs chosen to be representative of our curriculum. However, like you, we have no way of knowing which programs these will be.

The General Education Committee is a likely candidate for the audit for several reasons. First, over a three year period, an ad hoc committee of the Academic Senate created new learning goals after much deliberation and input from many academic programs on campus. For the past year and a half, the General Education Committee has been reviewing courses in the GE curriculum to monitor the implementation process. During this period, the GE Committee has received feedback about the learning goals and the coherence of the new program. We believe that the whole process of reforming the General Education curriculum exemplifies commitment to student learning as articulated by the University’s Mission statement. Thus we are encouraging our accreditation site team to use this program as one of its audit cases.

The site team will have only three days on campus to collect the information that it needs. During these three days, we expect the team to devote a half day, perhaps one afternoon, to talk to the programs that it selects for the academic audit

Our goal in explaining the interview process known elsewhere in the world as an "academic audit" is to alert the campus community to this aspect of our campus visit and to inform academic departments and programs, as well as the General Education Committee, of the possibility of being selected for the audit. For some—probably many –programs, preparing for the site team visit will involve little more than reviewing their most recent Program Performance Report (or for those programs that have them, their most recent accreditation reports), and ensuring that answers to the seven questions are current. Many programs have re-evaluated their GE courses to make sure they incorporate the new GE learning goals, and these programs have also engaged in important discussions about student learning in these courses.

We know, though, that some programs have not yet undergone the GE course review process, and that others are still in the midst of looking at their programs in terms of their students’ learning. Those programs will want to think about the seven audit questions a bit more intensely so that they will be ready for the possibility of being selected as one of the programs to be interviewed as representative of the campus as a whole.

WASC has selected an imposing group of scholars and experts to conduct our site visit next March. The team is chaired by Don Farmer, Vice-President of Kings College, Wilkes-Barre, PA., an assessment and student learning expert, nationally recognized for his leadership at Kings College for instituting assessment for quality improvement Others include:

Trudy Banta, Vice Chancellor for Planning & Institutional Improvement, Indiana University-Purdue University, Indianapolis; Robert Cox, Manager, Faculty Teaching, Workload and Enrollment Planning,

UCLA; David Dill, Professor of Public Policy Analysis, University of North Carolina, Chapel Hill; Amy Driscoll, Director for Teaching, Learning & Assessment, CSU Monterey Bay; Susan Nummedal, Professor of Psychologyat seating capacity represents a more than four-fold increase from the beginning of the 1990s. Also installed were more than 500 computers with full Internet capability. The physical addition to the Library permitted it to house centers for campus learning, including Information Technology, the Honors Program, the Faculty Development Center and computer laboratories used by Employee Training and Development for staff (and faculty) learning. Evidence is presented that showcases the many learning activitcopies of such exhibits as Program Performance Reviews, annual reports, and accreditation reports.

If you have materials that demonstrate your program’s implementation of learning goals for students, or other materials that evoke the university’s Mission, please contact the Self Study Steering Committee (X-3227) and tell us what you have. We’ve already received one suggestion—a video tape of Astronaut Tracy Caldwell at the 40^th Anniversary Convocation. Your suggestions are welcome!

Also, please read our draft report on our Web Site (available on December 1) and let us know what you think. If this brief overview here has inspired you, please let us know your thoughts. Are our conclusions and recommendations the ones you would make?

You will have an opportunity to talk with members of the site team when they are here. Please mark the dates on your calendar.

The WASC Self Study office is located McCarthy Hall 111 A. Our clerical assistant is Jennifer Robinson (Ph: 278-3227). Her hours are 8:00 a.m. to noon. Our research technician is Charlene Carr. Our email is WASC@fullerton.edu and our Web site is http://WASC.fullerton.edu.