Using Assessment to Improve the Freshman Year Experience

 

 

 

 

Sylvia Alatorre Alva, Ph.D.

Director of the Fullerton First Year Program and

Department of Child and Adolescent Studies

Julie E. Stokes, Ph.D.

Afro-Ethnic Studies and Psychology Departments

Juanita Razo, M.A.

Coordinator, New Student Programs and
Co-curricular Achievement Record Program

Suellen Cox, M.A., M.L.S.

Library Instruction Coordinator and

Assistant Faculty Librarian

Elizabeth Housewright, M.A., M.L.S.

Co-Coordinator of Reference and

Assistant Faculty Librarian

Jeannie Kim-Han, M.A.

Coordinator, Community-Based Learning and Service Center

 

 

Submitted to CSUF WASC Self-study Task Force

Learning Outcomes Assessment Grants

California State University, Fullerton

June 1999

Using Assessment to Improve the Freshman Year Experience

 

What exactly is a learning community? "A learning community is any one of a variety of curricular structures that link together several existing courses—or actually restructure the curricular material entirely—so that students have opportunities for deeper understanding and integration of the material they are learning, and more interaction with one another and their teachers as fellow participants in the learning enterprise" (Gabelnick, 1990, p. 19). In their most basic form learning communities require a block scheduling that enables students to take courses together and, typically, they are organized around a central theme.

The Fullerton First Year (FFY) is a yearlong program for first-time freshman that includes a freshman seminar, an information technology course, and a set of integrated General Education (GE) courses, which they take in their first two semesters on campus. In addition to the linked courses, FFY students participate in a 30-hour community-based service learning project linked to the freshman seminar and a GE course. The curricular and co-curricular components of the FFY are organized around a theme—Education, Social Responsibility, and Community. By linking the curricular and co-curricular learning opportunities of first-time freshman, the goal of the FFY is to enhance student learning and create a more coherent and integrated learning experience for freshman. The FFY program also attempts to create a more seamless experience for freshman through better coordination of student support services such as mentoring, academic advisement, and co-curricular involvement.


This report summarizes the results of a comprehensive assessment plan that was used to measure the learning outcomes of the FFY program. The FFY was piloted in 1997-98, with a cohort of 115 first-time freshman. In 1998-99, the FFY was expanded to include 150 students. The assessment plan called for the use of both quantitative and qualitative strategies to assess the individual and program-level outcomes of FFY, particularly when compared to first-time freshman who did not participate in the FFY program. The highlights of the assessment plan are described below.

Assessment of Curricular Components

Institutional data was used to compare the academic achievement and retention of the FFY students, compared to all other first-time freshman. In 1997-98, there were 115 FFY students and the comparison group consisted of 2,205 (909 males, 1,296 females) first-time freshman.

Student Grade-Point Averages. Table 1 illustrates the earned grade-point averages achieved by both the FFY students and the comparative cohort during the Fall 1997, Spring 1998, and Fall 1998 semesters. Means and standard deviations are reported. The grade points presented here are based on a traditional four-point grading system.

The FFY students’ mean grade-point average for the Fall 1997 semester represents a .33 increase over the comparative group, while their Spring 1998 mean grade-point average indicates a .13 increase over the comparative group. Although the Spring FFY overall grade-point average was not as impressive as the Fall, the results do show an increase in grade-point average for students who participated in the FFY program.

Table 1

Student Grade-Point Averages
for FFY Participants and Comparison Group

   

FFY

Comparison Group

Fall 1997

Mean

2.62

(115)

2.29

(2,203)

St. Dev.

.88

1.00

(t=3.452, p<.001)

Spring 1998

Mean

2.68

(108)

2.38

(2,203)

St. Dev.

.84

.92

(t=3.45, p<.001)

Fall 1998

Mean

2.67

(150)

2.53

(1718)

St. Dev.

.64

.71

(t=2.00, p<.05)

 

Percentage of Students in Good Academic Standing. Table 2 represents the percentage of students in good academic standing for both FFY students and the comparative cohort. A student is in good academic standing if their cumulative grade-point average is above 2.00 (based on a four-point grading scale). Because the students in this study were all first-time freshmen, the data here represent the number of students in each group who had first- and second-semester grade-point averages above 2.00.

The percentage of FFY students in good academic standing for the Fall 1997 semester represents a 13.2% (p<.002) increase compared to the comparative cohort, while the Spring 1998 FFY student data represent a 10.2% (p<.01) increase compared to the comparative cohort. These numbers are quite significant and point to a strong relationship between involvement in the FFY program and student academic achievement.

Table 2

Percentage of Students in Good Academic Standing
for FFY Participants and Comparison Group

 

FFY

Comparison Group

Fall 1997:

82.6%

69.4%

(x2=9.145, p<.002, df=1)

Spring 1998:

75.2%

65.0%

(x2=13.277, p<.010, df=1)

Information Technology Skills and Attitudes

A team of librarians collaborated with Management Science and Information Systems and Computer Science faculty to design a new course for the FFY program. The MSIS/CPSC portion of the two unit "Introduction to Information Technology and Presentation Skills" class taught computer basics, e-mail, Internet searching, and the use of PowerPoint. The Library component of the class focused on using and citing electronic library resources, such as the library catalog, electronic periodical indexes and full-text databases. Students also were taught to evaluate information found in these sources as well as on the World Wide Web. This course was taught to all FFY students in Fall 1997 and Fall 1998.

 

A pre-assessment instrument measured student experience and confidence using various computer applications, as well as attitudes towards technology. A post-assessment instrument measured confidence and attitudes. Results from the 1997/98 class indicated a marked improvement in confidence using e-mail, the Internet and electronic databases after taking the class. Although students entering the Fall 1998 class had more experience using computers than the previous class, similar improvements in confidence levels after the class were noted. For example, mean confidence ratings for using electronic databases, selecting the right database for a specific topic and selecting search terms for a database search were significantly higher after completing the course. Students were also significantly more confident evaluating Internet sites for appropriateness, relevancy and currency after the library component. All pre-post comparisons were significant at p<.000.

Assessment of Co-curricular Components

Co-curricular Involvement

Using the Student Development Task & Lifestyle Inventory, we also assessed and monitored several dimensions of student life and development, as part of the Co-curricular Achievement Record (CAR) program. CAR is a student development transcript and advising program that parallels and supports the goals of the academic curriculum by measuring student learning and development from out-of-class experiences. The CAR program also provides opportunities to assess student learning and development through a formal and systematic advisement model.

After completing the Student Development Task and Lifestyle Inventory, the FFY students were assigned a faculty or staff mentor to help them interpret their results on the Student Development Task & Lifestyle Inventory and to advise them of co-curricular opportunities that they might consider to address areas of weakness (e.g., campus involvement, time management, leadership skills, cultural awareness). In consultation with their faculty/staff mentor, each of the FFY students developed a set of personal goals and identified specific strategies for meeting their stated goals. In this format, students take greater responsibility for planning and monitoring their co-curricular activities.

Service Learning Experience

Service learning has been defined as "an instructional methodology that integrates community service with academic instruction as it focuses on critical, reflective thinking and civic responsibility. Service learning programs involve students in organizing community service that addresses local needs, while developing their academic skills, sense of civic responsibility, and commitment to the community" (American Association of Community Colleges, 1995).

Paper-and Pencil Questionnaire. The assessment plan included paper-and-pencil questionnaire that included a 40-item modified version of Conrad and Hedin’s (1981) Experiential Education Evaluation instrument. Using a 5-point Likert scale, this instrument assessed student attitudes about Social and Personal Responsibility, Social Welfare, Duty, Competence, Efficacy and Performance of Responsible Acts. The instrument was administered at the outset of their freshman year (September 1997) and after completing the 30-hour service learning experience at the end of their freshman year (May 1998), allowing us to measure changes in student attitudes. As reported in Table 3, after completing their 30-hour service learning project, the FFY students reported significantly higher levels of agreement on five of the six subscales. Consonant with the theme—Education, Social Responsibility, and Community--- the FFY students endorsed statements that reflect a sense of civic responsibility and commitment to society.

Table 3

Pre- and Post-comparisons of Student Attitudes on
Social Responsibility and Service to Community

________________________________________________________________________

Sept. 1997 May 1998

Attitudes Mean SD n Mean SD n t p.<

________________________________________________________________________

Social Welfare 18.77 4.62 88 2.160 4.40 76 3.13** .001

Duty 16.44 4.91 88 19.29 5.24 76 2.83** .01

Competence 13.59 4.31 88 15.89 4.17 76 3.10*** .001

Efficacy 20.14 3.78 88 20.96 3.32 76 1.80 .10

Performance 13.30 3.92 88 15.51 3.92 76 3.41*** .001

________________________________________________________________________

*p<.05; p<.01; p<.001.

________________________________________________________________________

Note. Sample items by subscale.

Attitudes on Social Welfare

Attitudes on Duty

Competence to Take Responsibility

Efficacy Regarding Responsibility

Performance of Responsible Acts

________________________________________________________________________

 

Focus Groups. The value of the service learning experience was assessed using focus groups and written feedback forms completed by the FFY students. The focus groups revealed that many of the students described their service learning experiences in terms of contributing to society and making a difference in the world.

Conclusion

By design, learning communities stretch students and shake them out of too-comfortable assumptions about course work or their own responsibility for their learning and for making sense of the seemingly overwhelming amount of information that forms the basis of their education. The same can be said for institutions and the instructional member as well-building a learning community forces us to stretch our curricular assumptions and how we teach undergraduate students. Preliminary findings of our assessment plan support the use of learning communities to improve the educational experience of first-time freshman.

References