Learning Outcomes Assessment Grant Report

to the WASC Self Study Steering Committee on

"Assessing Student Progress in Gendered Technoculture:

Collaboration across Disciplines and with the Community"

Renae Bredin

Assistant Professor

Women's Studies

Gerri McNenny, Ph.D.

Assistant Professor

Dept. of English

Students in English 575T, Teaching Writing in the Secondary Schools, collaborated with students in Women's Studies 320, Gendered Technoculture, in the assessment of student-authored web sites written for community organizations. This project involved several stages of assessment, including the formulation of student-generated criteria for success, feedback from English 575 students to Women's Studies 320 students, responses from Women's Studies 320 web authors, instructor's evaluations of WS 320 student projects, and evaluations by the professor of English 575 participants.

Four Goals of the Project

  1. Incorporating Internet technology into a non-technologically based discipline
  2. This was accomplished by transforming a more traditional writing assignment for a Women's Studies course into an assignment requiring that students incorporate their written work into a web site. The assignment obliged students to demonstrate their research and writing skills, as well as to learn and demonstrate new technology based skills like writing in HTML programming code, developing Internet-based information presentation strategies, and analyzing gender issues within a technological framework. While initial student responses were skeptical, they were enthusiastic about the final results, and about their new confidence and abilities in writing material for the Internet. These web sites can be viewed at http://hss.fullerton.edu/womens/bredin/gthome.html

  3. Cross fertilization of disciplines
  4. The partnership developed between Secondary English Education and Women's Studies is a new partnership, and one that enhanced student learning and collaboration in both disciplines. Secondary English Education students were introduced to a field of study that was new and unfamiliar to them. Women's Studies students understood that there would be an audience outside of the discipline for whom they would need to be clear, concise, and complete in making the connections between the community group and the question of gender issues for that group. Some students were more successful than others in this regard. For instance, on the web site for the Nite Owl Quilter's Guild, the connection between quilting as women's art and the guild was made. One evaluator suggested that this connection be made more clearly, while another evaluator had a different take:

    You have clearly addressed the issue of gender equity when you write that quilts are "part of women’s cultural history that have played a role in female creativity, community, cooperation, and communication." Then you continue with the following: "Quilts address issues of originality, tradition, individuality, and collectivity, content, and values in art, and the feminine sensibility."

  5. Application of theory to practice
  1. Replicability

Setting Assessment Criteria

In setting out to determine criteria for the student-authored web sites for community organizations, Renae Bredin and Gerri McNenny clarified the learning outcomes for Women's Studies 320 students as well as learning objectives for English 575 seminar participants. Students in WS 320 were assigned the task of web authoring and construction using html with the following criteria in mind:

Students in English 575 were given the challenge of developing a set of criteria for the evaluation of web sites that drew not only on the features given above, but also on the evaluation criteria for web sites as stipulated in the CSUF Library handout on web site evaluation. Those features included authority, objectivity, content/relevancy, currency, aesthetics, and accuracy. (See Appendix A for CSUF Library handout. See Appendix B and C for assignment handouts for English 575 students.).

The objective of this class exercise, then, was to introduce students as future teachers to the task of setting evaluation criteria with which they could then determine whether learning objectives had been achieved through student projects. The students developed criteria for success on this assignment based on learning objectives established by Prof. Bredin, since they themselves were not delivering instruction to WS 320 students and could only look for those instructional features that Prof. Bredin reported had been incorporated into her class.

Student-Authored Web Sites

Criteria for Students in WS 320 chose the following community organizations for their web page projects:

(Gay, Lesbian, Bi, Trans)

For a brief description of the assignment in WMST 320, go to http://hss.fullerton.edu/womens/bredin/techno.html. From these agencies, students in English 575 selected one web page and all of its links for an in-depth evaluation and two other web pages for holistic evaluations. In this way, students were able to develop two different kinds of evaluation criteria, the one, a holistic rubric that looked at features in general, and the other, a more extended evaluation model that looked at each specific feature. The goal in either case was to provide constructive feedback to all of the students in WS 320 based on the success with which the students directed their writing to specific audiences and purposes.

English 575 Student Assessments

English 575 students' reactions to the web sites that WS 320 students wrote were enthusiastic for the most part. Many reported that the writing was the most purposeful and lively that they had seen in a while. Foremost in their consciousness was the need to provide feedback to fellow students, to react as interested readers whose reactions were to be valued by the writers as indicators of the relative success or failure of their projects. Thus, students in English 575 were able to relate to WS 320 students' efforts with personal anecdotes and tactful suggestions. One writer, Joe Ciccoianni, responded to the Boys and Girls Club of Whittier web site in the following manner:

Joe then goes on to evaluate the aesthetic features, the author's point of view in presenting information, and the authoritativeness and accuracy of the page, particularly the ways in which the web page met the research needs of interested browsers. Fortunately, this thoroughness characterized most graduate seminar participants' reactions to these web sites, and student authors corresponded with English 575 students in sharing their reactions to the feedback they received.

Teacher Evaluation of Graduate Seminar's Evaluations

In evaluating graduate student's responses to WS 320 students' web pages, Gerri McNenny drew directly from the criteria the class collectively established at the beginning of the assignment. Students were accordingly assessed by considerations of how well they evaluated the writer's awareness and responsiveness to their target audiences, the ease with which users were able to navigate the web site, the usefulness of the site for the communities it addresses, the graphic and textual balance, and the ways in which the site addresses gender equity in the community. (See Appendix D for web site evaluation criteria for English 575 students).

Overall, English 575 students consistently applied the criteria we agreed upon to their evaluations of the web sites. Because of the tone English 575 students took, student authors were open to their criticism, couched as it was in an evaluative description of the project. At the end of the semester, students reported that they enjoyed seeing the potential that the web held for publishing student writing. Students agreed that the immediacy of a real audience with authentic needs in genuine communities had made the writing easier to evaluate and more interesting to read.

Final Comments

The extensive feedback from an outside source was extremely useful to WMST 320 students. Because those in ENGL 575 were being graded on their assessments, students in WMST 320 benefited from the complex and detailed responses they were given. As one student remarked to her reviewer, "Wow. I don’t think I could have asked for a more concise total content analysis. I am very appreciative of your comments. They were well thought out and practical in addition to being very flattering." The extensive and supportive tone and content of the evaluations calmed early student fears of being evaluated by outsiders who might be too harsh or not experienced enough. Many of the evaluations were up to 2 pages, single-space, in length--a depth often not possible for professors given the constraints of large classes and large numbers of assignments to assess. In the future, comments from the participating professors to the students on both ends of the conversation regarding the assessments would perhaps make clearer the overall benefits of this total feedback loop

In this case, technology produced the possibility for students to develop relationships with each other via email and the threaded discussion group that were productive. One student assessor asked questions of the author of the web site on women's bicycling, which gave the author an opportunity to explain and comment on the assignment and his web site. Assessment goes beyond giving a student a grade for demonstrating learning. The use of detailed assessment strategies as a part of the learning process contributes to more ingrained student outcomes, and to more clearly defined mastery of basic skills and analytical abilities.

 

Appendix A: The CSUF Pollack Library's Guide to Evaluating Web Pages:

Six Criteria for Evaluating Web Pages

http://www.library.fullerton.edu/Guides/evalsites.htm

 

SIX CRITERIA FOR EVALUATING WEB PAGES

 

AUTHORITY

Is there an author or sponsoring body (organization, company) and is the name of the author or sponsoring body listed?

Are the author(s) qualifications or credentials listed?

Is contact information (e-mail, address, or phone number) included?

Is the listed name different from the "Webmaster"?

Where is the document published? Check the URL domain (.edu, .org, .com, .gov).

OBJECTIVITY

Has the author or organization clearly stated the goals and/or aims of the site? (The Web often functions as a virtual soapbox).

If objectivity is important, is the information presented in an objective manner?

Is a particular point of view being presented?

Is there a bias, either explicit or implied?

Is the information presented free of advertising? If there is advertising, is it clearly differentiated from the informational content?

Is this an "Infomercial" Web page? (On the Web, the distinction between advertising and information can easily become blurred).

ACCURACY

If facts and figures are given, are they accurate?

Are the sources for any factual information clearly listed?

Can they be verified in another source?

How does this information compare with other sources of information on this topic?

CURRENCY

When was the page produced?

When was it last updated?

Is any information on the page outdated?

Are the links up-to-date? Are there references to sites which have moved?

CONTENT/RELEVANCY

Does the page meet your research needs or purpose?

Is the subject adequately covered?

Are the links (if any) evaluated/annotated and do they complement the page's theme?

If other sources are quoted, is the information accurately cited?

Is all the information on the site free, or is there a fee to link to some or all of the information?

AESTHETICS

Is there an appropriate balance between text and images?

Is the design of the page visually appealing or is it too cluttered?

Is there an option for text only, or frames, or a suggested browser for better viewing?

If page requires special software to view the information, how much are you missing if you don't have the software?

 

 

PUTTING IT ALL TOGETHER

AUTHORITY. If the page lists the author credentials and its domain is preferred (.edu, .org, .gov), and…

OBJECTIVITY. If the page provides accurate and objective information, and …

ACCURACY. If the page lists the author and institution that published the page and provides a way of contacting him/her, and…

CURRENCY. If the page is current and updated regularly (as stated on the page) and the links (if any) are also up-to-date, and…

CONTENT/ RELEVANCY. If the page meets your research needs and the subject is adequately covered, and…

AESTHETICS. If the page has a balance of images and graphics, is visually appealing,

...then you will have found a high quality Web page that should be of value to your research!

 

 

 

 

Appendix B:

Assignment Handouts for English 575:

Teaching Writing in the High Schools

English 575

Dr. McNenny

Evaluating Student Writing: Web Sites for the Community

To give you the experience of offering feedback to writers, we will collaborate with another CSUF class, Women's Studies 320: Gendered Techno-Culture, whose members have identified various non-profit organizations for whom they will produce web sites targeted to serve the needs of special groups within the community.

To develop criteria for evaluating these web pages, we'll need to engage in a dialogue with these writers to find out a number of things.

Starting the Dialogue

While it may be difficult for us to meet face-to-face with these writers, we can still dialogue with them via e-mail, and we can establish some familiarity with other community non-profit web sites, so that we have a basis for evaluation.

Here is a list of the non-profit agencies that students will be representing in their web page projects:

California State Educators Association Para-Educator Program

GLBTN Student Organization/CSU Fullerton

(Gay, Lesbian, Bi Trans)

Women's Bicycling

Women's Studies Student Association, CSU Fullerton

Boys and Girls Clubs of Whittier

CSU Fullerton Student Groups Home Page

Fullerton Senior Center

Night Owl Quilt Guild

Youth Education Services: The Center Orange County

FUEL (Lesbian Night Club)

Math, Science, & Technology Gender Equity Page

For Primary School Teachers and CSUF Teachers in Training

Women's History Page/CSUF Women's Studies Program

Girl Scouts of Orange County

New Directions for Women (chemical dependency treatment)

From those agencies listed, select three that you find interesting. For this project you will evaluate one web page and all its links in depth. You will also give a holistic evaluation for two other web sites. The goal is to provide constructive feedback to all of the students in WS 320. For that reason, you'll need to give feedback on three web sites each.

E-Mail Exchanges with WS 320

To begin a dialogue with the students in Women's Studies 320, you can go to the course web site for that class.

Establishing a Basis for Evaluation

To give you a basis for your evaluation, we'll look at some award-winning web sites.

http://www.webbyawards.com/best/ - EDUCATION

As you look at these model web sites, consider the following:

Due dates

************************************************************************

English 575

Dr. McNenny

Workshop on Evaluating Student-Authored Web Sites

As we set about to develop criteria for the student-authored web sites for Women's Studies 320: Gendered Techno-Culture, we need to look at some comparable web sites also set up for non-profit organizations. These include the following:

National Organization of Women (NOW)

http://www.now.org/

League of Women Voters

http://www.lwv.org/

Girls Incorporated

http://www.girlsinc.org/

Young Women's Christian Organization

http://www.ywca.org/

You can add your own list of non-profit organizations. This list functions to give us a start in our evaluations. As you look at these pages you might consider some of the criteria offered on these web pages, available from the CSUF Library, for evaluating web sites.

Evaluating Web Sites for Educational Uses: A Bibliography and a Checklist

http://www.unc.edu/cit/guides/irg-49.html

Six Criteria for Evaluating Web Sites

http://www.library.fullerton.edu/Guides/evalsites.htm

The Library Home Page for Searching the Net

http://www.library.fullerton.edu/interws.htm

As you look at these sites and guidelines for criteria, feel free to adapt these to your own evaluations. Consider the assignment the students worked under, and be generous in your feedback to them.

Good luck!

Appendix C:

Samples of English 575 Student Evaluations of Women's Studies 320 Student-Authored Web Sites.

Sample One : Holistic Review

Jacque,

Hello, my name is Frank Perez. I appreciated your response on the

threaded discussion, so I decided to evaluate your web page on

paraeducator development, for my Eng. 575 class. It is a very

professional looking page, and the picture on the front page gives it a

serious tone. The picture was a good decision, a couple of older, wiser

teachers and a couple of younger, aspiring educators. It grabs the

viewers eye, and draws interest to scroll down. Your opening paragraph

provides a clear explanation of the page, and the fact that it is geared

towards people interested in pursuing jobs in education. The page, in

conjunction with the class, maintains gender equity, however, it may be

more applicable to women. In my experience of working at a jr. high, all

of our paraeducators are female. In the same respect, it would be a very

informative site for a male not fully aware of the potential career

advancement working as a paraeducator.

The site was carefully designed in a progressive manner, with

regards to the links. Each link ends with a link to the next page, and

build on one another, starting with "Teacher Aide to Paraeducator" and

ultimately ending with "Program Certification". Each page is nicely

represented with an opening cartoon representing the page, therefore

evenly compensating for the text on the page. The information on all of

the pages is properly documented, credible (i.e. .gov, .edu), and

demonstrates the validity of your claim by using examples from all over

the country.

This site is extremely useful for anybody (male/female) currently

working as a paraeducator, or looking for a way to help pay for their

education towards teaching. It is also helpful for school faculty to

review, as you pointed out advantages for board members, administrators,

and the paraeducators, in "Paraeducator Conference". Next in

"Workshops", you do a good job of explaining how diversely trained

paraeducators can be. Then we are led to the "Career Ladder" page which

helps to prove to the skeptical viewer that there are indeed important

jobs for paraeducators. In the next page, it is brought even closer to

home, the Cal State Long Beach program. Beautiful choreography of pages.

I enjoyed your web page, but I wish that there would have been more

information on how to find workshops and conferences. Your summary's are

so interesting, that I practically expected to see scheduled dates of

local workshops. You did provide a solid platform for me to search from,

though, because the education and government links were full of

information. I was also wondering if this applied to grades k-12, or

just specific grade levels?

Sample Two: Holistic Review

Fuel" - Cynthia Khatib

I think you have targeted your audience well, noticing the overall look and

feel of the site. The graphics add a distinct dimension to the web site,

which I'm assuming targets lesbians who frequent clubs. One problem I see

with the home page is the absence of the phone number or address of the club

itself. How are your web site visitors suppose to contact or go to this

club without that information? I couldn't seem to find it on any page that

you have linked, although you do supply the address to a few of the other

clubs you link to.

I also like the Fuel logo(is that their official logo?) and I think it is a

great idea to include the calendar of events. The only problem with that is

the constant revisions that have to be made every month to ensure the

calendar is up-to-date. But if you can do that, I think it's a great

service to those who would view your web site.

Something else that might add to your site's appeal would be some pictures.

I am always a fan of seeing what a place really looks like. Especially

considering this is a club, you might want to go one night and take picture

of the outside and some of the inside, with people dancing and having a good

time, to place on the site. This would add some more dynamic to the site as

well as allow viewers to see what the club actually looks like and the some

of the people who frequent it.

Overall you have a very poignant and attractive web site, with some good

color choices for background as well as some effective, if not alluring,

graphics. I hope my review will help you. Thanks,

--Lance Kayser

Teaching Writing 575T

Dr. McNenny

Sample Three: In-Depth Web Page Review

Web Site Review: Sharon Lemelle-Chabran

Math, Science, Technology and Women Web Site

Overall Look and Feel:

I thought the side bar of various categories was a great way to organize information, allowing the viewer of the site to access the information easily and quickly. I also liked the added feature highlighting the category name as the arrow passes over it. The bottom of each page has two arrows and a graphic of a home, which I thought was great. The background color is soft and easy to look at, which is a plus when staring at a screen for long periods of time. I think your site might be improved by placing graphics in places where they can accompany text accordingly. Some of your graphics are stuck by themselves and others do not relate to the text they surround. I also think adding some pictures would be a great addition as well. Pictures would add both a compliment graphically to the heavy text and would capture some ethos from your audience. There were some grammatical errors that need to be addressed, but overall your text was insightful.

Subject Categories:

The "Mission" section is an important part of any web site, for it sets up the expectations of the reader and creates the framework of your site. I thought you had some good things to say in your opening, but they need to be validated with sources. For instance, you say, "By the third or fourth grade, many (if not most) females believe that math, science and technology are male domains. Although the girls start school without these ideas, change occurs." How many is "many" or "most"? Once you do highlight specific examples or statistics, make sure and cite the sources they came from. Even a parenthetical numeric note can help, as long as it corresponds to the bibliography.

The same is true with the section labeled "Feminism," which makes claims like, "Many of the belief systems present today have been largely influenced by these historical beliefs…" but you don’t say what belief systems were influenced and how. Specificity is important, especially when making statements about society’s belief systems. It’s not that you shouldn’t make such a claim, just define it properly to alleviate confusion. On the other hand, I thought the links to the other feminist web sites were a great idea and I also thought your specific definitions of the various types of feminism was wonderful. It was clear and easy to follow, even if your audience if fairly new to feminist beliefs.

Looking into the section on "Education," I saw this as an important backbone to your web site, considering its purpose is to inform your audience about equality in education. You had great things to say about the establishment of education that focuses on equity. I also thought it was insightful when you pointed out that males also suffer from gender bias, because not all of them fit into the "Math, Science, and Technology" field, like myself. One small problem I saw was in an incomplete sentence, "Female domains also pose barriers to success for males as" and I think you might be missing the word "well." The sub-categories of "teacher training," "teacher attitudes" and "methods" make a great addition to the information provided about education equity. I think here would be a great place to add photos of maybe children in a classroom or a teacher helping a female student on the computer(just a suggestion).

My critique of "Technology" and "Data" is that you have some graphics, which is fine, but nothing else. There is no data in the "Data" section. The technology section could be greatly enhanced by supplying names of books or materials that aid in the advancement of women on the Internet, or CD-Roms and computer software geared toward a female audience. Your web site viewer will be interested in how to access supplies and materials that can make a difference in gender equity. Also the "Links" section is empty. Although I do think you supplied some nice links in other sections, there is still nothing to link to when the viewer go into this category.

Lastly the "Bibliography" section contains an impressive list of sources, but one important factor in including a bibliography is to incorporate it into the site. If this is a general reference list of related books on the subject then that is great, just label it accordingly, but if you used these sources in writing and creating this web site then citing the sources within the text is vital.

I applaud you on your choice of topics, it is both a very large and cumbersome topic and one that can be overlooked easily as well. I think this information is great to have on a web site and I thank you for taking the time to read my comments and suggestions.

--Lance Kayser

Teaching Writing 575T

Dr. McNenny

Sample Four: In-Depth Review

Peer Website Evaluation

English 575 to WS 320

May 17, 1999

Peer Reviewer: Michelle Bean

Website: Four Points Cycling Club

Author: Andrew Aslesen

Dear Andrew,

Overall, you have done an exceptional job on your Four Point Cycling Club website! Let me share with you how impressed I am:

Content:

The home page is very enticing. The club’s name stands out appropriately and the greeting is both informative and personal—it made me want to read on. The first paragraph is great; it gives the reader just enough information about the club in the first sentence. The introductory paragraph also flows effectively as it describes the site’s other linked pages. Linking the appropriate pages to the words in the text is a great feature; it makes it easy to go straight to those pages. The "thank you " is also a wonderful touch; it lets the reader know that you care and are truly interested in having them continue on in your site.

The "Getting Started" page is also well done. You have included some great information regarding the various types of riders. This is very helpful for novices like me. The "Getting Involved and Making the Commitment" section is an excellent way to encourage the reader to take action and get involved in your sport. I love the "Follow me!" phrase, it really fits well into the cycling theme!!!

The equipment page is also great. It is informative and authoritative. It sounds like you really know what you are talking about and that you love what you are describing, which makes it more exciting and interesting for me to read. I believe you, and I want to take your advice. You do not make the process of preparing sound overwhelming in any way. I love the fact that you include a section for the family. This is definitely a concern for me as a mother, as I am sure it is for other women. My only question is how do you know so much about women? The site is very well suited to address the needs, concerns, and interest of women, yet could also appeal to men. After all, you do describe in the first paragraph that the membership includes men. Perhaps, you could include a little descriptive paragraph for men and how they contribute to this women’s club.

I absolutely loved your "News and Inspiration from our Members" page. The stories were wonderful and truly inspiring. What a great idea!!! Also including a link to the club’s email is great; this makes it easy for the reader to contact you.

Format and Attractibility:

The visual aspects of the website are also very well done. The balance between text and graphics/pictures is suitable; there is not too much text. The use of color is also very appropriate. The backgrounds are light and subtle enough to draw attention to the text, yet they are very interesting and well connected to the theme of the site. The home page background is great! Did you create the FPCC logo yourself? It’s awesome. It also explains what "Four Points" means. And I love the little cycling men background on the other pages. The soft lavender color is wonderful and pleasing to the eye. The color theme is consistent, which makes the site look very professional. The bicycle chain used as a page divider also fits wonderfully into the theme and highlights the club’s name quite well.

I also love the picture you use on the home page; it is very sentimental and beautiful. Using the pictures of the various types of bikes that coordinate with the type of biker is also informative and interesting. I only wish you had put the name of each bike in a caption so that I might know what type of bike you recommend purchasing. The pictures of the recommended books are also very helpful, and the links to other sites, along with the book recommendations, could be very useful to both a novice and an experienced rider.

The format of the site is great. I found it very easy to navigate the site with the convenient buttons at the bottom of every page. My only suggestion would be to add a few targets. For example on the "Getting Started" page you may want to list the various types of riders at the top so the reader might easily click to the desired definition. I also think that it would be helpful to have the club membership information and calendar on its own page or on the home page. This information seems important to me and should not be considered miscellaneous.

You may also want to revise and check for a few needed commas and word choice errors in the text. (For example, on the Getting Started page, you should change allready to already.) The little stuff like that is no big deal; there are no problems that distract from the fluidity of the writing.

You have done a wonderful job! It is very well done and professional, yet it has a lot of character and aspects of the "personal" that make it quite unique. In other words, you not only provide helpful information about cycling, you have also integrated personal aspects of other cyclists’ lives that make reading about your club very interesting and inviting. In fact, I am truly considering taking up cycling myself now!

 

 

Appendix D:

Web Site Evaluation Criteria for English 575 Students

English 575

Dr. McNenny

Web Site Evaluation Criteria