Creation of web-based lessons for the Global Citizen 2000 (GC2000) project commenced in spring 2001, and continued through summer 2002. During spring 2002, Dr. Eric Davis discussed the concept of initiating a field-based documentation study of lesson effectiveness with Bill Fernekes, supervisor of social studies at Hunterdon Central Regional High School, one of the charter members of the GC2000 consortium. Dr. Fernekes agreed to author a proposal that would outline the necessary elements to conduct such an evaluation study: overall design, data collection procedures, data interpretation and analysis, and completion of a final report to be eventually mounted on the GC2000 website. Following review of the proposal, Dr. Davis and Dr. Fernekes came to an agreement about how the evaluation study would be conducted, and the project moved forward beginning in fall 2002, with a completion date of summer 2003.

II. Evaluation Design

1. Goals: The purposes of the field test assessment were:
   (1) to evaluate the effectiveness of existing lessons and units by comparing learning goals, methodologies and assessments of student progress with evidence gathered from teachers and students in project schools; and
   (2) to recommend improvements for the lessons and units, as well as how the units can be integrated within existing school courses of study.
2. Participants and Sample Size: Participating teachers were solicited from member school districts in the GC2000 consortium. District supervisors and faculty were invited to participate during the period October through December 2002. A range of school districts from throughout New Jersey, ranging from Paterson in the north to Vineland in the South were represented, although most came from central New Jersey, in relative proximity to Rutgers-New Brunswick. Table One summarizes relevant data about participants and the sample of students engaged in the evaluation study.

TABLE ONE: Evaluation Study Participant Data
Faculty Participant	School District	Course Title	Grade	Sample
Lori Spinelli	East Brunswick	Spanish Language	5	28
Ruth Chang	East Brunswick	French Language	5	22
William Cahill	Edison Township	English	12	97
Nancy Wallace	Freehold Twp. HS	International Rels.	9	24
Barbara DeBaylo	Highland Park	Art	7 - 8	18
Keith Dennison	Hunterdon Central	Comp Wld Studies	11	37
Darrell De Tample	Hunterdon Central	Comp Wld Studies	11	23
Christopher Bellotti	Hunterdon Central	Comp Wld Studies	11	* -- See below
Paul Urban	Hunterdon Central	Comp Wld Studies	11	15
Elizabeth Downing	Hunterdon Central	Chinese Language	9 - 12	16
Melissa Pineiro	North Brunswick	World History-Cult.	9	21
Chris Muhammad	Paterson	Visual Art	9 - 11	60
Cristina Arnese	South Brunswick	Psychology	11 - 12	21
Lynne Poag	Vineland	Civics	8	18
Carol Fevola	West Windsor - Plainsboro HS	Language Arts 3	11	59
* -- Chris Bellotti coordinated the participation of HCRHS faculty in the study, having piloted a GC2000 unit during the prior school year.

3. Procedures: In November 2003, an overview of the evaluation project and the field-testing process was distributed to interested faculty from GC2000 member districts. Supervisors and school administrators responsible for oversight of their district's work with the GC2000 project were sent the same materials to insure comprehensive understanding of the project's timetable, procedures, and participant responsibilities. School districts were asked to finalize commitments to the field-testing process by mid-January 2003, and 10 districts eventually confirmed their commitment to the evaluation study. Table Two provides a timeline with brief descriptions of project tasks to be completed by participating faculty and supervisors.

   TABLE TWO: Evaluation Study Tasks and Timeline
   January 24, 2003	Commitment letters from districts received.
   Jan.-July 2003	Implementation of field testing and completion of final report.
   February	By 2-15. Curriculum integration plan completed and sent to district supervisor, who sends them on to the assessment director.
   February	By 2-15. Student sample finalized for data collection and documentation data form completed by faculty. Documentation data form sent to district supervisor and then onward to the assessment director.
   March/April	Lesson/unit implemented within existing course by participating faculty member. Faculty member completes mid-point reflection (at point where the unit is in process, but not completed--prior to collection of culminating assessment of student performance). Data collection (documentation) is ongoing and mid-point reflections are shared with district supervisor, and then with assessment director. District supervisors complete 1 page progress report on how field testing is proceeding in their districts. By end of April, assessment director prepares mid-point progress report and submits this to the principal investigator.
   May	By end of second week in May, field testing is completed. Faculty complete data collection and do final reflection. Documentation concludes and student work products are organized and readied for interpretation/analysis.
   May 14	Meeting held to interpret and analyze results of documentation. Recommendations for improvement of lessons/units and for curriculum integration are prepared by faculty participants and district supervisors.
   July	Assessment director prepares final report and submits report to principal investigator for GC2000. Final report reproduced and sent to all participating districts following review by principal investigator.

   Mid-course adjustments were made for faculty who experienced difficulties in completing specific tasks due to illness and unanticipated family emergencies. At the May 14 meeting, a summative evaluation response was prepared by each participant that was added to the data collected during the field-testing process. Appendix One contains all instruments utilized in the study for data collection and organization.

4. Data Collection, Interpretation and Analysis: Each participating faculty member engaged in the field-testing process was required to complete a registration form for the project, which identifies the prospective GC2000 on-line units they intended to field-test, the course in which they would integrate that unit, the anticipated dates (start and end) of unit implementation, and any questions they had about the data collection, interpretation and analysis process. Just prior to implementation of the unit, each participating faculty member was required to submit 2 additional documents: a curriculum integration plan and a documentation plan. The curriculum integration plan outlined the relationship between the field-test unit, local curriculum objectives, and state core standards, along with a brief description of their specific learning goals, anticipated adjustments in their curriculum/lesson planning and teaching practices, and their plans for assessment of student performance with the field test unit. The documentation plan defined the sample of students who would study the field test unit, categorized by gender, and described the types of artifacts of student work that would be collected during the unit for later interpretation and analysis.

   Two reflections were required of participating faculty, one at the mid-point of implementing the unit and another at the end of the field-testing process. Reflection questions elicited faculty responses regarding the relationship between unit learning goals and results based upon student performance, modifications in teaching practice during and after implementation of the units, modifications in assessment of student performance, and on the concluding reflection, summative judgments about unit effectiveness and recommendations for unit improvement.

   A summative evaluation document was prepared by all participants at the May 14 meeting of field test faculty (and at a later date by some who needed slightly more time to conclude their field-testing process). This document elicited responses to questions about GC2000 website accessibility, the relationship between GC2000 unit goals and overall curricular goals in their respective schools, judgements about student response to the content and instructional strategies in the units, and an overall evaluation of unit effectiveness (i. e., would you employ the unit again, based upon adult and student feedback).

   Artifacts of student work were collected from each participating faculty member, and in one case (Hunterdon Central - Keith Dennison and Elizabeth Downing), evidence from pre and post-test survey instruments also was collected concerning student response to the unit they implemented ("The Yi Peoples").

   Throughout the field-testing process, reminder emails were sent by Dr. Fernekes to all participants regarding deadlines for receipt of documentation materials. He also responded to questions posed by participating faculty about the completion of specific documents during the process. With only three exceptions, all participants submitted the entire set of field-testing documentation requirements prior to mid-June 2003. Efforts are continuing to obtain the remaining materials from the few faculty who need to complete submission of all field-testing documentation materials.

   Table Three provides a snapshot list of the units which were field-tested from the GC2000 website, organized by teacher name and participating school district.

   TABLE THREE: GC2000 Curriculum Units Field-Tested From February through June 2003
   Faculty Member	School District	Course for Integration	Unit Title
   Lori Spinelli	East Brunswick	Spanish Language	World Lang. Module: Beginner's Level
   Ruth Chang	East Brunwick	French Language	World Lang. Module: Beginner's Level
   William Cahill	Edison Township	English	Global Literature: Haiku Poetry
   Nancy Wallace	Freehold Twp. HS	International Rels.	Children's Rights: IV
   Barbara DeBaylo	Highland Park	Art	African Masks
   Keith Dennison	Hunterdon Central	Comp. Wld. Studies	The Yi Peoples: The Politics of Ethnicity
   Darrell De Tample	Hunterdon Central	Comp. Wld. Studies	Children's Rights
   Christopher Bellotti	Hunterdon Central	Comp. Wld. Studies	Minorities in a Cross-Cultural Perspective
   Paul Urban	Hunterdon Central	Comp. Wld. Studies	Religion/Spirituality
   Elizabeth Downing	Hunterdon Central	Chinese Language	The Yi Peoples: The Politics of Ethnicity
   Melissa Pineiro	North Brunswick	World History-Cult.	Religions of the World Poster Lesson
   Chris Muhammad	Paterson	Visual Art	African Masks
   Cristina Arnese	South Brunswick	Psychology	Stereotypes
   Lynne Poag	Vineland	Civics	Rights of Non-Citizen Immigrants
   Carol Fevola	West Windsor - Plainsboro HS	Language Arts 3	Global Literature: "Diamond Dust" and "House Opposite"

   Four subject fields were represented in the evaluation study: visual arts (2), language arts (2), social studies (6) and world languages (3). In one case (Dennison/Downing), cross-disciplinary teaching was employed so that students from both world languages (Chinese) and social studies (Comparative World Studies) experienced the unit simultaneously and participated together in the culminating assessment for the unit. Documentation data gathered from this teaching pair reflected that collaborative approach. Two teachers who utilized the Beginner's Level world language modules (Chang and Spinelli) also collaborated closely in the development and implementation of their field-testing activities, but their students did not actively collaborate because they were studying different languages (French and Spanish).

Goal One: Effectiveness of GC2000 Lessons/Units. Findings for this project goal are subdivided into four subtopics: ease of curriculum integration, developmental appropriateness of content, resources and teaching practices, accessibility to web-based lessons/units, and congruence between stated lesson/unit goals and student achievement. Evidence utilized in this section of the report is drawn from faculty reflections and student artifacts documenting lesson/unit results. Copies of each participating faculty member's concluding reflections and summative evaluations are provided in Appendix II.

1. Ease of Curriculum Integration. Participating faculty reported that they faced no significant problems integrating their chosen lesson/unit(s) into existing school curricula. Each faculty member was able to link the field-test lesson/unit directly to existing curricular goals and NJ Core Curriculum Content Standards for their respective subject area(s). Modifications were made by some faculty to the field-test lesson/unit(s) methodologies in order to meet time constraints, or to provide enhanced opportunities for student participation (i. e., by adjusting the size of cooperative groups). In at least one case, selected activities were not utilized because a specific lesson goal had been met through an instructional activity that had just been concluded (i. e., the instructor deemed that including the other lesson activity would have been redundant).

   In response to the final question on the May 14 summative evaluation document ("Would you use the lesson/unit(s) again based upon this experience?" Consider both your own response and student responses to the lesson/unit(s)?), all participating faculty stated "yes". Each had suggestions for lesson modifications, but none claimed that the units clashed with existing course goals or state core curriculum standards. On the contrary, the majority of faculty summative comments were highly favorable concerning the field test lesson/unit(s), and a number of the participating faculty commented that it had spurred them to learn more on their own about the content with which they had engaged students during the evaluation study.

2. Developmental Appropriateness of Content, Resources and Teaching Practices. The vast majority of students participating in the field-testing process were enrolled in heterogeneously grouped classes, irrespective of subject field. In a number of those classes, classified students were also enrolled, some in with in-class support provisions or Section 504 modifications. Faculty responses indicated that many students were consistently engaged by the content and strategies utilized during the field-testing process. With few exceptions, students were able to read, view and interpret lesson materials with little difficulty, and then use those materials to develop work products that were correlated to the lesson/unit learning goals. The inclusion of group work, discussion strategies, and oral presentations reflected an awareness on the part of the lesson/unit authors that such student-centered instructional strategies engender sharing of ideas and build commitment to completion of complex tasks. Participating faculty commented on the value of these strategies and their motivational value for students in their mid-point and concluding reflections, as well as their summative evaluation responses. Regarding the elementary grades World Languages lessons, the two field-test faculty commented that the lesson/unit(s) facilitated cross-disciplinary connections that helped students learn content from social studies, mathematics, music and applied technology (computer usage). These lessons were highly motivational for the students and the process of language acquisition functioned as the medium for acquiring knowledge and refining skills in other subject fields, addressing a broad, overarching theme in their district's elementary grades curriculum design.
3. Accessibility to Web-based Lessons/Units. With few exceptions, faculty experienced no difficulty in locating, viewing, and printing lesson/unit materials to implement the field-testing process. One unit was inaccessible due to technical problems on the GC2000 website, while in two schools the lack of computer access for students in classrooms made computer access solely a teacher activity. In these cases, teachers printed out materials for their students from the field-test lesson, or accessed the website to view examples of visual materials (i. e., African Masks). Some faculty commented on the inconsistent quality of resources located on specific lesson/unit websites, and recommended that specific sections of a curriculum module (i. e., Stereotypes, Religion and Spirituality) be revisited and modified to provide resources that would be more intellectually challenging to students. Overall, the website lesson/unit(s) appear to be designed for teacher rather than student use.
4. Congruence Between Lesson Goals and Student Achievement. Almost all participating faculty remarked that their students had met the learning goals set at the outset of the field-testing process. Evidence from student work products supported these claims, and in some cases, other learning goals were met that were unanticipated. Among these unanticipated results were development of generalizations by students about commonalities in world religions based upon student presentations (Religion and Spirituality), acquisition of content knowledge about Japanese art and aesthetics (Global Literature: Haiku Poetry), and the integration of personal experiences with content drawn from the lesson/unit being studied (Religion and Spirituality, Global Literature: "House Opposite", "Diamond Dust").

   Assessments of student performance ranged from quizzes and tests administered in the classroom (involving objective format and constructed response items) to performance assessments (creation of masks, policy debates, oral presentations, scored discussions), and in one case, a standardized test (U. S. Immigration and Naturalization Service Citizenship Test for Naturalization, used in the Rights of Non-Citizen Immigrants Unit). The samples of students from whom documentation evidence was gathered were relatively small, thus placing limits on our ability to construct broad generalizations about any one unit field-tested in this project. With that caveat in mind, when one examines the responses of faculty as well as the evidence of student performance collected during the field-testing process, the high percentage of faculty reporting they would use the lessons again, albeit with some modifications, speaks quite positively about the strength of the lessons/units in promoting student achievement. No faculty member involved in the field-testing process stated they would refuse to use the lesson again, and all were forthcoming in providing thoughtful, well-organized suggestions for lesson/unit modifications. Had these lesson/unit(s) been deemed ineffective by the field-test faculty, the faculty suggestions would have been far more extensive and detailed.

Goal Two: Recommended Improvements for Lessons and Units. Each participating faculty member provided suggestions for improvement in various aspects of the units (content, instructional strategies, assessments of student performance). In most cases, these recommended improvements emphasized modifications of instructional strategies: such as lengthening time on task for specific activities, modifying focus or discussion questions, clarifying the weight/significance of specific assignments in relation to others in the unit, and adjusting the design of specific activities (along with providing more challenging content resources) for higher ability students. Regarding assessments of student performance, some faculty offered suggestions that tended to provide a greater variety of assessment options to meet the needs of varying ability levels. In at least one case, the rubrics provided for the unit were a source of criticism, and faculty offered very detailed suggestions for rubric modifications (The Yi Peoples). In another case, specific suggestions were provided for lengthening the amount of time needed to complete a unit assessment (Children's Rights-Part IV), so that students would be able to more effectively address the challenging nature of the assignment (dealing with human rights activism).

Last but not least, in some cases recommendations were made to augment or change existing content resources for specific lessons and units. In "The Yi People" unit, the participating faculty recommended elimination of two subgroups for research due to the paucity of information available on them. In the "Stereotypes" lesson, the resources recommended in the web-based lesson for student research were found to be ineffective, thus requiring the teacher to guide students to other resources.

Overall, faculty offered detailed suggestions for improvement that were grounded in their field-testing experiences. The recommendations reflected a careful consideration of student feedback during the field-testing experience, the needs of their own school curricular, and evidence drawn from the mid-point and concluding reflections.

IV. Conclusions

Goal One: Effectiveness of GC2000 Lessons/Units.

1. The GC2000 units do not pose problems for curriculum integration into existing school programs based upon evidence gathered during this field test. No faculty reported difficulty in establishing clear and compelling linkages between field test lessons/units, existing curriculum objectives in school courses, and state core curriculum content standards. Faculty were able to include content, methodologies and assessment strategies within their curricular and instructional planning while continuing to meet established curriculum objectives irrespective of subject field emphases, or broad curricular themes.
2. More often than not, the content and instructional methodologies employed in the field test units were successful in actively engaging students in focused learning experiences that were congruent with the stated goals of the lessons/units. Learning resources were comprehensible to the vast majority of students, while (with very few exceptions), instructional activities and assessment strategies could be completed successfully by students across the full range of ability levels.
3. Faculty experienced few difficulties in accessing the web-based lessons/units, and the suggestions offered for improvement can easily be implemented.
4. There is strong congruence between the stated learning goals and student achievement, based upon the limited sample of students involved in the field testing process. Faculty reported a strong commitment to use the lessons/units again, and the suggestions for improvement, while varying in their focus from one lesson/unit to another, constituted refinements of the lessons/units rather than major redesigns.

Goal Two: Recommended Improvements for Lessons and Units.

No consistent pattern was evident regarding the recommendations for improvement. Recommendations were lesson/unit specific, and thus reflected areas where the individual faculty member noted a deficiency or weakness, which they sought to remedy through modifications in content, methodology and/or assessment strategies. Such modifications also reflected curricular needs of the participating faculty member in teaching specific courses in their own subject field, since the broader curricular goals in those courses were met to some degree through the medium of the field-test lesson/unit.

Implementation of Field-Testing Process

1. A range of lessons/units were successfully field-tested from the GC2000 website, and the subject fields into which those lessons/units were integrated spanned elementary through senior high school. School districts represented in the field-testing process included two Abbott Districts (Vineland and Paterson) along with others that were decidedly higher in socioeconomic standing.
2. Faculty who participated in the field-testing process made a strong commitment to the documentation and meeting requirements, and with only a few exceptions, all requested documentation was submitted to the evaluation study director.
3. Meetings held at Rutgers University - New Brunswick provided an opportunity for review and modification of field-testing process requirements, and at the conclusion of the study, for sharing of insights from faculty about the overall field-testing process and the results obtained in each participating school.

1. Validation of Field-Test Results. A second evaluation study should be designed and implemented with faculty in GC2000 consortium-member schools and other interested school districts to determine if the results presented in this evaluation study can be replicated in other settings. This should be done following modification of the lessons/units employed in this evaluation study. Participants in a replication study should have access to this report and the individual faculty documentation prepared during this study. A replication study should also provide opportunities for participating faculty and students to complete both pre and post-test interviews and assessment instruments that will gather a richer set of data, so that a broader range of measures can be utilized to gauge lesson/unit impact on student performance.
2. Website Redesign. The existing GC2000 website should be modified in light of specific suggestions made about unit accessibility and learning resources. While a wholesale redesign of the GC2000 website is unnecessary, serious consideration should be given to an introductory page where the teacher-based emphasis of the site is explained. Likewise, a guide for students who access the site on their own should be prepared so they can benefit from the resources available on the site that may contribute to their own learning minus guidance from their own teachers or other adults.
3. Access to This Report and Student Work Products. This evaluation study should be placed on the GC2000 website, possibly in "executive summary" and full text format. Should specific samples of student work be identified by the participating field-test faculty for mounting on the GC2000 website, permissions must be obtained from students to make their work products publicly available. It is strongly recommended that annotations accompany the presentation of student work products which are prepared by field-test faculty, since these annotations will provide a context for understanding the student work products, along with links to the lesson activities that generated such work products.