UPDATE Fall 2003
Study confirms importance of discussion-based instruction to student literacy performance
Janet Angelis
A team of CELA researchers (Arthur Applebee, Judith Langer, Martin Nystrand, and Adam Gamoran) recently completed a study that showed a strong relationship between discussion-based instructional approaches and challenging academic content to student literacy performance in a diverse set of schools across the country. Rather than focus on particular instructional strategies (e.g., question-asking techniques), the study was designed to accommodate the many different ways that teachers support student literacy development through classroom discourse. Results are based on data on 974 students in 64 middle and high school English classrooms in 19 schools in 5 states.
In an article in to be published in the Fall 2003 issue of the American Educational Research Journal, Applebee, Langer, Nystrand, and Gamoran report that “high academic demands and discussion-based approaches were significantly related to spring performance, controlling for initial literacy levels, gender, socioeconomic status, and race/ethnicity. Moreover, . . . these approaches were effective across a range of situations, for students of varying levels of academic ability, whatever classrooms they may have been in.”
Classrooms were selected for the study following a process of nominations, initial screening, and site visits to ensure full understanding of the project and support for it by teachers as well as their department chairs and administrators. Field researchers observed each participating classroom four times during the course of the academic year, two observations each in fall and spring.
The study focused on three major aspects of discussion-based instruction as well as the relationship among variables and between variables and student performance. Protocols were designed to capture and measure the extent of each classroom’s 1) dialogic approach to discussion, 2) emphasis on the development of student understanding, or envisionment building, and 3) curricular cohesion.
Discussion-Based Approaches
Dialogic approach. To measure dialogic discussion, researchers used a classroom observation system developed by Nystrand (1999) called the Classroom Language Assessment System, or CLASS. Using CLASS, an observer codes classroom interactions on a laptop computer in real time. The program is designed to capture classroom discussion and related activities. In particular, it focuses on the types of questions asked (both by teachers and students), the classroom interactions, and the materials being used. These data were then analyzed to identify those interactions that constituted open discussion, which the researchers defined as the free exchange of information among at least 3 participants (which may include the teacher) that lasts longer than 30 seconds.
Envisionment building. During as well as at the end of each classroom observation, observers made two separate sets of ratings of the session’s emphasis on what Langer has defined as envisionment building (1995) -- instruction that is focused on supporting students to develop ever more complete and complex understandings of a text. One set of ratings had to do with the nature of instructional activities -- e.g., the frequency with which students expressed an opinion or took a position in relation to the text. The other rated the lesson overall in terms of whether the teacher had offered thought-provoking experiences that provided students opportunities to raise questions, learn new ideas and skills, try them out, and explore and use the new ideas and skills in ways that lead to more elaborated understandings.
Curricular cohesion. In addition, observers sought to capture the extent to which the lesson observed was part of a larger conversation in the discipline (Applebee 1996) by noting if the lesson was part of a continuing topic, if comments made by students or teacher connected the lesson to previous or future topics, etc. This aspect of the discussion was also measured by calculating the teachers’ ratings on a survey that asked seven questions about continuity across lessons and units or across reading, writing, and discussion activities.
High Academic Demands
To measure the amount of work expected of the students, students were asked to report on a) the nature and amount of revisions they were expected to make to their writing; b) the hours of English homework they were assigned each week; and c) how often they completed their reading and writing assignments. Responses were aggregated to the classroom level.
Classroom observers also noted the materials used in each lesson (e.g., classical fiction, nonfiction, young adult literature, poetry) and the nature of the assignments (e.g., short-answer writing, note-taking, analytic writing).
Performance Assessments
Three writing tasks were used to assess student performance, one in fall and two in spring. The fall and one spring assessment related to a fictional character of the students’ choosing; the second spring question asked about a personal experience. Student answers were scored both for the overall difficulty of the attempted response and their success in making that response. For overall difficulty, readers scored the level of abstraction attempted – from recording factual data to reporting on something to analyzing an observation or experience to theorizing. To measure students’ success in their responses,
raters scored responses on a four-point scale that ranged from unsatisfactory (only the barest of information provided) to elaborated (highly wrought, well-developed, tightly organized text).
Results
The instructional emphases in the classrooms studied varied in terms of the types of reading materials and writing activities assigned, as well as in the quantity and quality of discussion-based approaches students experienced. Overall, in both middle and high schools in both urban and suburban settings, discussion-based approaches supported higher levels of student literacy growth.
A Word about Tracking
Although results indicate that the literacy performance of all students benefited from discussion-based approaches, the study also revealed major differences in instruction between high- and low-track classes. Not only were the academic demands for low-track classes generally (but not always) lower, but students in low-track classes also experienced much less open discussion – a maximum of 3.7 minutes per 60-minute period, as compared with a 14.5-minute maximum in the high-track classes. As the authors note, to some extent, effective instructional approaches have not been tried with lower-track students.
Conclusion
Because the study sought to measure the multi-dimensional aspects of a discussion-based classroom rather than specific instructional techniques, it avoided the risk of documenting teacher behaviors (e.g., asking higher-order questions, employing adequate “wait time”) whose effectiveness depends on the teacher’s underlying understanding of what the behavior is intended to accomplish (i.e., the larger conceptual frame that calls for such behaviors). Classrooms in which these aspects of discussion-based instruction are evident – dialogic discussion, development of student understanding, and curricular conversations – are fundamentally different than traditional classrooms. Differences include different views of teacher and student roles, different assumptions about effective teaching, and, even, different ideas about what it means to do English well. What the current study tells us is that discussion-based approaches work. What we do not know yet is the best balance among the many kinds of activities that teachers use.
Applebee, A.N. (1996). Curriculum as conversation: Transforming traditions of teaching and learning.
Chicago: University of Chicago Press.
Langer, J.A. (1995) Envisioning Literature: Literary understanding and literature instruction. New York:
Teachers College Press.
Nystrand, M. (1999) Classroom language assessment system (CLASS 3.0). Madison, WI: Center on
English Learning & Achievement.
This article is based on
“Discussion-Based Approaches to Developing Understanding: Classroom Instruction
and Student Performance in Middle and High School English,” A.N. Applebee, J.A.
Langer, M. Nystrand, A. Gamoran. American Educational Research Journal
(Fall 2003). Janet Angelis is associate director of the
Center on English Learning & Achievement. Applebee, Langer, and Nystrand are
center directors, and Gamoran a senior researcher at the center.
The Center on English Learning and Achievement