CALPER Language Assessment

Center for Advanced Language Proficiency Education and Research at The Pennsylvania State University

Getting Started

Our general philosophy of assessment is that assessment should be about providing information that better helps teachers meet learners’ needs. This is quite different from other perspectives that would have assessment as a measuring stick for comparing and ranking learners, as an instrument of teacher accountability, or as a means of motivating and controlling students. Bringing assessment into the general activity of education means that the goals of assessing and teaching are aligned, and this has several consequences for how assessment can be carried out in the classroom. For example, rather than a one-time event that tests learners under a set of unusual constraints such as time limits and the requirement to perform in isolation, assessment may co-occur with instruction.

The assessment approach one decides to follow will depend upon a number of factors. Three questions in particular may prove helpful during the selection of assessment approaches. As you consider each of these questions, keep in mind that it is likely you will find more than one assessment approach to be relevant to your classroom context. If we consider that each approach is a source of information about learners, then it seems reasonable that a combination of different assessments will yield multiple perspectives and that, taken together, they will offer a detailed view of the processes of learner development. We might liken this to the multiple camera angles that are used in filmmaking – only when they are finally brought together do we have a movie.

1. What specific information is the assessment intended to provide?

When planning an assessment, it is important to clearly define the knowledge and abilities to be assessed. Lyle Bachman (1990), a leading authority on language testing, explains that assessments may be based on a theory of ability or they may be tied to a set of criteria. For example, a speaking test could be informed by a theory of language proficiency that defines proficiency using terms such as grammatical competence and metalinguistic awareness (and these in turn would also need to be explained).

A theory-based approach to assessment is particularly common in the development of large-scale tests such as the TOEFL or the ACTFL OPI. In the classroom, however, it is more common for assessments to be derived from curricular objectives or desired outcomes stated on a course syllabus. For instance, an advanced foreign language conversation course might have as an objective that learners demonstrate ability to control grammatical features of the language during spontaneous speech production. An assessment of whether or how learners are progressing towards this objective would need to be constructed so as to engage them in activities that require use of specific aspects of grammar. Learners might be asked to engage in role plays, give a speech or presentation, or participate in an oral interview. If control of verbal tense is of interest during the assessment, then these tasks will need to involve learners in discussion of past and future events. In this way, defining the ‘what’ of the assessment can help to determine the ‘how.’ In other words, asking learners to write an analysis of a piece of literature would likely not be an effective means of assessing their control of grammar during speaking.

Of course, sometimes it is difficult or even undesirable to assess a single modality such as speaking. Increasingly, language teachers are designing learning tasks that are similar to the kinds of activities and situations students are likely to encounter outside the classroom. As a result, learners are often engaged in tasks that require multiple modalities, as when they listen to a presentation, discuss issues and ideas with classmates, and then prepare a written reflection on the experience. Assessing such tasks is clearly complex but can be ameliorated through the judicious use of guidelines and rubrics (see discussion below).

The matter is further complicated by the fact that many advanced language courses are either content-based or are infused with content matter ranging from literature and art history to cultural and political studies. In this case, learners’ performance of specific tasks will necessarily involve their linguistic as well as non-linguistic knowledge and abilities. Thus, learners might read a series of historical documents in the target language and then prepare an argumentative essay based on what they have learned. To be sure, this involves much more than their ability to read and write in the language. While designing such an assessment, it is important to conduct a task analysis that brings to light the demands of the task as well as the resources, knowledge, and abilities learners will need to carry it out. This will be crucial to successfully planning and implementing the assessment and appropriately preparing learners.

2. How and to whom will this information be reported?

The information about learner development obtained from assessment is no doubt of great interest to teachers and to the learners themselves. However, individual teaching contexts will vary regarding others who may be identified as stakeholders – that is, as those who have some interest in assessment outcomes. Parents, administrators, school board members, counselors, and academic advisors are all potential stakeholders who may be informed about the performance of an individual learner or an entire class.

The information gleaned from assessments must therefore be expressed in a meaningful and comprehensible manner. Thus, simply stating that a learner earned a mark of “3” on a recent assessment will have little value unless this is accompanied by details that help with its interpretation, such as a descriptor of what a “3” represents, the range of possible scores, the range of scores earned by the learner’s peers, the class average, etc. However, if the primary interest is an individual’s development, then scores and comparisons to other learners may not be the most effective means of assessment reporting. One alternative is a qualitative profile that includes a learner’s strengths, areas of improvement, ongoing needs, and immediate goals. Such an approach offers the advantage of a “moving” picture of learner development – it is both historical, showing where the learner has been, and future-oriented.

When planning to implement an assessment with learners, it is also advisable to either present them with a rubric or to recruit them in the development of a rubric. Rubrics simply list the possible grades or marks learners might earn and describe what is expected for each grade. In this way, teachers can refer to the rubric when assigning grades and learners understand, prior to the assessment, what they will need to do to earn a given grade. The descriptors that compose the rubric should be as detailed and specific as possible so that differences between, say, an A and a B or a 4 and a 5 are not confusing. When possible, providing exemplars of “A work” or the like is also helpful.

An additional point concerning assessment reporting has to do with learner participation in the assessment process. Much can be said for involving learners in the assessment of their own work (self-assessment) or the work of other students (peer assessment). Both encourage learners to assume a role other than “student” or “examinee” and to engage in critical reflection, evaluation, and argumentation to support their conclusions. Peer and self-assessment can be greatly ameliorated by providing learners with guidelines that, similar to rubrics, give learners a basis for making assessments.

3. How will this information be used?

The information gained about learners through assessment can be used for any number of purposes, including determining their mastery of material or their readiness to move on but also for more official purposes such as assigning grades, meeting standards, and demonstrating to administrators the effectiveness of instruction.

The extent to which an assessment provides the information it was intended to provide is known as validity. Thus, validity is not an inherent property of any assessment but is derived from the uses to which assessment information is put. Take, for example, a portfolio learners are asked to create that demonstrates their writing development in an advanced composition course. Assuming the portfolio has been carefully assembled to include the various kinds of writing studied during the course, then it may be a valid basis for making claims about learners’ writing development. However, it is less clear that this same portfolio could be used to infer learners’ overall language proficiency. Obviously such inferences could be questioned on several grounds, and it would no doubt be determined that this was not a valid use of the writing portfolio.

In the language classroom our primary goal is to support learner development. The validity of classroom-based assessment is a function of how it helps meet that goal. As mentioned, assessment information may be used for other purposes, and so the validity of each use needs to be evaluated separately. However, the development-orientation of classroom assessment means that the benefits to teaching and learning must take precedent over all other purposes. To return to the example of the writing portfolio, its validity may be considered in terms of the insights it provides into the processes of learner development but also the fact that the activity of creating the portfolio may promote learner development by engaging them in self-evaluation and critical reflection.

 


Reference:
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.


Next section: Multiplism Perspective

Skip to toolbar