Delphi module

14.4.4
Target-language testing

Activity 22

What is your department's policy on testing in the FL?
With comprehension questions, do you think there are grounds for insisting on exclusive use of the FL in both question and answer?

Assessment of FL listening and reading is closely tied to the question of whether L1 or the FL should be used as the medium of assessment. The key issue here is validity: we must be sure that what the test aims to assess is indeed being assessed, rather than something else altogether. For example, FL instructions can introduce a reading element into assessment.

Page (1993) points out three problems relating to validity:

The difficulty of avoiding comprehension problems with instructions can limit the range of tasks to simple and unimaginative ones.

In setting reading assessments, the range of tasks can be severely restricted by the need to avoid questions in which the answer is 'signalled' or even given by the question, thus reducing the need for the student to show understanding of the text or what they are writing. This is the 'Jaberwocky' problem:

Q: What did the slithy toves do?
A: They gyred and gimbled.
Q: Where did they do it?
A: In the wabe.' (Page, 1993: 6)

This can result in questions that focus on minor aspects of a text, rather than the key issues.

When students are required to answer in the FL, taking account of the quality of their language use threatens to invalidate the test, as language production becomes confused with the real issue, comprehension.

The argument often advanced in favour of FL testing is that it is done effectively in the EFL world. The counter argument is that the EFL world usually has no choice, whereas in FL teaching, with a common mother tongue, we are able to choose which language is more appropriate to a particular assessment situation.

Despite such serious reservations, the past ten years have seen the spread of target language testing both to public exams at secondary level and university assessment. A major reason has been the danger of negative washback on classroom practice. Two reports from the School Curriculum and Assessment Authority (Neather et al, 1995; and Powell et al, 1996) have made the case for target language testing. The following list of possible approaches draws in part on these documents; some will be familiar from the taxonomies included in Activities 19 and 21 above.

At lower levels:

choosing/ticking pictures and other visuals to show knowledge of phrases and single lexical items;
answering one-word multiple-choice questions;
ticking boxes containing single words;
answering questions that require a number as an answer;
writing single words or short phrases to complete a form, grid, chart or list;
completing gapped FL sentences - eg: 'The reason for the _______ of the plan was lack of resources';
correcting a list, phrase or sentence;
putting true/false next to a list of statements.

At higher levels:

completing sentences - eg: 'The argument used to counter the proposal was that ___________________';
finding synonyms or equivalent words/phrases in the text;
using context to provide definitions of words/phrases from the text;
drawing a diagram or map based on instructions in a text (eg 'Complete the plan of the supermarket based on the information contained in the text');
questions that require deduction rather than a mere reworking of the text;
carefully formulated mark schemes that stipulate the degree to which communication, as opposed to accuracy, is taken into account.

In listening, the demands of the activity mean that questions requiring FL writing beyond single words or short phrases, are likely to be too difficult at anything other than advanced levels. In all cases, the key is to provide instructions that are simple, to the point and that follow a predictable pattern - and preferably are standard from year to year.

In conclusion, there are clearly arguments in favour of both target language and mother tongue testing. Alderson (2000: 147) sums up the debate by saying that the most important thing is to make sure instructions are as comprehensible as possible, which usually (my emphasis) means using L1. However, it is clear that there are numerous ways in which L1 use in tests can be minimized. In the interests of continuity between teaching and assessment, and of ensuring positive washback on classroom practice, these techniques ought to be maximized to promote majority target language use. But the interests of validity demand that dogma has no place here: where L1 will avoid ambiguity or a confusion of skills, it should be used.