Paper 2 Task 1–?information, tips and strategies

This task is about testing and it’s worth 18 marks. This is what the rubric says:

?

Using your knowledge of relevant testing concepts,?evaluate the effectiveness of the tasks in this test for this learner in this situation. ?

Make a total of?six points.?You must include both positive and negative points.

?

However, there are a few other things which you should know and should be doing to ensure you maximize your score.

?

·?It’s not necessary that the split between the positive and negative points is equal. You can go for 5:1, 4:2 or 3:3 – as long as there is?at least 1 positive and 1 negative. For some of you and some tests, it might be easier to think of the positives (as was the case in June 2015), or it can be the other way around.

·?Since the rubric specifies the number of answers,?don’t make more than 6 points. Only the first 6 are marked and so writing more than that would be a waste of time. Each correct point gets you 2 marks.

·?You also?need to outline the ways in which the test might have a positive or negative effect on the learner?(sometimes mysteriously called ‘a(chǎn)pplications’)?– that’s less clear from the instructions but it’s equally important as each can gain you 1 mark out of the 6 marks.?Read the rubric carefully to get all the information about the learner(s) and the situation.

·?Don’t use testing terms to organize your answers. Only use the testing terminology where appropriate – if you do, you need to show you understand the term so saying e.g. the test has high content validity is not going to get you marks, you’d need to explain why it is so ?(i.e. to demonstrate your understanding of the term/concept).

·?Regarding the layout of the answer – you have to?clearly signal which points are positive and which ones are negative. Failing to do so will inevitably mean no marks for Task 1.

·?Number your points (to see you don’t make more than 6) and?label what the Point is and in what way it can have a positive or negative effect.

·?You’re not required to give any examples.

·?If you are not confident in this area, consider doing some more reading before the exam – e.g. Hughes (Testing for Language Teachers), McNamara (Testing) or Burgess & Head (How to Teach for Exams) quite clearly explain the key terms and concepts. For the last minute revision, Thornbury’s An A to Z of ELT contains quite a few testing terms, too.

·?If there is too little time to do the reading, then you may want to start with the mark-heavy tasks on Paper 2 (i.e. Task 2 (42 marks) & 3 (40 marks)) to ensure you complete these before moving back and attempting the testing-focused Task 1.

?

?

Use bullet points?to save time and for clarity.

??

Reading

?see the suggestions above; if you go to the Moodle reading list and the Testing & Assessment subsection, you will find a range of sources there

?

Activities

?next week, you’ll be presenting some of the most common exams; you can mention who they are suitable for and why

?look at some sample test-types and try to analyse them

?think of a specific learner and a specific situation – consider what test would work for them

?study the guideline answers to get a better idea of what is required in this task

?

Now, let’s look at the description of the situation. While reading it, try to identify the key points which should have impact on the test design:

1 ? ? K is two weeks into an intensive eight-week course in a pre-intermediate General English class

2 ? ? ?in the UK. The class has been following an integrated syllabus that includes work on all 4 skills????

3 ? ? ?but is primarily organised around grammar. K has been told that the class will have a test every ??????????

4 ???? two weeks based on what they have been studying.?

5 ? ? ?In a needs analysis at the beginning of the course, K stated she wanted to improve her General

6 ? ? ?English, particularly her speaking, so that she could communicate more confidently in social ???

7 ? ? ?situations as well as function effectively in everyday transactions and interactions in the UK ????

8 ? ? ?(e.g. shopping, using public transport etc.).

(From June 2008 Paper 2)

What?key information is provided in lines 1, 2, 3, 4, 6, 7? Note it down before you look at the answers.

?

1 level – pre-intermediate

2 course content - all skills

3 course focus – grammar

3/4 assessment mode – K knows there is a test every two weeks and she knows what it checks

6 learner needs – speaking

6/7 target situation / context – communication in social situations, functioning in everyday transactions & interaction

From this it’s clear we need to focus on:?level,?topics/test content,?language content?&?skills content,?and?assessment mode.

You can also look at:?test type,?test format,?task types,?instructions,?range of language covered,?relevance,?marking?etc.

Regarding the ways in which it might have an effect the learner, you can consider:?relevance?to?learner’s needs,?first?impression on the learner,?learner’s?trust in the test?and the?reliability of the results,?how easy it is for the learner to know what to do,?level of?opportunity for the learner to perform their best,?level of encouragement for the learner within the test, ability of the test to confirm a sense of progress for the learner, amount of?evidence provided to the marker.

?

?

Paper 2 Task 1– practice task

(18 marks) – recommended time:?20 minutes

?

The text for this task is reproduced below. ?It is being used in the following situation.

?

N has lived in London for six months and, in order to improve her English, has enrolled on a course that teaches ‘ESOL Skills for Life’. These classes focus on language and language skills that are needed for people living, working and studying in England. The college gave her a test to assess her level in order to help find the most appropriate class for her. This task is part of the writing section of the test. The other language skills are tested in other parts of the test.

?

Using your knowledge of relevant testing concepts, evaluate the effectiveness of the tasks in this test for this learner in this situation.

?

Make a total of?six?points. You must include both?positive?and?negative?points.

?

Write your answer in your answer booklet.

?

Guideline answer

Positive points?

PP1. May produce varied sample of language (vocab, grammar, spelling etc.) so may be good for placement

PP2. A direct test of writing skills so generally learner can show her abilities directly / has validity?

PP3. Suitable for low levels because can be completed with limited language

PP4. Tests punctuation, letter formation, writing on the lines

PP5. There are six fresh starts

PP6. Open ended so learner can write what they like

PP7. Picture prompts mean learners don’t need to understand written prompts

PP8. Prompts supplied

PP9. Standard task type

PP10. Content relevant to learner’s needs/context

Positive applications

PA1. N may be placed at the correct level

PA2. Allows N to demonstrate her writing abilities

PA3. If N’s level is low, she will still be able to do the test. Can use compensation strategies.

PA4. N’s ‘mechanical’ skills will be tested (relevant to students in a second language learning situation)

PA5. If N can’t do one, she still has the opportunity to demonstrate her abilities in the others

PA6. N can relate to the pictures in ways which are relevant to their own life experiences / ideas

PA7. If N has low reading level, she will be able to respond to the pictures

PA8. N will not have to hunt for ideas on what to write about

PA9. N will be used to this kind of task type so this will help make her performance more reliable ?

PA10. N can identify with the situations

?

Negative points?

NP1. May not test range of language sufficient for placement at all levels / may induce error avoidance (poor predictive validity)

NP2. It only tests sentence level writing / there is no test of more complex features of writing (paragraphing, complex sentences, linking etc.)

NP3. It does not look like a test so lacks face validity

?

NP4. Task won’t test realistic genre / communicative competence relevant to skills for life e.g. application forms / is inauthentic / lacks content validity

NP5. Pictures are not always clear

NP6. May not be reliable because different learners will write different things

NP7. The pictures may elicit limited language e.g. present continuous

NP8. No example of / guidance on what kind of sentence to write e.g. describe the picture?

NP9. Tests specific items of vocab e.g. football, bucket

NP10. May be time consuming / impractical to mark

NP11. Range of different answers may make answers difficult to mark in terms of focus of marking/ Subjective element in marking may lead to unreliability

NP12. Situations may lack personal/cultural relevance to the learner

Negative applications

NA1. N may be misplaced because she hasn’t been allowed to demonstrate enough

NA2. May not discriminate in terms of N’s level

NA3. N won’t think this is a serious test

NA4. N may be placed in an unsuitable class

NA5. N may be confused about content to write

NA6. N won’t have confidence in her placement

NA7. N may not feel she can demonstrate her range of language

NA8. N may be confused about what to write

NA9. N may produce nothing for some items because she doesn’t know the vocab

NA10. N may have to wait for results

NA11. N may not get the mark she deserves and possibly be misplaced

NA12. Candidate may not be motivated to write / candidate may be put off their course of study / institution