3. OUR OWN PRETESTING JOURNEY

We conducted a speaking test following MUET standards for our second assignment. We made few changes compared to the actual MUET test. In our test we had 5 candidates instead of 4 candidates.

We had the vetting session prior to the main trial whereby we vetted the questions together with our big group. So in sense we consider this as our “Pilot Test”. We spent at least 2 hours on the questions and other aspects. We had many disagreements regarding the timing and also mark divisions. As our test was rather a subjective test, we had less trouble in constructing the test itself but had troubling in assessing the candidates. There were few spelling errors and other errors highlighted in the vetting session.

We conducted our Main Trial on 30 students. We had separate sessions for every group. Most of the session was conducted in different venues. It is not an easy task to conduct the test as it took time and effort from both the test takers and makers. Some of the test even couldn’t be completed in the required time as some were rushing or they had nothing to speak about. We even received few feedbacks from the students stating that the topic of our test was not interesting. We realised many new things that we did not come across in the earlier part of our vetting session.

It is very interesting that there are certain things that only the candidates are able to see and judge. So pretesting is something that is to be given priority in designing and coming up with exam questions. 

our exam question

instruction to candidates

1. WHAT IS PRETESTING

Pretesting is actually a process where the test items (the questions) are actually tested before the items are released into tests and also exams. Pretesting ensures the exams are accurate and fair for all students. It also ensures the test content is appropriate and that students from various background can equally score for it as well. There are few parts of pretesting that would be explained in this blog.

Pilot testing

Before the main trial is carried out, a less formal pretesting would be carried out that is called “pilot testing”. This pilot testing varies from testing it on some colleagues to students. The aim of this pilot testing would be to identify and rectify all problems before the main trial. There are few methods of carrying out the pilot testing. There are two types given here:

>Testing on colleagues/fellow teacher

-at least 2 of them are native speakers of English

They are to check whether:

-The instructions are clear

-The language of the items are acceptable

-The answer key is accurate   

>Testing on a group of students from the similar background and level. This will provide information on:

-The time students need for completing it

-The clarity of instructions

-The kind of language

-The accuracy and comprehension of any answer keys

-The usability of the marking scales

-Pilot testing will reveal many flaws and thus will save time and effort when the main trials are carried out. 

The main trials

Here we would be discussing on how to conduct main trials.

The main trials are where the test items are tested on students possibly replicating the real exam. The focus here would be on multiple-choice items as they are difficult to construct. The test constructers usually miss the ambiguities in the distractors. Thus these type of questions would require more pretesting compared to other types of questions.

There are few aspects that should be taken into consideration when it comes to the main trials:

v  The sample should be representative of the final candidature

The pretest must have a similar range of abilities and background. The results of the pretest would be useless if the expected test population is not similar. Henning 1987 recommends 1000 students for trial multiple-choice test. Basically, the more students taking the pretest, the better it would be.

v  Trial test students should take the test seriously

If the students do not take the test seriously then the whole trialling process would be useless.

v  Trial test should be administered exactly the same way as the final exam will be

So that the administration guideline together test item can be presented under same circumstances as in the live exam.  

5. BENEFITS OF PRETESTING

 

Have you watched the video above? Did you notice how the driving instructor pretested the condition of the car and the driver's readiness before proceeding with the test? Well, this video clearly portrays how beneficial pretesting can be where what is being used as a medium for testing (the car) and the student's (driver) readiness is checked from various angles through the appropriate instructions given by the examiner (the driving instructor).

Well, we have given you clear insights on what pretesting in education is all about. Hence, just like what you observed in the video, it is vital to pretest students before assessing them for the final test. The test paper needs to be checked if it is of good 'quality' and the students' abilities or knowledge needs to be kept as reference in coming up with a solid test paper. 

Therefore, there are several benefits of pretesting as follows:-

A) For the teachers/examiners, they will be able to...

1.  detect the learning gaps that occurs in students.

A learning gap is the difference between what a student has learned—i.e., the academic progress he or she has made—and what the student was expected to learn at a certain point in his or her education (The Glossary of Education Reform, 2013). Pretesting will enable teachers to detect any visible learning gaps in students as if this gap is left unattended to, it may lead students to struggle academically and socially. Teachers can use the results of the pretest and prepare the right methods to close this gap using appropriate instruments in their lessons.

2. determine if the target of the assessment is achieved or not.

When test constructors creates tests,  they will have a target that is to achieve from the assessment that is to be carried out. Hence, pretesting will enable teachers to determine if their instructions or test contents have actually brought out their students' maximum potential according to their learning levels. Hence, from the pretest results analysis, teachers can gain the data to be clear if their target has been achieved or not as well as its contributing factors to its success or failure to do so.

3. ensure the instructions to be more individualized.

Individualized instruction is instruction that considers the needs of the students (A Hypertext History of Instructional Design). Ideally, the students would control the pace at which they progress through instruction and the materials they use would be suited to their cognitive skills and learning styles (Gagne et al., 1992). Thus, pretesting will enable teachers to know to what level does their instruction caters to the students' individual needs in performing well.

4. make appropriate changes in their teaching methodology.

 

Through the results of a pretest, teachers will be able to make appropriate changes to the teaching methodologies. There is a possibility that students poor performance in a test may be due to their inability to connect with what is being taught in the classroom. Thus, teachers use pretests to detect which part of their teaching that students find it hard to cope with and see to it that they can coach from a different angle to stimulate their interest towards topics their are not good in.

B) For the students, they will be able to have a preview of the final test. This is where they can recognize their strengths and areas that need improvements. Thus, pretesting will ensure that students know their level of progression within their classroom learning and will make efforts to do better in the final test.