7. FREQUENTLY ASKED QUESTIONS (FAQ)

1. What is the difference between pretesting and trialling?

They both involve trying out test materials on a representative sample of the test group to gather various types of information about their performance and measurement characteristics.

·         Pretesting is a general term for this kind of activity, but it is also use more specifically to refer to occasions when test materials are administered to large groups of test-takers in order to carry out a range of statistical studies on the scores produced.

·         Trialling is often used to refer to a form of pretesting involving much smaller groups of test-takers who can provide useful feedback on different performance aspects of the test materials.

2.      How can pretesting help teachers and learners?

It helps learners to know which areas they need most practice in, and gives them experience and confidence in taking tests. For teachers, it helps highlight areas where their learners need more help.

3. Are items/questions pretested? If yes, how are students selected?

Most of the examination boards did pretesting; as most of them think that it has pretested items on a few students “as security of material is essential” and the other said test-types and difficult items were pretested informally by settlers/examiners in their own school using secure candidates.

4.If items/questions are pretested, how are they being analyzed?

Majority wise, for objective tests, there are two methods being applied to calculate for each objective test item.

(i) The facilty value (F.V.)

(ii) The discrimination index (D.I.)

However, for subjectively marked tests like summaries, essays and oral interview, there will be no definite statistic can be used. However, there will be marking guideline and criteria to ensure the trial marking session should follow the general patterns.

5.What happens if trial items/ questions are unsatisfactory?

If items proved to be unsatisfactory, they were either rewritten or discarded. However, there are some testing centres like UCLES which have said that any revised items will have to go through another pretesting cycle. Every time an item is rewritten it should be tried out again, as there is no guarantee that an edited item will be any more successful than the previous item. In reality, retrialling of items seems to be impossible, for that case, item analysis should be carried out after the live test has been administered, and faulty items should be omitted from the final scores.

6.What steps, if any, are to be taken to monitor the quality of individual item/ test writers        

·The designers or the objectively marked papers were involved in editing sessions and were monitored by the overseeing chair and the subject officer.

·A training course for item writers is provided.

·Draft tests were monitored and feedback was given to writers before the exam.

· Markers provide feedback after the administration of an exam, and the item writers’ work was then monitored.

·Those item writers with “moderation revealed quality” while unsatisfactory writers were “discontinued”.