مقایسه عملکرد آزمون‌دهندگان موفق، متوسط و ناموفق در امتحان درک مطلب انگلیسی در بکارگیری راهکارهای آزمون‌دهی

نشریه علمی آموزش و ارزشیابی (فصلنامه)

مقاله 7، دوره 10، شماره 38، شهریور 1396، صفحه 135-155   

نوع مقاله: مقاله پژوهشی

نویسندگان

مریم بهاردوست^{* 1}؛ علیرضا احمدی²

¹گروه زبان‌انگلیسی، واحد قشم، دانشگاه آزاد اسلامی، قشم، ایران

²دانشگاه شیراز دانشکده زبانهای خارجه و زبانشناسی

چکیده

به علت اهمیت نقش اساسی راهکارهای آزمون دهی در روند آزمون دهی ،هدف این مقاله بررسی تفاوت آزمون دهندگان زبان انگلیسی در بکارگیری راهکارهای شناختی و فراشناختی در آزمون درک مطلب زبان انگلیسی میباشد. همچنین هدف این تحقیق تفاوت عملکرد آزمون دهندگان در پاسخدهی به آیتمهای آزمون زبان میباشد.به این منظور 130 زبان آموز دختر که بین 19 تا21 سال سن داشتند  بر اساس دسترسی به شرکت کنندگان در این بررسی انتخاب شدند. ابزار مورد نیاز برای جمع آوری اطلاعات آزمون درک مطلب، پرسشنامه و مصاحبه بوده است.  راهکارهای حدس زدن، دانش قبلی ، ترجمه ، یادداشت برداری، و خط کشیدن زیر کلمات راهکارهای شناختی به کاررفته توسط آزمون دهندگان بود. علاوه بر این، راهکارهای فراشناختی شامل برنامه ریزی، خودمدیریتی و نظارت بود.تحلیل داده ها نشان داد که تفاوت معنادار در استفاده از راهکارهای شناختی و فراشناختی در آزمون درک مطلب زبان انگلیسی میان  آزمون دهندگان موفق نیمه موفق و ناموفق وجود نداشت. بااین وجود بر اساس یافته های مطالعه تفاوت معنادار در عملکرد آزمون دهندگان موفق نیمه موفق و ناموفق در پاسخدهی به سوالات(آیتمهای) امتحان درک مطلب وجود داشت. به نظر رسید که آزمون دهندگان موفق نسبت به آزمون دهندگان نیمه موفق و ناموفق در پاسخدهی به سوالات آزمون عملکرد بهتری داشتند

کلیدواژه‌ها

استراتژی‌های آزمون‌دهی؛ استراتژی‌های شناختی؛ استراتژی‌های فراشناختی؛ امتحان درک مطلب خواندن؛ آزمون‌دهی

اصل مقاله

Introduction

Undoubtedly, during the history of language assessment, English language testing has undergone various shifts and trends within the domain of language teaching and testing. By the introduction of the notion of “communicative competence” (Hymes, 1967), it became more obvious that learning a language is a process more than simple grammatical control.  Based on this approach, learners’ ability should be assessed within a context so that students could manipulate the language more efficiently (Behrahi, 2010). Therefore, during the last decades, the emphasis of language testing has shifted away from language form to language use leading to new demands on language testing domain (Canale, 1984).

On the other hand, advances in new theories including item response theory, in the 1990’s, brought about another revolution in the scope of language testing. In other words, language tests have been adapted to the ability of “individual test-takers” (Bachman, 1999). Obviously, “individual test takers” as well as diverse “strategies” have received substantial attention by testing practitioners and scholars.

Language testing domain, in general, has developed from teacher-centered approach and teacher-made tests towards learner-centered approach and test-taking strategies. Seemingly, the notion of learner autonomy has recently confirmed the growing importance of learner-centered approach. The mentioned approach tends to assume that exerting control on the learning process by autonomous learners will lead to greater success in their performance (Annani Sarab & Seif Reihani, 2010). 

As Bachman (1990) noted “language testing both serves and is served by research in language acquisition and language teaching” (p.2). Teachers can receive feedback about the effectiveness of instruction program. Moreover, they can make different decisions about different learners based on the results of the tests. Considering both learner’s strength and weaknesses, these decisions can involve using specific different types of learning materials and techniques (Bachman and Palmer, 1996).

Unfortunately, it has been observed that the ultimate goal of evaluation by some teachers has rarely been diagnosing strengths and weaknesses in test takers’ performance. Specifically, little attention has been paid to the process of taking tests by EFL students. Instead of paying considerable attention to test scores as the manifestation of learners' knowledge, teachers should stress constantly the importance of the other cognitive and psychological factors including test-taking strategies. Dodeen (2008) believed that the ability during test-taking process cannot be considered as the only element influencing learners’ performance. It is completely obvious that teachers must put more emphasis on both effective and ineffective ways the learners employ in test performance. Also, Fan (2003) argued that the strategies adopted by successful language learners have captured considerable attention of teachers, researchers, and scholars. Accordingly, how to perform better on tests can be considered as a source of worry for most students and teachers in almost all areas. Consequently, a large number of different teaching and learning studeis have already discussed various strategies in order to improve test performance some of which are actually used by learners taking tests (Zhang, Liu, Zhao & Xie, 2011).

In spite of their sufficient knowledge of English, EFL learners may not display good performance on their tests. Normally, Iranian English learners as EFL learners seem not to be an exception. It appears that Iranian EFL learners often act as passive recipients of knowledge. Ghafournia and Afghari (2013) argued that "through using appropriate strategic patterns, language learners can act as active self-reliant constructors of knowledge , able to self-direct, organize and undertake the process of language learning and test taking" (p.80). English teachers have noticed that Iranian EFL learners are often unsuccessful in their test performance because they have not been probably acquainted with test-taking strategies.

Likewise, attitudes towards reading process have recently been revolutionized. New perspective on reading process involves not only examining the words and structures of phrases and sentences, but also adopting various reading strategies. Specifically, the early studies on reading comprehension concentrated mainly upon analyzing the role of language elements (vocabularies, structures, etc.). However, currently, what has greater importance for researchers in reading comprehension and testing areas is the active role of the readers in the reading process. In other words, today, the center of attention is how the reader can direct his/her knowledge to the text in order to comprehend the writer’s intended meaning. Undoubtedly, one of the elements that readers can direct to the text is how to employ reading strategies (Rokhsari, 2012). 

Accordingly, there is consensus among researcher tht L2 reading can be considered as an interactive meaning-making activity (e.g., Alderson, 1984, 2005; Anderson, 1999; Carrell, 1988; Hudson, 1998; Zhang, Gu, & Hu, 2008). Therefore, it is essential for readers to use various techniques to fulfill the aim of reading comprehension (Zhang & Wu, 2009).

English teachers have generally observed that greater emphasis was laid on vocabulary learning and structural rules in language teaching settings of Iran. In other words, in spite of tendency of scholars and researchers towards communicative approach, teaching reading process in Iran has still followed grammar translation method in most of academic settings. Therefore, it seems that a fundamental shift away from traditional methods of teaching reading towards modern approaches is necessary in Iranian language learning contexts. By stressing the use of effective techniques including different strategies, Iranian EFL readers are more likely to achieve the goal of reading comprehension. 

Since the late 1970s, various studies have focused on test taking strategies employed by test takers (Cohen, 1998). In recent decades, there has been a growing concern among researchers about actual processes that test takers go through. Sheorey and Mokhtari (2001), for instance, mentioned that by the advent of psycholinguistic models of L2 reading, utilizing proper strategies for more effective reading comprehension has attracted much more attention. They continued that proper strategies may include previewing the written text, making use of contextual clues, and drawing inferences. Indeed, the researchers have recently explored how the test takers arrive at the answers to language assessment measures. Othman and Hj Jaidi (2012) claimed that some students do not employ proper reading strategies while reading texts. Similarly, Vattanapath and Jaiprayoon (1999) cited that some learners are expected to perform successfully in tests. However, they do not perform well due to the lack of test-taking strategies or use of poor strategies.

Different attitudes and definitions about test taking strategies were presented over a few decades. Test-taking strategies originated from the concept of ‘test-wiseness’ which was defined as "one’s capacity for using test characteristics and formats and/or test-taking situations to raise test scores" (Millman et al., 1965, cited in Ritter & Idol-Maestas, 1986, p. 50). Meanwhile, Jimenez et al. (1996) defined test-taking strategies as operations or steps employed by test-takers to facilitate the retrieval of information. Deanna (2002) maintained that cognitive and metacognitive strategies were involved in doing reading comprehension tests and that the former could be classified into key words, deduction, reasoning and reconstruction; and the latter, could be divided into planning, monitoring and evaluation.

Bachman and Palmer (2010) defined cognitive strategies as "the mental processes directly related to information processing to obtain, store, retrieve, or use information in learning or assessment settings" (p.56). Metacognitive strategies "allow learners to control their own cognition" (e.g., ‘I look for people to talk to in English’) (Oxford, 1990, p. 135).

All the definitions given by different scholars and researchers include common strategies that are somehow related to what test takers do and might do to solve test problems. Several Studies have indicated that, in reading, certain types of strategies are used by test-takers. For instance, Annani Sarab and Seif Reihani (2010) explored the use of cognitive and metacognitive strategies by adult learners with different levels of education across different fields of study. The results indicated that there was a moderate, positive, and significant correlation between strategy types and reading performances. Also, the results showed that the preferences for the cognitive and metacognitive strategies differed across levels of education. The findings of the study suggest that foreign language learning involves more than the acquisition of the target language, as learners develop cognitively, socially, and linguistically at the same time.

Ghafournia and Afghari (2013) investigated the significant interaction between reading comprehension cognitive test-taking strategies and the level of reading proficiency of Iranian MA students. More specifically, they sought the probable significant interaction between cognitive test-taking strategies and reading strategies. To this end, 947 Iranian MA students took a reading comprehension test and answered a cognitive strategy questionnaire. The findings showed that the participants at the high level of reading proficiency used cognitive test-taking strategies more significantly than the participants at the intermediate level. Significant interaction was found between cognitive test-taking and reading strategies.

 Furthermore, Phakiti (2003) carried out a study in order to examine the relationship between test-takers' use of cognitive and metacognitive strategies and their EFL reading test performance. The study used both quantitative and qualitative data analyses. There were 384 Thai university students for quantitative data analyses. Eight students including four highly successful and four unsuccessful students were selected for retrospective interviews.  All mentioned students took an 85-item multiple-choice reading comprehension achievement test and filled in a cognitive-metacognitive questionnaire. The results showed that the use of cognitive and metacognitive strategies had a positive relationship to the reading test performance. Moreover, more successful test-takers reported significantly higher metacognitive strategy use than less successful ones.

Besides, Rezaee (2005) made an attempt to find out whether knowledge and employment of EFL students' test taking strategies affected their achievement language test performance. Moreover, the aim of the study was to discover to what extent test takers employed test taking strategies differently in various parts of the test. To this end, the EFL learners were asked to respond to the items of a questionnaire of test taking strategies. By responding to the questionnaire items, the respondents could show the knowledge of test taking strategies as well as the degree of test taking strategies use. Another instrument used in the study was a syllabs-based language achievement test developed based on the courses the EFL learners covered over the first year of university. The results of the study revealed that there was a significantly high correlation between total scores of achievement test and the scores of the questionnaire. The findings also showed that the subjects in the study employed strategies with different degrees of inclination in different parts of the test. Rezarr (2005) claimed that "performance on language tests can be improved if both language teachers and test designers have a better insight into different strategies that the students apply" (p. 27).

Based on the issues mentioned above, the findings of this study may be helpful in filling the gap on test taking strategies use and its relations to reading comprehension performance. Despite the recent attempts made by researchers to deal with test-taking strategies, a basic need exists for more research on different test-takers in different contexts, especially in foreign language settings. More specifically, test takers with different levels of success in the use of different test-taking strategies have not extensively been investigated.

The specific aim of the present study inspired by inadequacy of research in test taking-strategies domain includes discovering the use of cognitive and metacognitive test-taking strategies by EFL learners in taking reading comprehension tests. Possible differences in their performance on different aspects of reading comprehension are also considered as the second objective of the present study.

To achieve the objectives of the study, the researchers developed the following questions:

 

1. How do the highly successful, moderately-successful and unsuccessful test-takers differ in the use of cognitive and metacognitive test-taking strategies?
2. How do the highly successful, moderately-successful and unsuccessful test-takers differ in terms of their performance on different EFL reading comprehension items (main idea, reference, detail, and vocabulary)?

 

 

 

 

 

Method

1. Participants

The participants in this study were 130 female EFL learners who ranged in age from 19-21 and were selected based on their availability. They were studying at Technical/Vocational University of Kashan (FADAK). These university students were majoring in different fields of study including architecture, electronics, computer science, and hotel management. All of the participants were native speakers of Persian learning English as a foreign language. Following Phakiti (2003), the participants were divided into three groups of highly successful, moderately successful, and unsuccessful test takers based on their performance on the reading comprehension test.

 

2. Measurement instruments

Research instruments in this study were as follows:

a)      A reading comprehension test

b)      A cognitive and metacognitive test taking strategies questionnaire

c)      Retrospective interviews

a)      Reading comprehension test: A multiple-choice reading comprehension test was developed by the teachers in Technical University of Kashan. The test consisted of ten passages with certain range of words. Each passage was followed by five multiple-choice questions. It was mainly designed to measure the test-takers' ability to read English texts for main ideas, details (factual information), references, and vocabularies (synonyms and antonyms). The topics in the test were related to the topics taught in the class, such as food, mysteries, and business. It is noteworthy that the reliability of the reading comprehension test was checked through Cronbach’s alpha formula. The reliability estimate was as high as .86. Content validity was also checked through content analysis done by two EFL professional experts. Some texts and items were modified accordingly.

 

b)      Cognitive and metacognitive test-taking strategies questionnaire: This questionnaire was adapted from Phakiti (2003) to find out cognitive and metacognitive test-taking strategies. The questionnaire was translated into Persian to improve the test-takers’ understanding of different strategies in items. It is noteworthy that two professional translators revised and edited the questionnaire in Persian in order to achieve an optimal validity. The questionnaire was then piloted with similar students doing the same course of General English in this university. The questionnaire was evaluated for reliability before its administration and distribution in the main study. The Cronbach’s alpha reliability coefficient turned out to be .92 which is high enough. Thus, no item was excluded. The questionnaire was a 5-point Likert Scale questionnaire including 5 options of 1(Never), 2(Sometimes), 3(Often), 4(Usually), and 5 (Always).

c)      Retrospective semi-structured interviews: Triangulation refers to "the generation of multiple perspectives on a phenomenon by using a variety of data sources, investigators, theories, or research methods with the purpose of corroborating an overall interpretation" (Dornyei, 2007, p.165).  By combining the test and questionnaire data with retrospective interview data, triangulation was performed in this study. The interviews were conducted to collect extra information in relation to research questions. The quantitative results appear to be explained more precisely and properly by the retrospective interview data. The interviews were carried out in group and in Persian language. More specifically, highly successful, moderately-successful, and unsuccessful students were interviewed separately.   About ten minutes were devoted for each interviewee. They were asked to report on strategies they adopted while completing the reading comprehension test.

 

3. Procedure

The participants, firstly, took the reading comprehension test as their final exam. The multiple-choice reading comprehension test was followed by a 35-statement questionnaire. Before the reading comprehension test was conducted, the participants received a full briefing on the way to answer the test and to complete the questionnaire. After marking the tests, the students’ level of success in the test was determined based on their scores. Accordingly, they were divided into three groups of highly successful, moderately successful, and unsuccessful test takers. Those with more than 1.5 standard deviations above or below the mean were considered to be highly successful and unsuccessful respectively and the ones in between were considered moderately successful. Afterwards, 10 highly successful, 10 moderately successful and 10 unsuccessful EFL learners were randomly chosen and interviewed. The interviewees were requested to express how to perform the test procedure and what strategies they employed in taking the reading comprehension test. The interviewees’ statements and impressions were recorded and then transcribed. In the next step, the researchers scrutinized the interviews and a set of test-taking strategies was identified.

 

4. Data analysis

To analyze the research questions, descriptive statistics, Chi-square, and MANOVA were used for quantitative data. Qualitative data analysis was also carried out on the interview data in order to supplement the findings of the study from the quantitative part. Therefore, the researcher reduced and divided the data into thematic categories and then the frequency of each category was calculated.

 

Results

In order to test the normality assumption of variables, one-sample Kolmogorov-Smirnov test was used. The results are shown in Table 1.

 

Table 1 One-Sample Kolmogorov-Smirnov Test
		Comprehending	Retrieval	Planning	Monitoring	Cognitive	Metacognitive	MARK
N		135	135	135	135	135	135	135
Normal Parametersa,b	Mean	21.806	12.030	45.791	29.737	67.350	152.565	28.76
	Std. Deviation	4.8529	3.3206	12.2725	7.1640	15.1872	37.0938	9.784
Most Extreme Differences	Absolute	.059	.070	.058	.068	.046	.053	.124
	Positive	.059	.070	.037	.052	.046	.053	.124
	Negative	-.049	-.064	-.058	-.068	-.035	-.047	-.064
Kolmogorov-Smirnov Z		.685	.816	.671	.795	.530	.617	1.439
Asymp. Sig. (2-tailed)		.736	.518	.759	.553	.942	.841	.032
a. Test distribution is Normal.
b. Calculated from data.

 

The results in Table 1 showed that all of variables including comprehending (Z=0.685, P=0.736), retrieval (Z=0.816, P=0.518), planning (Z=0.671, P=0.759), monitoring (Z=0.795, P=0.553), cognitive (Z=0.530, P=0.942), metacognitive (Z=0.617, P=0.841), have normal distribution at P< 0.01 (Z=1.439, P=0.032).

Table 2 also presents the results of frequency, percentage, and chi-square of cognitive and metacognitive strategies employed by highly, moderately, and unsuccessful EFL learners in reading comprehension test.

 

Table 2

The Results of Frequency, Percentage, and Chi-square of Strategies Used by the learners

Highly successful	Moderately successful	Unsuccessful	X2 P < 0.10
Cognitive Strategy	Inferring	Frequency	8	5	4	7.20
		% of Total	80%	50%	40%
	Guessing	Frequency	7	6	10	4.85
		% of Total	70%	60%	100%
	Translating	Frequency	5	7	4	1.87
		% of Total	50%	70%	40%
	Prior knowledge	Frequency	7	7	3	5.83
		% of Total	70%	70%	30%
	Note taking	Frequency	3	4	1	2.38
		% of Total	30%	40%	10%
	Underlining	Frequency	2	0	0	4.29
		% of Total	20%	0%	0%
Metacognitive Strategy	Planning	Frequency	7	7	8	.341
		% of Total	70%	70%	80%
	Self-management	Frequency	4	2	2	1.364
		% of Total	40%	20%	20%
	monitoring	Frequency	1	0	0	2.069
		% of Total	10%	0%	0%

 

As shown in Table 2, cognitive strategies used by highly successful learners, from the most frequent to the least frequent, included guessing, prior knowledge, translating, note taking, and underlining, respectively. Moreover, moderately successful learners used translating and prior knowledge strategies as the most frequent and note taking as the least frequent. Finally, cognitive strategies employed by unsuccessful learners, from the most frequent to the least frequent, involved guessing, translating, prior knowledge, and note taking. It is worthy to mention that neither moderately nor unsuccessful learners used underlining strategy. The results of chi-square also indicated that there was not a significant difference between translating, note taking, underlining, and the success level in reading comprehension test. However, there was a statistically significant difference between inferring, guessing and prior knowledge with the success level at P< 0.10. Furthermore, metacognitive strategies employed by highly successful learners, from the most frequent to the least frequent, involved planning, self-management, and monitoring, respectively. Moderately successful learners, also, used planning as the most frequent strategy and self-management and monitoring as the least frequent strategies. Finally, the most frequent metacognitive strategies employed by unsuccessful learners was planning; while the least frequent metacognitive strategies included self-management and monitoring, respectively. More specifically, neither moderately nor unsuccessful learners employed monitoring strategy. The results of chi-square also showed that there was not a significant difference between planning, self-management, monitoring, and the success level in reading comprehension test.

The descriptive statistics for the difference in three levels of success in terms of the performance on different reading comprehension items (Main idea, Factual information, Vocabulary, and Reference) are reported in Table 3.

 

Table 3

Descriptive Statistics for the Performance on Different Reading Comprehension Items in Three levels of Success

Items	Main idea		Factual information		Vocabulary		Reference
Mean & SD	M	SD	M	SD	M	SD	M	SD
Highly Successful	7.90	1.79	8.30	0.675	18.20	1.31	9.60	0.516
Moderately Successful	7.30	1.94	4.90	1.28	12.10	3.51	6.50	1.58
Unsuccessful	4.60	2.11	3.30	1.41	4.90	1.79	3.90	1.91

 

As displayed in Table 3, the means and standard deviations of highly, moderately, and unsuccessful learners in the item related to main idea were 7.90 (1.79), 7.30 (1.94), 4.60 (2.11), respectively. Furthermore, the means (and standard deviations) of highly, moderately, and unsuccessful learners in items of factual information were 8.30 (0.675), 4.90 (1.28), 3.30 (1.41), respectively. For vocabulary items, the means (and standard deviations) of highly, moderately, and unsuccessful learners were 18.20(1.31), 12.10(3.51), 4.90 (1.79), respectively. Also, for reference items, the means (and standard deviations) of highly, moderately, and unsuccessful learners were 9.60 (0.516), 6.50 (1.58), 3.90 (1.91), respectively.

       Moreover, the results of MANOVA test based on Royʼs largest Root for cognitive and metacognitive test taking strategies are reported in Table 4.

 

 

Table 4

Multivariate Test for reading comprehension test

Effect	Value	F	Hypothesis df	Error df	Sig.
Pillai's Trac Wilks' Lambda Hotelling's Trace Roy's Largest Root	.133 .870 .146 .121	.958 .936b .914 1.639c	4.000 4.000 4.000 2.000	54.000 52.000 50.000 27.000	.438 .450 .463 .213

 

The results of MANOVA in Table 4 showed that there was not a significant difference among the mean scores of highly, moderately, and unsuccessful learners, Roy=.121, F (4, 2) =1.64, p=.213.

Table 5, also shows the results of MANOVA test based on Roy largest root in four types of reading comprehension items.

 

 

Table 5

Multivariate Test for 4 types of items

Effect	Value	F	Hypothesis df	Error df	Sig.
Pillai's Trac Wilks' Lambda Hotelling's Trace Roy's Largest Root	1.111 .043 18.773 18.579	7.818 23.011b 53.972 116.118c	8.000 8.000 8.000 4.000	50.000 48.000 46.000 25.000	.000 .000 .000 .000

 

The results displayed in Table 5 showed that there was a significant difference between highly, moderately, and unsuccessful learners and their performance on different reading comprehension items, Roy=18.58, F (4, 2)=116.12, p=0.0001.

Furthermore, Table 6 reported the results of tests of between-subject effects.

 

 

 

 

 

 

 

 

 

 

 

Table 6

Tests of Between-Subjects Effects for test taking strategies

Source	Dependent Variable	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
Success Level	Cognitive	662.026	2	331.013	1.619	.217 .440	.107 .059
	Metacognitive	2284.191	2	1142.095	.846
Error	Cognitive	5520.039	27	204.446
	Metacognitive	36447.711	27	1349.915

 

As illustrated in table 6, there was not a significant difference among highly successful, moderately successful, and unsuccessful in using of cognitive strategies [F (2, 7) = 1.62, P=0.217] and metacognitive strategies [F (2, 7) = 0.846, P=0.440].

The results of the tests of between-subject effects for different item types are also presented in Table 7.

 

Table 7

Tests of Between-Subjects Effects for items of reading comprehension test

Source	Dependent Variable	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
Success Level	Main Idea  Factual Information                  Vocabulary                   Reference	61.800 130.400 886.467 162.867	2 2 2 2	30.900 65.200 443.233 81.433	8.069 47.450 77.010 38.040	.002 .000 .000 .000	.374 .779 .851  .738
Error	Main Idea  Factual Information Vocabulary   Reference	103.400  37.100 155.400 57.800	27 27 27 27	3.830 1.374 5.756 2.141

 

As indicated in Table 7, there was a significant difference among highly successful, moderately successful, and unsuccessful in responding to the main idea [F (2, 27) = 8.07, P=0.002, ŋ² =.37], factual information [F (2, 27) = 47.45, P=0.0001, ŋ² =.78], vocabulary [F (2, 27) = 77.01, P=0.0001, ŋ² =.85], and reference [F (2, 27) = 38.04, P=0.0001, ŋ² =.74]. Thus, according to the effect size, the greatest difference among highly, moderately, and unsuccessful learners would be in vocabulary, factual information, reference, and main idea, respectively.

Discussion

As mentioned above, the primary cognitive strategies employed by EFL test takers in reading comprehension test performance were inferring, guessing, translating, prior knowledge, note taking, and underlining. Planning, self-management, and monitoring can be also taken into account as the metacognitive test taking strategies used by the test takers. In other words, both cognitive and metacognitive test taking strategies were employed by the test takers in the present study. This finding was congruent with studies done by Purpura (1997), Phakiti (2003), Nikolov (2006), and Anani Sarab and Seif Reihani (2010). The results of their studies revealed that test takers used cognitive and metacognitive strategies in the process of answering to the reading comprehension items. However, it seems that the variety of employed cognitive test taking strategies was a bit more than metacognitive test taking strategies. If the participants in this study implemented broader types and categories of metacognitive strategies, they could probably improve their test taking process. The importance of metacognitive strategies in facilitating and improving reading comprehension has been recently emphasized by several researchers. Liu (2002) claimed that the Chinese successful learners performed much better than unsuccessful ones in the use of metacognitive strategies. Moreover, Xu (2007) conducted a study on the metacognitive awareness of reading strategies used by the college students. The results showed that the process of reading comprehension could be facilitated by participants’ awareness of various kinds of metacognitive strategies. Baker and Brown’s (1984) investigation also revealed that highly successful learners were higher metacognitive strategy users and more self-directed than unsuccessful learners. They carried out a study on the relation between metacognitive strategy and effective reading proficiency. With respect to the significance of metacognitive strategies for scholars, the participants in the present study were found to employ fewer categories of metacognitive strategies as compared with cognitive strategies. 

 Furthermore, as cited previously, highly, moderately, and unsuccessful learners did not differ significantly in the use of cognitive test-taking strategies. Highly successful test takers appeared to implement inferring metacognitive strategy as the most frequent strategy. By employing inferring strategy, the highly successful test takers might make reasonable inferences in specific items resulting in better performance on reading test. As reported in interviews, the item of main idea seemed to be problematic for some unsuccessful learners. More specifically, one unsuccessful test taker expressed that I found main idea question as the most difficult item, because when I completely read the passage, I couldn’t deduce the best main idea logically and appropriately from the passage. Consequently I resorted to guessing strategy. One explanation for poor performance of unsuccessful test takers in the reading test may lie in the fact that it has been too difficult to infer some ideas and statements in the passage. Moreover, the most frequently used cognitive strategy in the performance of both highly and unsuccessful learners were guessing. This finding supported the findings reported in the study conducted by Nevo (1989) and Anani Sarab and Seif Reihani (2010).The results in their studies revealed that the most frequently cognitive strategy employed by test takers during test taking process was guessing. The unsuccessful learners, however, employed guessing strategy even more than highly successful learners in the present study. Due to the results of their performance in reading comprehension tests, guessing strategy could not probably help unsuccessful learners in finding the correct answers. As it was revealed, highly and moderately successful learners employed prior knowledge strategy more frequently than less successful learners. Less successful learners, perhaps, did not use sufficient prior knowledge strategy due to the gap in their knowledge in reading comprehension tests. More specifically, lack of sufficient and careful study on reading comprehension texts in days prior to the examination day led to the less frequent use of this type of strategy by unsuccessful learners. In their interviews, majority of successful learners expressed that they used their prior knowledge in choosing the correct answeres especially in vocabulary items. They declared that they found the best synonyms or antonyms in alternatives regarding their prior knowledge and even sometimes without referring to the text. For example, one highly successful interviewee reported that “at first I considered the questions and their items. In vocabulary section, if I knew the meaning of the word, I chose the best synonyms or antonyms as soon as possible, because I had prior knowledge of that word. In case of vocabulary items, I preferred to apply my prior knowledge; Otherwise, I returned to the passage to find the meaning of the word regarding the context”.  It seems that successful learners have been so confident about their prior knowledge that they selected their best options without paying attention to the vocabularies in the context. The findings in Table 2 also indicated that underlining strategy was only used by highly successful learners. Neither moderately nor unsuccessful learners employed this sort of strategy. It was likely that the lack of various strategies use including underlining led to poor performance in the test by moderately and unsuccessful test takers.

The results of chi-square also showed that there was not a significant difference between planning, self-management, and monitoring, and the participantsʼ success level in reading comprehension test. Lack of awareness of diverse metacognitive strategies may lead to lower level of reading comprehension. Surprisingly, even highly successful learners did not utilize various metacognitive strategies. The most frequent metacognitive strategy utilized by unsuccessful test takers was planning. Even the frequency of using planning strategy by unsuccessful test takers has been higher that the frequency of planning strategy used used by highly successful test takers. It is obvious that if planning strategy had a positive effect on the unsuccessful test takers’ performance, they might have responded properly to the items of the test. In addition, highly successful test takers implemented self-management strategy more than moderately and unsuccessful test takers. It is likely that management of time, sequence of passages and items, and deciding on priority of items may affect positively on the performance of reading test process.  Furthermore, whereas highly successful test takers used monitoring metacognitive strategy, the moderately and unsuccessful respondents paid no attention to this type of strategy. The majority of respondents in this study had not apparently sufficient awareness of monitoring metacognitive strategy resulting in achieving low scores and therefore in unsatisfactory performance on reading comprehension test.

Moreover, the results of MANOVA revealed that test takers with different levels of success differed in terms of their performance on different reading comprehension items. More specifically, as shown in Table 3, highly successful test takers performed much more properly on reading test items than unsuccessful test takers. As mentioned above, the means of highly successful test takers in four types of items (main idea, factual information, vocabulary, and reference) have been higher than the means of moderately and unsuccessful test takers. In other words, highly successful test takers’ success was not related to a specific aspect; rather, they outperformed moderately and unsuccessful readers on all aspects of reading comprehension. It can be concluded that test takers in the present study have not actually employed specific strategies. But highly successful test takers seemed to use their general and prior knowledge in their performance rather than employing various strategies. Although interviewees claimed that they employed cognitive and metacognitive strategies, in practice, they have not actually used these strategies. As can be inferred from the results of chi-square and MANOVA, specific strategies were not probably implemented by test takers due to a non-significant difference in the use of test taking strategies. Based on to the findings of the present study, it appear that the participants of this study, like most of Iranian EFL learners, have not sufficiently been exposed to different strategies in educational settings. The researchers of this study have never experienced any special courses in learning and teaching reading or test taking strategies.

 

Conclusion and implications

This study was an attempt to examine a possible difference in employing cognitive and metacognitive test-taking strategies by highly, moderately and unsuccessful test takers in reading comprehension tests. Moreover, the performance of these three groups on different reading comprehension items was explored. The study demonstrated that there was not a significant difference between the used strategies and the success level in reading comprehension test. The results also revealed that highly successful test takers performed much better than moderately and unsuccessful test takers on all aspects of the reading comprehension test. On the contrary, unsuccessful readers received lower scores on reading tests due to the fact that they performed poorly on all aspects of reading comprehension and not on specific aspects. The findings of the present study and other pertinent previous researches attract wide attention to the role of test taking strategies. This study, especially, emphasized the importance of the role of metacognitive strategies due to the limited and poor use of this strategy by the test takers. The finding of this study has some implications for teachers in the realm of TEFL. Indeed, teachers can play essential roles in raising the awareness of learners’ cognitive and metacognitive test taking strategy use in order to develop their reading comprehension ability. It is likely that test takers’ performance would improve in testing process if they were taught various useful reading and test taking strategies. Rather than focusing on old methods such as GTM (grammar translation method), Iranian teachers should present more effective techniques in order to raise EFL learners’ language comprehension. The results of this study revealed that the participants were partly familiar with translating strategy; on the contrary, they might ignore the other important strategies (elaboration, summarizing, recombination, resourcing, selective attention, problem identification, etc.) due to inadequacy of instruction of diverse strategies in Iran’s educational and academic settings. Finally, if teachers monitor the method of test takers in test taking process through what they are actually doing and suggest more effective strategies, better results may be achieved.

مراجع

Alderson, J. C. (1984). Reading in a foreign language: A reading problem or a language problem? In J. C. Alderson & A. H. Urquhart (Eds.), Reading in a foreign language(pp. 1–27). London, UK: Longman.

Alderson, J. C. (2005).  Assessing Reading. Cambridge, England: Cambridge University Press.

Anani Sarab, M. R., & Seif Reihani, M. (2010). Cognitive and metacognitive strategy use and second language reading test performance: The case of Iranian intermediate EFL learners. IJAL, 13 (2), 1-18.

Anderson, N. J. (1999). Exploring second language reading: Issues and strategies. Boston, MA: Heinle & Heinle.

Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

Bachman, L. F. (1999). Language testing at the turn of the century: Making sure that what we count counts. AAA Letter, 20(2).

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford, UK: Oxford University Press.

Bachman, F., & Palmer, A. S. (2010). Language assessment in practice. Oxford: Oxford University Press.

Baker, L., & Brown, A. L. (1984). Metacognitive skills and reading. In P. D. Person (Ed.), Handbook of Reading Research. New York: Longman.

Behrahi, L. (2010). TheHistory of Language Testing. Retrieved from http://www.lorenglish.blogfa.com/post-5.aspx                  

Canale, M. (1984). Testing in a communicative approach. In G.A. Jarvis (Ed.), the challenge for excellence in foreign language education (pp.79-92). Middlebury, VT: The northeast conference organization

Carrell, P. L. (1988). SLA and classroom instruction: Reading. Annual Review of Applied Linguistics, 9, 233–242.          

Cohen, A. D. (1998). Strategies in learning and using a second language. England: Addison Wesley.

Deanna, L. N. (2002). Language learning strategies and English proficiency of Chinese University students(Unpublished doctoral dissertation). Regent University, China.

Dodeen, H. (2008). Assessing test-taking strategies of university students: Developing a scale and estimating its psychometrics indices. Assessment & Evaluation in Higher Education, 33(4), 409-419.

Dörnyei, Z. (2007). Research method in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford: Oxford University Press.

Fan, M. Y. (2003). Frequency of use, perceived usefulness, and actual usefulness of second language vocabulary strategies: A study of Hong Kong learners. Modern Language Journal, 87, 222-241.

Ghafournia N, Afghari A. (2013). The Interaction between reading comprehension cognitive test-taking strategies, test performance, and cognitive language learning strategies. Procedia- Social and behavioral Sciences, 70, 80-84.

Hudson, T. (1998). Theoretical perspectives on reading. Annual Review of Applied Linguistics, 18, 43–60.

Hymes, D. H. (1967). Models of interaction of language and social setting. Journal of Social, 33, 8-2

Jimenez, R. T., Garcia, G. E., & Pearson, D. P. (1996). The reading strategies of bilingual Latin students who are successful English readers: opportunities and obstacles. Reading Research Quarterly, 31, 90-112.

Liu, Y.C. (2002). Difference in choice of using English reading strategies between successful and unsuccessful learners. Foreign Language Teaching, 3, 39-42.

Millman, J., Bishop, H.I., & Ebel, R. (1965).  An analysis of test wiseness.  Educational and Psychological Measurement, 25(1), 707-726.

Nevo, N. (1989). Test-taking strategies on a multiple-choice test of reading comprehension. Language Testing, 6 (2), 199-215.

Nikolov, M. (2006). Test-taking strategies of 12-and 13- year-old Hungarian learners of EFL: Why whales have migraines. Language Learning, 56 (1), 1-51.

O' Malley, M. J., & Chamot, A.U. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press.

Oxford, R. L. (1990). Language learning strategies: What every teacher should know. Boston, MA: Heinle.

Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test performance, Language Testing, 20(1), 26-56.

Purpura, J. E. (1997). An analysis of the relationship between test-takers’ cognitive and metacognitive strategy use and second language test performance. Language Learning, 47(2), 289-325.

Rezaee, A. (2005). Test-taking strategies and performance in language achievement tests. Pazhuhesh-e Zabanha-ye Khareji, (20), 27-50.

Ritter, S., & Idol-Maestas, L. (1986). Teaching middle school students to use a test-taking strategy. Journal of Educational Research, 79, 350-357.

Rokhsari, S. (2010). An investigation of reading strategies used by Iranian EFL intermediate readers. Journal of Academic and Applied Studies, 2(8) & 2(9), 1-21.

Sheorey, R., & Mokhtari, K. (2001). Differences in the metacognitive awareness of reading strategies among native speakers and non-native speakers. System, 29, 431–449.

Vattanapath, R., & Jaiprayoon,K. (1999).An assessment of the effectiveness of teaching test-taking strategies for multiple-choice English reading comprehension tests. Occasional Papers, 8, 57-71.

Xu, X.M. (2007). A study of first-year college students’ metacognitive awareness of reading strategies. Chiang Mai University Journal, 1(1), 93-107.

Yahya Othman & Hj Jaidi, N. (2012). The employment of metacognitive strategies to comprehend texts among pre-university students in Brunei Darussalam. American International Journal of Contemporary Research, 2(8), 134-141

Zhang, L. J., & Wu, A. (2009). Chinese senior high school EFL students’ metacognitive awareness and reading-strategy use. Reading in a Foreign Language, 21, 37–59.

Zhang, L. J., Gu, Y. P., & Hu, G. (2008). A cognitive perspective on Singaporean bilingual children’s use of reading strategies in learning to read in English. British Journal of Educational Psychology, 78, 245–271.

Zhang, W., Liu, M., Zhao, Sh., & Xio, Q. (2011). English test-taking strategy use and students’ test performance. Asian EFL Journal, 13(2), 133-168.