Learning Objectives of IELTS Listening and Reading Tests: Focusing on Revised Bloom’s Taxonomy

Research in English Language Pedagogy

دوره 9، شماره 1 - شماره پیاپی 16، فروردین 2021، صفحه 182-199   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.30486/relp.2021.1916940.1244

نویسندگان

Samira Baghaei¹؛ Mohammad Sadegh Bagheri^{* 2}؛ Mortaza Yamini³

¹Department of English Language, Shiraz Branch, Islamic Azad University, Shiraz, Iran

²Department of English Language, Shiraz branch, Islamic Azad University, Shiraz, Iran.

³Zand Institute of Higher Education, Shiraz, Iran

چکیده

The quantitative-qualitative content analysis study reported in this paper investigated if there was any significant difference between the listening and reading sections of IELTS tests with regard to the representation of learning objectives of Revised Bloom’s taxonomy. For this purpose, 16 Academic IELTS listening and reading tests from Cambridge IELTS Academic: Authentic Practice Tests (IELTS 12, 13, 14, and 15) were selected as the material of the study. The content of the tests was codified based on a coding scheme developed by the researchers. The reliability of the coding was evaluated through the inter-coder and intra-coder reliability analyses. The frequency, Chi-square and Cramer's V tests were employed to analyze the data.The results indicated that IELTS listening and reading tests mostly included Understanding Factual and Conceptual Knowledge, respectively. Furthermore, the results showed that there was a substantial difference between IELTS listening and reading tests with regard to the inclusion of learning objectives. It was concluded that the listening and reading tests of IELTS assessed different learning objectives. The implications of the study suggest that IELTS candidates, teachers, and researchers should take the different learning objectives represented in IELTS listening and reading tests into consideration.

کلیدواژه‌ها

IELTS Learning objective؛ Listening؛ Reading؛ Revised Bloom’s Taxonomy؛ Learning objective

اصل مقاله

1. Introduction

The proficiency level of applicants is considered as an essential admission criterion in most English-medium universities. IELTS as the most widely used academic language test provides reliable evidence of applicants’ proficiency which are then employed in the admissions decision-making process. Besides, as Suryaningsih (2014) pointed out, IELTS tests have the power to refine knowledge suggesting that these tests can control what should be taught and what should be learned.

A narrow focus on the learning objectives and cognitive dimensions delineated in the IELTS tests may be a plausible reason why the construction of IELTS receptive skill tests are unfamiliar and problematic for most IELTS candidates (Aryadoust, 2012). An attempt to bridge this knowledge gap should be made, because the cognitive domains presented in test items can determine the difficulty of the test (Brown, 2000). In fact, understanding what learning objectives and cognitive levels are expected in the test items of high-stakes tests is a pressing issue facing language testing.

In recent decades, the cognitive processes of test-takers performing the IELTS exams stimulated plethora of studies (e. g. Aryadoust, 2013; Bax, 2015, 2013; Chan, Bax, & Weir, 2017). All these studies focused on how test-takers employ cognitive processes in performing under the IELTS test conditions. What remains unclear is the cognitive processes incorporated the IELTS tests beyond test-takers’ cognitive operations completing the tests.

Revised Bloom’s Taxonomy as a framework which has been broadly used in determining the cognitive processes included in test items (Baghaei, Bagheri, & Yamini, 2020; Ebadi & Sahbazian, 2015; Momsen et al., 2013) and evaluating textbooks with regard to the inclusion of the learning objectives (Mizbani & Chalak, 2017; Mizbani, Salehi, & Tabatabaei, 2020; Razmjoo, & Kazempourfard, 2012) can usefully supplement and extend the knowledge regarding different levels of learning objectives and cognitive processes represented in the IETLS tests.

Due to the determining role of IELTS tests in test takers’ lives and language teaching and learning practices, the present study aimed at shedding further light on this issue by examining if there was any significant difference between IELTS listening and reading tests in their inclusion of learning objectives provided in the Revised Bloom’s Taxonomy. The results of the present research contribute to IELTS candidates. The results can be used as a reference framework to IELTS applicants to understand the learning objectives and the cognitive processes included in the IELTS listening and reading tests. Additionally, the results can be of broad use to IELTS trainers. Equipped with the knowledge of the learning objectives of the IELTS listening and reading tests, IELTS instructors can align their practices with the nature of questions.

Furthermore, it is important that the cognitive domains presented in IELTS tests be examined because the presentation of cognitive domains may further justify the differences among test-takers’ performance. To date, little is known about cognitive domains included in IELTS tests. Consequently, conducting more research on this topic seems to be quite productive.

 

2. Literature review

IELTS as one of the most prominent international high-stakes English language proficiency test, thoroughly assesses candidates’ proficiency in the four main communication skills (IELTS, 2015). Concerning the educational and social effects of the IELTS as a high-stakes test, it is really essential to know about the language structure of this test.

There has been a growing amount of literature in recent years on various aspects of IELTS tests. However, a search of the literature revealed that few studies have explored the cognitive domains and learning objectives included in the IELTS tests (Aryadoust, 2013; Aryadoust & Goh, 2009; Baghaei et al., 2020, Chan, Bax, & Weir, 2017; Field, 2009). For instance, adopting the Revised Bloom’s Taxonomy, Baghaei et al. (2020) compared the IELTS and TOEFL listening and reading tests. Revised Bloom’s Taxonomy classifies learning objectives based on cognitive processes and knowledge dimensions. Baghaei et al. (2020) found that IELTS listening tests mostly focused on Remembering and Understanding Factual Knowledge, respectively, while the listening section of TOEFL tests focused on Remembering, Understanding, and Analyzing Factual Knowledge, respectively. The researchers also reported that lower-order thinking skills had a higher frequency in the listening section of IELTS compared with that of TOEFL. Moreover, a statistically significant difference was found between the reading test items of IELTS and TOEFL in terms of the inclusion of learning objectives. The results of their study showed that the reading section of TOEFL included seven learning levels, while the reading tests of IELTS encompassed three ones. The findings also demonstrated that IELTS reading tests covered only lower-order thinking skills, while TOEFL reading tests included both lower and higher-order thinking skills.

Owen (2016) who adopted Khalifa and Wier’s (2009) cognitive hierarchy model of reading analyzed IELTS and TOEFL iBT test-takers’ verbalizations to investigate the cognitive processes activated by IELTS and TOEFL iBT reading tests. He found that, including creating an intertextual representation, inferential reasoning and the formation of a text-level representation, some of the higher-level cognitive processes in the Khalifa and Weir’s model were not sufficiently reflected in the IELTS and TOEFL.

Ghahramanlou, Zohoorian and Baghaei (2016) explored the cognitive processes underlying the listening comprehension section of IELTS and examined whether they differ in terms of difficulty. They discovered that the most challenging operations for the listeners were to keep up with the speed of the speaker and to understand reduced forms. Taylor and Weir (2012) also argued that The IELTS listening test does not recreate genuine experiences encountered by its examiners in educational contexts.

To determine the cognitive processing of IELTS test takers performing the reading tests, Bax (2013) compared the reading behaviors of successful and unsuccessful test takers while completing the tests. Focusing on local reading, he found that proficient test takers use different eye movement behavior from less proficient ones. Bax (2013) presumed the different behavior of proficient test takers as the representative of different cognitive processing.

Changing the focus of the study from local reading to both local and general reading, Bax (2015) applied an eye-tracking method to scrutinize the cognitive processes of multinational readers during an IELTS reading test. The results were in accord with the findings of his earlier study. He reported that cognitive strategies used by the successful IELTS candidates were congruent with the types of strategies expected in real-life academic situations, whereas the cognitive strategies used by unsuccessful candidates were not.

An item analysis of students’ performance was conducted by Alderson and Lukmani (1989) on the items decided by judges measuring specific subskills at lower or higher levels of difficulty. The findings revealed no association between the difficulty of the item and the level of the item.

Detailed examination of the cognitive validity of the lecture-listening component of the Listening tests of IELTS by Field (2009) showed that multiple choice question and gap filling items in the fourth section of the IELTS listening test engage construct-irrelevant skills. Overall, Field (2009) concluded that the lecture-listening component of the IELTS Listening tests mostly included the lower level comprehension processes. As the audio materials had limited redundancy and encompassed detailed information, high cognitive demand was imposed on the test takers. Field’s (2009) results were supported by Aryadoust and Goh (2009) who explored Rasch-based differential item functioning of the IELTS listening test.

Weir, Hawkey, Green and Devis (2009) who followed Khalifa and Weir’s (2009) paradigm of cognitive processing in reading tests investigated the cognitive domains included in the IELTS academic reading. They concluded that in the reading section of IELTS, the relationship between item type and response strategy is not straightforward meaning that item type cannot reliably predict the pattern of used strategies.

Although, as stated before, a number of studies explored the cognitive processes underlying IELTS tests based on the effects of test tasks on test takers’ performance, little is understood about the cognitive levels and learning objectives represented in the test items. More specifically, previous studies were most often descriptive determining the cognitive processes in test takers’ performance in one section of IELTS tests. In fact, to the researchers’ best knowledge, only Baghaei et al. (2020) investigated the underlying learning objectives in two parts of IELTS and TOEFL tests. However, much uncertainty still exists about the difference between the sections of IELTS tests regarding the underlying learning objectives and cognitive processes. Therefore, the current research set out to compare the two receptive skill parts (i.e., listening and reading) of IELTS tests.

 

2.1 Theoretical framework

Revised Bloom’s Taxonomy (Anderson & Krathwohl, 2001) emerging out of Bloom’s original taxonomy was adopted as the theoretical framework of the current research study. Revised Bloom’s Taxonomy is one of the most famous and commonly used taxonomies in the field of education. The revised version of Bloom’s taxonomy offers a multi-tiered model of learning objectives and cognitive domains.

Focusing on the thinking skill levels that are pivotal to learning, the Revised Bloom’s Taxonomy encompasses two-dimensions. This taxonomy embraces the six levels of the cognitive domains: Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating (Anderson & Krathwohl, 2001).

Remembering as the lowest cognitive level of this taxonomy is defined as remembering learned information or recalling and retrieving learned material from long-term memory. Understanding, as the second cognitive domain, includes identifying the meaning of oral, written or graphic material. Applying, the third domain, refers to the use of learned material in a novel context. Analyzing concerns the ability to evaluate a problem area and break it down into its individual constituents, determining the relationship between different parts.  Evaluating includes the ability to make judgments on the basis of criteria or to integrate different sections to construct a new idea. Creating, as the highest level of the cognitive hierarchy, encompasses elements of all the other categories, plus generating ideas or novel ways of viewing things (Anderson & Krathwohl, 2001). Figure 1 demonstrates the hierarchical cognitive levels of the Revised Bloom’s Taxonomy.

 

 

Figure 1. Cognitive levels of the Revised Bloom’s Taxonomy (Anderson & Krathwohl, 2001)

 

The Knowledge dimension of the Revised Bloom’s Taxonomy consists of four types of knowledge: factual, conceptual, procedural, and metacognitive. Anderson and Krathwohl (2001) defined the four knowledge dimensions as follows.

• Factual Knowledge refers to knowledge of basic elements and discrete facts.
• Conceptual Knowledge includes the knowledge of the relationship between facts and elements, and categories.
• Procedural Knowledge is the knowledge of various processes such as using skills, methods, and techniques.
• Metacognitive Knowledge as self-knowledge refers to the knowledge of strategies and cognitive tasks.

The structure of Revised Bloom’s taxonomy is depicted in Table 1.

 

Table 1

Revised Bloom’s Taxonomy (2001)

Knowledge Dimension	Cognitive Process Dimension
	Remember	Understand	Apply	Analyze	Evaluate	Create
Factual Knowledge
Conceptual Knowledge
Procedural Knowledge
Metacognitive Knowledge

 

3. Research Question

The present study seeks to address the following research question:

1. How do listening and reading tests of IELTS differ in terms of the representation of learning objectives presented in Revised Bloom’s taxonomy?

 

4. Methodology

4.1. Design

The current study followed a quantitative-qualitative content analysis design. The content analysis is a systematic, rule-guided method used to evaluate the informational contents of textual data (Mayring, 2000).  Quantitative content analysis is a research method to systematically categorize and record features of textual, visual, or aural material in order to statistically analyze them (Coe & Scacco, 2017). On the contrary, the qualitative content analysis aims to interpret the content of text data through systematically classifying the coding and recognizing patterns or themes (Hsieh & Shannon, 2005). According to Krippendorf (2004), the best content analyses should incorporate both quantitative and qualitative methods. Following a quantitative-qualitative content analysis design, the present study attempted to examine both manifest and latent content of patterns included in the IELTS listening and reading tests.

 

4.2 Coders

IELTS listening and reading test items were coded by three coders (three researchers of the present study): a PhD candidate and two university professors of Teaching English as a Foreign Language (TEFL). Before the study, a training session was conducted and the coders were provided with the coding scheme.

 

4.3 Materials

As the live IELTS test versions were not accessible for researchers, 16 IELTS tests from four editions of Cambridge IELTS Academic Practice Test were selected as the materials of the study. Cambridge IELTS Academic Practice Test Series contain four authentic and complete IELTS practice tests for academic module prepared by Cambridge ESOL. The tests presented in these books correspond to the IELTS specifications and are representative of authentic IELTS test materials.

The researchers selected 16 listening and reading IELTS tests represented in the four most recent editions of Cambridge IELTS Academic Practice Test, IELTS 12(Cambridge University Press 2017), IELTS 13(Cambridge University Press 2018), IELTS 14(Cambridge University Press 2019), and IELTS 15(Cambridge University Press 2020),as the materials of the study. As the IELTS listening and reading tests included 40 items, totally 640 IELTS listening and 640 IELTS reading test items were codified and analyzed.

 

4.2.1. Academic IELTS Listening Test

The academic IELTS listening test comprises four sections and a total of 40 questions. Focusing on everyday social conversation, sections one and two include a conversation between two speakers and a speech by one person, respectively. Sections three and four consist of a conversation including three or four speakers and a monologue regarding academic subjects. IELTS listening test encompasses a wide range of item types, such as multiple choice, sentence completion, matching, plan/map/diagram labelling, form/note completion, and table/flow-chart/summary completion. It is also noteworthy that the number of different question types is variable.

 

4.2.2. Academic IELTS Reading Test

Academic IELTS Reading Test includes three long texts ranging from the descriptive and factual to the discursive and analytical and 40 questions with a variety of question types such as multiple choice items, identification of the writer’s views, matching, headings, features, and sentence endings, sentence completion, summary, short-answer items and flow-chart/diagram label completion. It is also worth noting that the number of various item types is different.

Various reading skills (e.g., skimming, reading for gist, reading for the main idea, reading for details, understanding logical argument and identifying writers’ opinions, purpose and attitudes) are evaluated in the IELTS reading tests.

 

4.3. Coding Scheme

Based on the Revised Bloom’s Taxonomy, a coding scheme including six cognitive levels ranging from Remembering to Creating and four knowledge levels ranging from Factual to Metacognitive Knowledge was used to codify and classify the test questions. Concerning the thinking skills, as Anderson et al. (2001) indicated in their book, the lower-order thinking skills included the three lower levels (Remember, Understand, and Apply) and the higher-order thinking skills embraced the three higher levels (Analyze, Evaluate, and Create). For better understanding, Table 2 depicts the coding scheme.

 

Table 2

Coding Scheme based upon Revised Bloom's Taxonomy

Knowledge Dimension	Cognitive Process Dimension
	1. Remember	2. Understand	3. Apply	4. Analyze	5. Evaluate	6. Create
Factual Knowledge	F1	F2	F3	F4	F5	F6
Conceptual Knowledge	C1	C2	C3	C4	C5	C6
Procedural Knowledge	P1	P2	P3	P4	P5	P6
Metacognitive Knowledge	M1	M2	M3	M4	M5	M6

 

4.4. Data collection

In the first step of assessing the learning objectives of test questions, based on the Revised Bloom’s Taxonomy, three coders separately codified the test items. It is worth noting that the coders were prepared for the task through a two-hour training session in which Revised Bloom's Taxonomy including the cognitive and knowledge domains was explained to them in detail by one of the researchers.

As Rawadieh (1998) conceded, classification of test items into the learning objective levels presented in the Revised Bloom's Taxonomy is a difficult process. In fact, the complexity of coding and fuzzy boundaries among the learning levels included in this taxonomy made the researchers to run the intra-coder and inter-coder reliabilities. One of the researchers codified the data twice over a span of two weeks. According to Altman’s (1999) guidelines, the findings of Cohen’s Kappa (κ =.85) showed that the two sets of coding had a perfect agreement. Additionally, the data were codified by two other researchers of the study. To finalize the coding, three coders resolved the areas of disagreement and came to a consensus on the codes. Fleiss kappa was run to determine the level of agreement among the three sets of codings. The results of the analysis revealed an acceptable consensus (κ = .71) among the three sets of codings.

 

4.5. Data analysis procedures

Data were analyzed following this procedure:

1. To establish whether the coding was reliable, the researchers ran both inter and intra-rater reliabilities. To estimate the consistency between the coders, Cohen’s Kappa reliability was performed in Statistical Package for Social Sciences (SPSS). Additionally, the coders codified the test items twice in a two-week time span and the researchers ran the Fliess Kappa reliability in Minitab software to explore the level of consistency between the two coding attempts.
2. Statistical analysis was performed using SPSS 25. First, the frequency was estimated for the learning levels presented in each test. The Chi-square test for independence was carried out on the frequency of learning objectives to know if there was any statistically significant difference between the reading and listening sections of IELTS tests with regard to the representation of learning objectives. Another Chi-square test for independence was also performed on the frequency of lower-order and higher-order thinking skills to assess if there was any statistically significant difference between the representations of these thinking skills in the reading and listening sections of IELTS tests.

 

5. Results

The objective of the study was to explore if there was any statistically significant difference between the representations of learning objectives in the IELTS listening and reading tests. Tables 3 and 4 display sequentially the results of the frequency and chi-square test for independence.

 

Table 3

Frequency of Learning Objectives Included in IELTS Listening and Reading Tests

		Category					Total
		F1	F2	C2	C4	C5
Skill	Listening	124	427	41	48	0	640
	Reading	37	83	494	0	26	640
Total		161	510	535	48	26	1280
a. Test = IELTS

 

As Table 3 illustrates, IELTS listening items covered four learning objectives: F1 (Remembering Factual Knowledge), F2 (Understanding Factual Knowledge), C2 (Understanding Conceptual Knowledge), and C4 (Analyzing Conceptual Knowledge), and IELTS reading items included four learning objectives: F1 (Remembering Factual Knowledge), F2 (Understanding Factual Knowledge) and C2 (Understanding Conceptual Knowledge), and C5 (Evaluating Conceptual Knowledge).

 

Table 4

Chi-square test on Learning Objectives of IELTS Listening and Reading Tests

	Value	df	Asymp. Sig. (2-sided)	Cramer’s V
				Value	Approx. Sig.
Pearson Chi-Square	736.612a	4	.000	.759	.000
Likelihood Ratio	858.397	4	.000
Linear-by-Linear Association	499.766	1	.000
N of Valid Cases	1280
a. Test = IELTS

 

The Chi-square test revealed a statistically significant difference between the listening and reading tests of IELTS (X² (4, N = 1280) = 736.61, p < .01). Therefore, it can be inferred that the primary emphasis of IELTS listening test was mostly on Understanding Factual Knowledge (F2), while the majority of IELTS reading items covered Understanding Conceptual Knowledge (C2). Figure 2 demonstrates the frequency of learning objectives in the listening and reading tests.

 

 

Figure 2. Clustered Bar Chart of Learning objectives in the Listening and Reading Tests of IELTS

 

In the next step, the researchers investigated the frequency of lower-order and higher-order thinking skills in IELTS listening and reading tests. Table 5 summarizes the results of frequency.

 

Table 5

Frequency of Learning Objectives in the Listening and Reading Tests of IELTS

		Thinking Skill		Total
		Lower-order	Higher-order
Tests	Listening	592	48	640
	Reading	614	26	640
Total		1206	74	1280
a. Test = IELTS

 

As can be seen in Table 5, both IELTS listening (F=592) and reading (F=614) tests mostly emphasized the lower-order thinking skills. Table 6 shows the results of the chi-square test for independence.

 

Table 6

Chi-square Test on Lower and Higher-order Thinking Skills in the Listening and Reading Sections of IELTS

	Value	df	Asymp. Sig. (2-sided)	Exact Sig. (2-sided)	Exact Sig. (1-sided)
						Value	Approx. Sig.
Pearson Chi-Square	6.942a	1	.008			.074		.008
Continuity Correctionc	6.325	1	.012
Likelihood Ratio	7.042	1	.008
Fisher's Exact Test				.011	.006
Linear-by-Linear Association	6.936	1	.008
N of Valid Cases	1280
a. Test = IELTS

 

The Chi-square test for independence indicated a significant difference between the listening and reading tests of IELTS with regard to the inclusion of the two levels of thinking skills (X² (1, N = 1280) = 6.94, p < .01). Based on the results of frequency in Table 5, lower-order thinking skills were more prevalent in IELTS reading tests (F=614) than IELTS listening tests (F=592). In addition, the association strength test (Cramer's V) demonstrated that the strength of relationship between the assessments and inclusion of learning objectives was very weak.

 

6. Discussion

The main goal of the current research was to compare the IELTS listening and reading tests with respect to the inclusion of learning objectives represented in the Revised Bloom’s taxonomy.

The findings showed that although IELTS listening and reading tests emphasized on Understanding which is “the largest category of transfer-based educational objectives” (Anderson et al., 2001, p. 70), the inclusion of learning objectives in these two tests was significantly different.

The findings indicated that the IELTS listening test items mostly covered Understanding Factual Knowledge (F2), while most IELTS reading items included Understanding Conceptual Knowledge (C2). It implies that in the majority of IELTS listening test items, the incoming knowledge (included in the listening and reading) should be integrated with existing schemas and cognitive frameworks. In fact, in most of IELTS listening test items, the candidates were expected to construct meaning from the information presented in the tests, whereas in IELTS reading items, the candidates were required to distinguish how various knowledge pieces and elements of knowledge are associated and incorporated in a more systematic pattern. The study also found that although both IELTS listening and reading tests mostly included lower-order thinking skills, the prevalence of lower-order thinking skills in IELTS reading tests was significantly higher than that in IELTS listening tests.

The findings of the present research regarding the representation of learning objectives and thinking skills in IELTS listening and reading tests may seem quite surprising. When it comes to possible explanations and speculations, it can be argued that IELTS is developed essentially to assess readiness to enter the academic context in English language. In fact, it does not presume candidates have already acquired the high-level academic skills that might be required for their subsequent studies (Taylor & Weir, 2012). Such high-level skills may well need to be mastered during their studies. Therefore, IELTS listening and reading tests include a higher frequency of lower-order thinking skills than higher-order thinking skills. Another reason may be that as Alderson and Lukmani (1989) indicated

 if the objective of the assessment is to measure language ability rather than cognitive skills, it would be better to design a test with lower order questions which will maximize the language content validity of the test and minimize the contamination from cognitive or higher order skills. (p. 268)

IELTS is considered as a high-stakes language test. Therefore, the representation of lower-order thinking skills in IELTS may maximize its content validity.

The results of the study accord with those of the research conducted by Baghaei et al. (2020). They also conducted a study that looked in depth at learning objectives represented in the listening and reading test items of IELTS and TOEFL. Concerning the IELTS test items, they found that Understanding Factual Knowledge and Understanding Conceptual Knowledge were the most dominant learning levels included in the listening and reading sections, respectively.

An important question to answer is why two sections of the IELTS tests are different in terms of learning objectives. One possible explanation for this finding is that IELTS reading test, due to the domination of multiple-choice items which potentially evoke a variety of cognitive processes (Owen, 2016) and impose high load on the reading skill (Field, 2009), provide the opportunity to represent cognitive processes other than just Understanding Factual Knowledge.

Thus, the differences between the two IELTS sections can be attributed to the different item formats provided in these tests. The finding of the study regarding the focus of IELTS listening tests on Understanding Factual Knowledge confirms Field’s (2009) finding that the IELTS listening test format limits cognitive processing.

 

7. Conclusion

This study probed into the difference between the listening and reading sections of IELTS with regard to representing the learning objectives of Revised Bloom’s taxonomy. The results of the study demonstrated that IELTS listening and reading tests were significantly different in terms of the representation of learning objectives.

 Both IELTS listening and reading tests covered four learning objectives: Listening (F1, F2, C2, and C4) and Reading (F1, F2, C2, and C5). IELTS listening test items focused on Factual Knowledge, while IELTS reading test items clustered on Conceptual Knowledge to the largest extent. It was also concluded that although both tests mostly included lower-order thinking skills at level 2 (Understand), IELTS reading tests included higher frequency of lower-order thinking skills compared with the IELTS listening tests.

To conclude, the present study provided further insight into high-stakes tests research and opened avenues for future research by indicating that even the two sections of the same high-stakes test may represent different learning objectives and thus the consideration of potential differences in learning objectives of different parts of IELTS test should be a priority for future IELTS research.

The study holds a number of implications for high-stakes tests research, teaching and testing. The findings that IELTS listening and reading tests are significantly different with regard to the representation of learning objectives provide important considerations for the field of language assessment.

Understanding the learning objectives frequently tested in the listening and reading tests of IELTS informs the assessment practitioners to define possible pathways for modifying and enhancing the assessment of high-stakes tests.

This study revealed that IELTS listening and reading test items focused on Factual Knowledge and Conceptual Knowledge respectively. Therefore, IELTS classes should prioritize these types of knowledge in instruction and practice.

Conducting the content analysis of the listening and reading tests of IELTS with 16 tests generated deep data, but also limited the broad application of findings across language assessment.

Informed of the limitations of the research such as lack of access to the live test versions and the limited number of the tests under investigation, future researchers are suggested to conduct the same research with the lives test versions.

Furthermore, as the main focus of the study was listening and reading sections of IELTS, the similar studies can be conducted on writing and speaking sections. In addition, the cognitive domains used by IELTS candidates while answering the questions would be a valuable feature for further research.

 

مراجع

Alderson, J. C., & Lukmani, Y. (1989). Cognition and reading: Cognitive levels as embodied in test questions. Reading in a Foreign Language, 5(2), 253–270.

Altman, D. G. (1999). Practical statistics for medical research. New York: Chapman & Hall/CRC Press.

Anderson, L.W. (Ed.), Krathwohl, D.R. (Ed.), Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., & Wittrock, M.C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of educational objectives. New York: Longman.

Aryadoust, A. (2012). Differential item functioning in while-listening performance tests: the case of international English language testing system (IELTS) listening module. International Journal of Listening, 26(1), 40–60.

Aryadoust, V. (2013). Building a validity argument for a listening test of academic proficiency. Newcastle, UK: Cambridge Scholars Publishing.

Aryadoust, V., & Goh, C. (2009). A Rasch analysis of an international English language testing system sample listening test. Paper presented at The 3rd Redesigning Pedagogy International Conference, National Institute of Education, Nanyang Technological University, Singapore.

Baghaei, P., Zohoorian, Z., & Ghahramanlou, M. (2016). Understanding the cognitive processes underlying performance in the IELTS listening comprehension test. Language and Translation Studies Quarterly, 2, 97-112.

Baghaei, S., Bagheri, M. S., & Yamini, M. (2020). Analysis of IELTS and TOEFL reading and listening tests in terms of revised bloom’s taxonomy. Cogent Education, 7(1), 1-23. doi: 10.1080/2331186X.2020.1720939

Bax, S. (2013). The cognitive processing of candidates during reading tests: Evidence from eye-tracking. Language Testing, 30(4), 441–465.

Bax, S. (2015). Using eye-tracking to research the cognitive processes of multinational readers during an IELTS reading test. IELTS Research Report Series, 2, 1-21.

Brown, J. D. (2000). Observing pragmatics: Testing and data gathering techniques. Pragmatics Matters, 1(2), 5-6.

Cambridge IELTS (2017). Cambridge IELTS 12 academic: Authentic examination papers. Cambridge: Cambridge Press and UCLES.

Cambridge IELTS (2018). Cambridge IELTS 13 academic: Authentic examination papers. Cambridge: Cambridge Press and UCLES.

Cambridge IELTS (2019). Cambridge IELTS 14 academic: Authentic Practice Tests. Cambridge: Cambridge Press and UCLES.

Cambridge IELTS (2020). Cambridge IELTS 15 academic: Authentic Practice Tests. Cambridge: Cambridge Press and UCLES.

Chan, S., Bax, S., & Weir, C. )2017(. Researching participants taking IELTS Academic Writing Task 2 (AWT2) in paper mode and in computer mode in terms of score equivalence, cognitive validity and other factors. IELTS Research Reports Online Series, 4, 1-47.

Coe, K., & Scacco, J. M. (2017). Content analysis, quantitative. In Matthes, J. (Ed.), The international encyclopedia of communication research Methods (pp. 346-356). Hoboken, NJ: Wiley-Blackwell.

Ebadi, S., & Shahbazian, F. (2015). Exploring the cognitive level of final exams in Iranian high schools: Focusing on Bloom’s Taxonomy. Journal of Applied Linguistics and Language Research, 2(4), 1-11.

Field, J. (2009). A cognitive validation of the lecture-listening component of the IELTS Listening paper. In P. Thompson (Ed.), IELTS research report (Vol. 9), (pp. 17-66). UK and Australia: British Council and IELTS Australia.

Hsieh, H., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.

IELTS (2015). IELTS guide for educational institutions, governments, professional bodies and commercial organizations. Retrieved from https://www.ielts.org/-/media/publications/guide-for-institutions/ielts-guide-for-institutions-2015-uk.ashx?la=en

Khalifa, H., & Weir, C. (2009). Examining reading. Cambridge: Cambridge University Press.

Krippendorf, K. (2004). Content analysis an introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage Publications.

Loschert, K. (2000). Raising the ante for students, teachers, and schools. Alexandria, VA: Association for Supervision and Curriculum Development.

Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1(2). Retrieved from http://217.160.35.246/fqs-texte/2-00/2- 00mayring-e.pdf.

Mizbani, M. & Chalack, A. (2017). Analyzing reading and writing activities of Iranian EFL textbook Prospect 3 based on Bloom's Revised Taxonomy. Journal of Applied Linguistics and Language Research. 4(2), 13-27.

Mizbani, M., Salehi, H., & Tabatabaei, O. (2020). Content evaluation of Iranian EFL textbook vision 1 based on Bloom’s Revised Taxonomy of cognitive domain. International Journal of Foreign Language Teaching & Research, 8(29), 11-24.

Momsen, J., Offerdahl, E., Kryjevskaia, M., Montplaisir, L., Anderson, E., & Grosz, N. (2013). Using assessments to investigate and compare the nature of learning in undergraduate science courses. CBE-Life Sciences Education, 12, 239–249.

Owen, N. (2016). An evidence-centered approach to reverse Engineering: Comparative analysis of IELTS and TOEFL iBT reading sections. (Unpublished Ph.D. thesis), University of Leicester.

Rawadieh, S. M. (1998). An analysis of the cognitive levels of questions in Jordanian secondary social studies textbooks according to Bloom taxonomy. (Unpublisheddoctoral dissertation). Ohio University, Japan.

Razmjoo, S. A., & Kazempourfard, E. (2012). On the representation of Bloom's Revised Taxonomy in Interchange course books. The Journal of Teaching Language Skills, 4(1), 171-204.

Suryaningsih, H. (2014). Students' perceptions of International English Language Testing System (IELTS) and Test of English as a Foreign Language (TOEFL) tests. (Doctoral dissertation). Indiana University of Pennsylvania, Indiana.

Taylor, L., & Weir, C. J. (2012). IELTS collected papers 2: Research in reading and listening assessment. Cambridge: Cambridge University Press.

Weir, C., Hawkey, R., Green, T., & Devi, S. (2009). The cognitive processes underlying the academic reading construct as measured by IELTS. British Council/IDP Australia IELTS Research Reports, 9(4), 157–189.