Developing and Validating a Questionnaire to Assess Strategic Competence in EFL Listening Performance: A Structural Equation Modeling Approach

Research in English Language Pedagogy

مقاله 7، دوره 7، شماره 2 - شماره پیاپی 13، مهر 2019، صفحه 336-362   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.30486/relp.2019.665886

نویسندگان

Maryam Haghighi¹؛ Mojgan Rashtchi^{* 2}؛ Parviz Birjandi¹

¹Department of Foreign Languages, Faculty of Literature, Humanities, and Social Sciences, Science and Research Branch, Islamic Azad University, Tehran, Iran

²TEFL Department, Islamic Azad University, North Tehran Branch.

چکیده

This study aimed to describe the development and validation process of a listening questionnaire designed to assess strategic competence in listening comprehension. The 34-item Cognitive and Metacognitive Listening Strategies Questionnaire (CMLSQ) was administered to a relatively large sample (N=397) of L2 learners. The results of the 370 completed questionnaires were subjected to exploratory factor analysis to identify the underlying structure of the questionnaire, and to a confirmatory factor analysis to examine the validity of the model as a good fit for the data. Based on the results of exploratory factor analysis, a three-factor model underlying metacognitive strategies and a five-factor model underlying cognitive strategies were hypothesized. These models were then evaluated through confirmatory factor analysis using AMOS 18. The results indicated that the current models and their parameters well fit the data gathered from the questionnaires.

کلیدواژه‌ها

Cognitive strategies؛ Metacognitive strategies؛ Strategic Competence؛ Validation

اصل مقاله

1. Introduction

Since the paradigm shift from behaviorist to cognitivist views in language learning during the 1970s, learner strategy use has gained increasing attention among second language researchers. Several studies empirically supported the relationships between language learners' and test takers' strategy use and language performance (e.g., O’Malley & Chamot, 1990; Oxford, 1990; Politzer & McGroarty, 1985; Phakiti, 2003, 2008; Purpura, 1997, 1999;  Rashtchi & Khani, 2010; Zhang & Zhang, 2013; Zhang, Goh, & Kunnan, 2014). Also, most language competence models considered strategic competence as one of their components (e.g., Bachman, 1990; Bachman & Palmer, 1996, 2010; Canale & Swain, 1980; Oller, 1979). Of these models, Bachman and Palmer (1996) specified a fundamental role for strategic competence as a non-linguistic factor contributing to communicative language ability. They conceived strategic competence as "a set of metacognitive strategies, which can be thought of as higher-order executive processes that provide a cognitive management function in language use, as well as in other cognitive activities" (p. 70).

Investigating the notion of strategic competence, however, some researchers (e.g., Purpura, 1997; Phakiti, 2003, 2008) found a strong relationship between metacognitive and cognitive processing, whereby the integration of metacognitive knowledge with cognitive behaviors would result in better second language test performance. Following this line of research, Bachman and Palmer (2010) updated the framework of language use and incorporated cognitive strategies into it. Bachman and Palmer’s (2010) model provides a theoretical framework within which we can investigate the critical role of cognitive and metacognitive strategies used by language learners in language performance. In general, many studies have shown that cognitive and metacognitive strategy use is closely related to reading comprehension (Brown, 1980; Carrell, 1989; Paris & Jacobs, 1984; Paris, Lipson, & Wixson, 1983; Phakiti, 2003; Sheorey & Mokhtari, 2001; Zhang, 2010). However, a question can be posed as to whether this finding applies to other language skills such as listening comprehension. Therefore, one area of strategic competence, in need of more research, deals with how the use of cognitive and metacognitive strategies influences listening performance.

Listening studies have placed increasing emphasis on the role of strategy use in listening comprehension. Many studies in this area (e.g., Bacon, 1992; Bidabadi & Yamat, 2011; Chang, 2008; Chen, Zhang, & Liu, 2014; Goh, 1998, 2002; Rubin, 1994; Teng, 1997; Vandergrift, 1999, 2003) have indicated the importance of using listening strategies in the process of listening comprehension. Research into strategic listening has initially focused on identifying and classifying strategies used by learners when involved in the listening process (e.g., Vandergrift, 1997; Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006). Follow-up studies explored the relationship between second language learners' levels of proficiency and their degree of strategy use (e.g., Goh, 2002; Liu, 2008; Vandergrift, 2003). Further research in this respect has addressed the relationship between listeners’ strategy use and their listening performance (e.g., Goh & Hu, 2013; Zeng, 2012). All in all, the effect of strategy use on listening performance is undeniable.

As discussed above, Bachman and Palmer’s (2010) model provides a theoretical framework to investigate the primary role of strategic competence including cognitive and metacognitive strategies in language performance. Accordingly, the need to develop a questionnaire to assess strategic competence (i.e., perceived use of cognitive and metacognitive strategies) in EFL learners' listening performance is felt. L2 listening researchers have developed a few instruments to elicit students’ strategy use in the process of listening comprehension. However, only a few of them investigated the questionnaires’ psychometric characteristics before relating them to second language performance. Accordingly, to make the results from such studies in the field of L2 listening strategies more reliable, a questionnaire, to which multi-level methods validly apply, needs to be developed to measure the use of L2 listening strategies. To the best of the researchers’ knowledge, only Vandergrift, Goh, Marshal, and Tafaghodtari (2006) have described the development and validation of a questionnaire designed to assess second language listeners’ metacognitive awareness while listening to oral texts. As such, it is imperative to design a questionnaire for the calibration of language learners' cognitive and metacognitive strategy use. The assumption is that such a questionnaire can contribute to Bachman and Palmer’s (2010) notion of strategic competence in listening comprehension. Also, such a questionnaire can help students become more strategic listeners and test takers.

 

2. Literature Review

The role of strategy use in listening comprehension has been a topic of discussion in the L2 listening literature. As reported by Schwartz (1998), strategic listening can be defined as "the process of being aware of listening processes, having a repertoire of listening strategies, and knowing which one works best with which listening tasks" (p. 7). Many studies (e.g., Bacon, 1992; Bidabadi & Yamat, 2011; Chang, 2008; Chen, Zhang, & Liu, 2014; Goh, 1998, 2002; Rubin, 1994; Teng, 1997; Vandergrift, 1999, 2003) have indicated the importance of using listening strategies in the process of listening comprehension. Initially, researchers based their work on general language learning strategy taxonomies (e.g., Oxford, 1990; O’Malley & Chamot, 1990). Later, strategy models were developed primarily for listening comprehension. Two such models were based on O’Malley and Chamot’s model (Vandergrift, 1997, 2003; Flowerdew & Miller, 2005). Strategies in these two models are organized under the three main types of metacognitive, cognitive, and socio-affective strategies.

The most common type of strategies used by L2 listeners is cognitive strategies that refer to the direct manipulation or transformation of listening materials. This category includes such strategies as repeating the listening task, using imagery or keywords, transferring, translating, taking notes, summarizing, and contextualization (Vandergrift, 1997). Metacognitive strategies, including planning, monitoring, and evaluating go beyond cognitive strategies and help learners regulate their listening process (Vandergrift, 1997). The third category, socio-affective strategies, refers to strategies listeners employ to collaborate with others, verify understanding, or to lower anxiety (Vandergrift, 2003). Socio-affective strategies are less often reported by second language learners but have an essential role in instructional systems designed for second language learners (Chamot & O’Malley, 1987). These strategies encompass cooperative learning, questioning for clarification, and affective control over learning experiences. (O'Mally, Chamot, & Kupper, 1989).

In a more recent work, Vandergrift et al. (2006) developed a taxonomy of listening strategies that was more comprehensive and used different labels of strategy categories: problem-solving, planning and evaluation, mental translation, person knowledge, and directed attention. According to Vandergrift et al. (2006), problem-solving includes strategies used by listeners to make inferences and monitor them during listening performance. Planning and evaluation represent the strategies listeners use to prepare themselves for the listening skill and to evaluate the results of their listening efforts. Mental translation represents strategies that listeners must learn to avoid if they are to become skilled listeners. Person knowledge includes listeners’ perceptions of the task difficulty and their self-efficacy in L2 listening. Directed attention involves strategies that listeners use to concentrate and to stay on task.

Previous questionnaires on listening strategies assessed metacognitive awareness of listening (Goh, 2002; Vandergrift, 2005; Vogely, 1995). Although they were relatively reliable based on the Cronbach’s alpha reliability index, none of these earlier self-report measures had been subjected to rigorous validation procedures. Only Vandergrift et al. (2006) have described the development and validation of a listening questionnaire designed to assess second language listeners' metacognitive awareness and perceived use of strategies while listening to oral texts. However, Vandergrift et al. (2006) searched beyond strategic knowledge and included items related to the person and task knowledge in their questionnaire.

The review of studies on listening strategies suggests that no research has been carried out to develop and validate a questionnaire for the assessment of strategic competence in listening comprehension. Hence, the present study was designed to present a standard measure to assess strategic competence comprising EFL learners' perceived use of cognitive and metacognitive strategies in listening comprehension. For this purpose, two research questions were formulated:

1. What is the factorial structure of metacognitive listening strategies?
2. What is the factorial structure of cognitive listening strategies?

 

3. Methodology

In this part, the methodological considerations of the study are presented in two separate phases; that is,instrument development and instrument validation.

 

3.1. Design and Context of the Study

This study was non-experimental. Quantitative methods were used for instrument development and instrument validation. The data were gathered from EFL learners majoring in English Translation and TEFL from Islamic Azad University, Shiraz, and Dezful Branches.

 

3.2. Phase I: Instrument Development

First, the literature on listening strategy use (Goh, 1998, 2002; Vandergrift, 1997, 2005; Vandergrift et al., 2006; Vogely, 1995) was examined to meet the standard criteria for the development of valid and reliable questionnaires (Brown, 2001; Dornyei, 2003; Gilham, 2000). Drawing on the work of these researchers, the cognitive and metacognitive listening strategies, considered useful by proficient L2 listeners, were identified and used as sources for item development. Accordingly, 52 items were selected. Based on Oxford (1990) and Vandergrift (1997), these items were assumed to tap cognitive and metacognitive strategies.

Then, following Dornyei (2003), the initial list of items was subjected to expert judgment for redundancy, content validity, clarity, and readability. This stage resulted in the elimination of some unrelated items. In cases where the judges could not agree on the acceptability of an item, it was retained for the initial field testing. The process of expert judgment reduced the questionnaire to 46 items. Each of the remaining 46 items was then examined critically for clarity and readability. Then the problematic items were reworded.

 

3.2.1. Piloting the Questionnaire

          The revised items which passed the judgment of the reviewers were piloted with a group of students for additional feedback on the clarity of the items, resulting in further fine-tuning. The participants consisted of 55 (36 female and 19 male) EFL learners studying English translation and TEFL, at different levels of language proficiency selected randomly from Islamic Azad University, Dezful Branch.

The questionnaire allowed the participants to mark their strategy use on a 6-point Likert scale: 1 (never), 2 (rarely), 3 (sometimes), 4 (often), 5 (usually), and 6 (always). Following Dornyei (2003), items were grouped in sequences logically organized by content (strategies used before, during, and after listening to an oral text). Furthermore, some items were negatively worded so that respondents would not fall into a pattern of marking only one side of the rating scale. The students were asked to mark the items perceived as problematic, confusing, or unclear regarding wording, format, and content. After administering the questionnaire, four students were asked to explain the responses they had chosen for the items so that the researchers could gather extra information concerning the items. Accordingly, the questionnaire was depleted to 34 items tapping into two types of cognitive and metacognitive strategies, each with subscales which had to be validated separately.

3.3. Phase II: Instrument Validation

3.3.1. Participants

The participants in the second phase consisted of a total of 397 respondents. Researchers would mostly recommend using sample sizes of at least 200 or 10 cases per parameters for factor analysis (Kline, 2011). Given this rule, the sample size for the present study is considered acceptable. There were 276 female and 121 male students in this study, ranging in age from 19 to 37. The participant pool was composed of both BA students (junior and senior) and MA students majoring in English Translation and TEFL from Islamic Azad University, Shiraz, and Dezful Branches. As large samples are required for SEM analyses, the researchers were forced to include all the available students in this study; therefore, the students were selected non-randomly based on convenience sampling.

 

3.3.2. Instrument

The participants filled out the 34-item Cognitive and Metacognitive Listening Strategies Questionnaire (CMLSQ) that was developed based on the supportive literature and which was revised after the pilot study. The CMLSQ (see Appendix) consists of two typesof strategies, namely Metacognitive Listening Strategies (MLS) and Cognitive Listening Strategies (CLS). Items of the questionnaire were in statement format. The researchers selected a 6-point Likert-scale ranging from "Never" to "Always" to assess the participants' perceived use of cognitive and metacognitive listening strategies.

 

3.3.3. Data Collection Procedure

A total of 397 questionnaires were administered through face-to-face contacts. All respondents completed the questionnaire in English. Following Vandergrift et al. (2006), the questionnaires were administered in different classes after the students had engaged in some listening activities so that they would have a specific task on which to base their responses. That is, the students were presented with three oral passages followed by some comprehension questions to be discussed so that the researchers could confirm their understanding of the texts. The passages of differing levels of difficulty were selected so that the students would use different types of strategies.

Instructions were reviewed and clarified before participants completed the questionnaires. It was emphasized that there were no right or wrong answers and that the researchers were only interested in an accurate appraisal of how students attempt to understand oral texts. Also, the researchers tried to encourage the respondents to comment on any item that seemed unclear to them. Out of the 397 questionnaires, 27 were discarded. Some questionnaires were not filled out completely, and some were filled haphazardly.  Thus, 370 questionnaires proved useful for data analysis.

 

3.3.4. Data Analysis Procedure

The participants' responses to the items of the questionnaire were fed into SPSS 18 and AMOS for data analysis. The reliability of the CMLSQ and its two parts (MLS and CLS) were calculated using Cronbach’s alpha. Factor analysis embracing both exploratory and confirmatory analyses was run to examine the construct validity of the instrument including the constructs of cognitive and metacognitive strategies. The results are presented and explained in the next section.

 

4. Results                                                           

The CMLSQ was distributed and responded in two phases to measure the internal consistency firstly and factorial validity secondly. In the pilot phase, the researchers administered the questionnaire to 55 participants and estimated the reliability of the CMLSQ via Cronbach's alpha. The reliability estimate of the instrument (CMLSQ) was 0.87, which is a high-reliability index. In the main study, the CMLSQ was administered to 397 participants to investigate the construct validity using factor analysis. The ultimate goal of factor analysis based on Pallant (2013), is the identification of any underlying relationship among a set of measured variables. It involves two main stages: Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). The former is used when a research study aims to develop a scale and explore the interrelationships among a set of variables. The goal of CFA, on the other hand, is to test specific hypotheses or theories concerning the structure underlying a set of variables

 

4.1. Exploratory Factor Analysis

An investigation of the factor structure was conducted through SPSS to determine whether there was empirical support for separate factors related to MLS and CLS and to identify any items that might be removed from the questionnaire. The data obtained from the 370 copies of the questionnaires were fed into SPSS version18 to examine the internal consistency. The reliability of the instrument and its two subparts were calculated using Cronbach’s alpha. The reliability index for the CMLSQ was 0.88 and for its two sub-sections (MLS and CLS) were 0.86 and 0.90, respectively, which are very high-reliability indices.

To find the correlations among the items of the questionnaire, and to label the extracted factors, the MLS and CLS data were subjected to factor analysis with Varimax rotation using a Principal Component Analysis (PCA). Before performing PCA, it was necessary to demonstrate the suitability of the data for factor analysis. Thus, the data was subjected to factor analysis with Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of sphericity to assess the factorability of the data. The KMO measure of sampling adequacy obtained for this set of items was 0.86 for MLS and 0.69 for CLS, indicating the eligibility of the data for the factor analysis (usually, if the KMO is less than 0.5, then a factor analysis is not a good idea). Another statistical measure, which helps to assess the factorability of data, is Bartlett’s test of sphericity that should be significant (p < .001). In this study, Bartlett’s test was significant (p < .001) for both MLS and CLS; therefore, factor analysis was appropriate.

The data were subjected to PCA on the 17 items of the MLS and 17 items of CLS to estimate the maximum number of factors. The results of PCA for MLS revealed the emergence of three factors with eigenvalues exceeding 1 (5.626, 1.947, 1.735, respectively) accounting for 54.75% of the total variance. Table 1 shows the internal consistency of each factor and the total reliability ofthe MLS. The scree test of eigenvalues plotted against factors was also examined. These three factors are observable in the scree-plot (Figure 1).

 

Table 1.

Reliability Statistics for Metacognitive Listening Strategies

	Cronbach's Alpha	N of Items
Factor 1	.829	7
Factor 2	.819	6
Factor 3	.723	4
Total	.868	17

 

       

Figure 1. Scree-plot of the MLS factors

 

The researchers performed Oblimin rotation to interpret the three factors. The factors went under five rotations. The minimum loading for keeping any item was set at 0.4. The results showed that seven items loaded on factor1, six items on factor 2, and four items on factor 3 (Table 2).

 The related literature was used to verify the rationality of the results and to label the factors in the following way: factor (1) planning, factor (2) evaluation, and factor (3) monitoring. Based on the related literature, these factors are in congruence with O'Mally, Chamot, and Kupper's (1989), O’Malley and Chamot’s (1990), and Wenden's (1991) classification of metacognitive strategies. They are also in partial congruence with Vandergrift's (1997) classification of metacognitive listening strategies. Table 3 presents the taxonomy of metacognitive listening strategies.

 

 

 

Table 2.

Rotated Component Matrix for Metacognitive Listening Strategies

Items
	1	2	3
1	.780
3	.707
2 12	.700 .671
7	.626
9	.623
10	.521
30		.816
32		.788
34		.757
27		.639
31		.591
33		.569
26			.771
22			.752
18			.686
28			.563

                       

 

 

 

 

 

 

 

 

Table 3.

Taxonomy of Metacognitive Listening Strategies

Processing	Subscale	N of Items	Items
Metacognitive	Planning	7	1, 2, 3, 7, 9, 10, 12
Strategies	Evaluation	6	27, 30, 31, 32, 33, 34
	Monitoring	4	18, 22, 26, 28

 

The results of PCA for CLS revealed the emergence of five factors with eigenvalues exceeding 1 (4.467, 2.429, 1.657, 1.357, 1.224, respectively) accounting for 65.49% of the total variance. Table 4 shows the internal consistency of each factor and the total reliability of the CLS. The scree test of eigenvalues plotted against factors was also examined. The scree-plot (Figure 4.2) shows the five factors.

        

Table 4.

Reliability Statistics for Cognitive Listening Strategies

	Cronbach's Alpha	N of Items
Factor 1	.800	7
Factor 2	.756	2
Factor 3	.758	4
Factor 4	.954	2
Factor 5	.708	2
Total	.901	17

 

 

 

 

Figure 2. Scree-plot of the CLS factors

 

Afterward, Oblimin rotation was performed. The factors went under nine rotations. The minimum loading for keeping any item was set at 0.4. The results of exploratory factor analysis showed that seven items loaded on factor 1, two items on factor 2, four items on factor 3, two items on factor 4, and two items on factor 5 (Table 5).

Table 5.

Rotated Component Matrix for Cognitive Strategies

Items	Components
	1	2	3	4	5
16	.755
19	.746
17	.705
20	.535
23	.529	.511
21	.526	.414	.478
14	.491
29		.783
11		.519	.780
6			.728
5			.587
4			.576
8					.433
25				.950
24				.936
13					.780
15					.686

 

It should be mentioned that items 14, 21, and 23 in the first factor also had loading higher than 0.40 on factors 2 and 3. Additionally, item 8 in the third factor had a high loading on factor five. However, these additional loadings were ignored because these items had a higher level of loading in the first and third factors, respectively. Therefore, all 17 items of CLS were kept in the final version and survived this analysis. Based on the loading of the items and their underlying theme, the components were labeled in the following way: factor (1) inferencing, factor (2) summarization, factor (3) prediction, factor (4) note-taking, and factor (5) elaboration. The related literature suggests that these factors are in partial congruence with Vandergrift's (1997) and Goh's (2002) classification of cognitive listening strategies. Table 6 presents the taxonomy of cognitive listening strategies.

Table 6.

Taxonomy of Cognitive Listening Strategies

Processing	Subscale	N of Items	Items
Cognitive	Inferencing	7	14, 16, 17, 19, 20, 21, 23
Strategies	Summarization	2	11, 29
	Prediction	4	4, 5, 6, 8
	Note-taking	2	24, 25
	Elaboration	2	13, 15

 

4.2. Confirmatory Factor Analysis

Based on the results of exploratory factor analysis, the researchers hypothesized a three-factor model for MLS and a five-factor model for CLS, and these factors were assumed to covary with each other. Using AMOS 18, CFA was conducted to verify and extend the factor models of MLS and CLS and to examine how well the factor models and the empirical data match one another.

Because no single universally accepted criterion exists to judge model fit (Heubeck & Neil, 2000), several widely accepted goodness-of-fit indices were computed for both MLS (Table 7) and CLS (Table 10). The first index is the chi-square. However, it cannot be used in the evaluation of model fit because chi-square is known to be strongly dependent on the sample size (Cliff, 1987; Floyd & Widaman, 1995); thus, it is divided by the degree of freedom called normed chi-square (/df).  The value of normed chi-square less than 3 is considered acceptable.

Besides the normed chi-square, the other criteria usually reported in CFA-AMOS studies as indicators of a model fit include CFI (Comparative Fit Index), GFI (Goodness-of-Fit Index), TLI (Tucker & Lewis Index), PNFI (Parsimonious Normed Fit Index), and the RMSEA (Root Mean Square Error of Approximation). Models with a GFI, CFI, and a TLI greater than 0.90 are considered acceptable; however, RMSEA is expected to be less than 0.08.

Regarding MLS, it appeared that CFI and GFI, and TLI are greater than 0.90. Bearing in mind the point that, the closer the value to 1, the better fitness, this scale shows a good fit. Inspecting the normed chi-square and other goodness-of-fit indices showed a significantly fit model with /df =2.692, CFI=0.997, GFI=0.996, and RMEAS=0.065 (Table 7). The internal consistency of the total scale was found to be 0.86. Overall, the MLS shows a good and high model fit, confirming the three-factor structures behind the instrument.

 

Table 7.

Goodness-of-Fit Indices for the MLS Model

Model		df	df/	GFI	CFI	TLI	RMSEA	PNFI
Three-Factor	298.844	111	2.692	0.966	0.997	0.974	0.792	0.065

 

Figure 3 illustrates the hypothesized SEM model for MLS. This model shows the relations between the factors of the scale.

 

 

Figure 3. SEM model for the metacognitive listening strategies

 

Regarding CLS, it appeared that CFI, GFI, and TLI are larger than 0.90. As mentioned before, the closer the value to 1, the better fitness; therefore, this scale shows a good fit. The normed chi-square and other goodness-of-fit indices showed a significantly fit model with /df =2.205, CFI=0.947, GFI=0.962, and RMEAS=0.073. The internal consistency of the total scale was found to be 0.90. Overall, the CLS shows a good and high model fit, confirming the five-factor structures behind the instrument (Table 8).

 

Table 8.

Goodness-of-Fit Indices for the CLS Model

Model		df	df/	GFI	CFI	TLI	RMSEA	PNFI
Five-Factor	222.761	101	2.205	0.962	0.947	0.908	0.818	0.073

 

Figure 4 illustrates the hypothesized SEM model for the CLS. This model shows the relations between the factors of the scale.

 

 

 

 

Figure 4. SEM model for the cognitive listening strategies

 

5. Discussion

This study described the development and validation of a listening questionnaire to assess L2 learners’ perceived use of cognitive and metacognitive strategies while listening to oral texts. This section discusses the results regarding the two research questions of the study.

Exploratory and confirmatory factor analysis was run to answer the first research question, "What is the factorial structure of metacognitive listening strategies?" The results demonstrated a three-factor model underlying MLS (planning, evaluation, and monitoring). Having obtained the strong Amos confirmation, the researchers attempted to elaborate on the components of metacognitive listening strategies using both the supportive literature and the findings of the current study. The followings are the three overarching components of the MLS:

1. Planning: it refers to "developing an awareness of what needs to be done to accomplish a listening task, developing an appropriate action plan to overcome difficulties that may interfere with successful completion of the task" (Vandergrift, 1997, p. 392).  The planning phase prepares listeners to be proactive in their listening efforts. Proactive listeners decide what to listen for and establish the necessary conditions for successful listening, which can enable them to pay close attention to meaning while listening. During the critical planning phase, listeners prepare themselves for what they will hear and what they are expected to do, instead of barreling into the activity without thinking (Vandergrift & Goh, 2012). The seven items in this factor include strategies such as having a plan for L2 listening, having a goal in mind while listening, prepare the conditions for listening by clearing their minds of distractions and focusing their attention, and not giving up when experiencing difficulties understanding.
2. Evaluation: it represents a group of strategies used by listeners to check the outcomes of one’s listening comprehension against an internal measure of completeness and accuracy (Vandergrift, 1997, p. 392). Listeners need to evaluate the effectiveness of the approach adopted and/or decisions made during the listening process after completion of the activity (Vandergrift & Goh, 2012, p. 107). The six items in this factor comprise strategies such as periodically examining one’s satisfaction with the ongoing interpretation while listening, reflecting on difficulties encountered, what went wrong, and how to overcome them, and evaluating the level of comprehension after the task. These strategies represent the purposeful nature of the comprehension process (Richards, 1990) and the online appraisal of whether comprehension goals were being realized.
   1. Monitoring: it includes "checking, verifying, or correcting one’s comprehension or performance in the course of a listening task" (Vandergrift, 1997, p. 392). While listening to the text, listeners monitor their comprehension in light of their predictions and make adjustments, as necessary (Vandergrift & Goh, 2012, p. 107). The four items in this factor include strategies such as evaluating what they understand, checking for consistency with their predictions, verifying predictions and accept the fact that they do not need to understand every word, and identifying problematic areas, and limitations of the task.

The second research question of the study “What is the factorial structure of cognitive listening strategies?”was also answered by employingexploratory and confirmatory factor analyses. The analyses demonstrated a five-factor model underlying the CLS (inferencing, summarization, prediction, note-taking, and elaboration). Regarding the strong AMOS confirmation, the researchers attempted to discuss the components of cognitive listening strategies using both the supportive literature and the outcomes of the study. The followings are the five critical components of the cognitive listening strategies:

1. Inferencing: It refers to using the information within the text or conversational context to guess the meanings of unfamiliar language items associated with a listening task, to predict outcomes, or to fill in missing information (Vandergrift, 1997). The seven items representing this factor include strategies such as using known words to deduce the meaning of unknown words, using the general idea of a text and the context to deduce unknown words and using the tone of voice and the relationship between the speakers to understand the text.
2. Summarization: Making a mental or written summary of language and information presented in a listening task. The two items representing this factor involves strategies such as organizing the points heard of catching the overall meaning, and making a mental summary of what was heard (Vandergrift, 1997).
3. Prediction: Anticipating the content of the listening text before or during listening (Goh, 2002). The four items representing this factor include strategies such as predicting the possible content according to the title, the instruction, and the questions before and during the listening task.
4. Note-taking: Writing down key words and concepts in abbreviated verbal, graphics, or numerical form to assist the performance of a listening task (Vandergrift, 1997). The two items related to this factor involves taking down the main points and referring back to these notes during listening.
5. Elaboration: Using prior knowledge from outside the text or conversational context and relating it to knowledge gained from the text or conversation to predict outcomes (Vandergrift, 1997). The two items in this factor consist of the use of experience and knowledge about the topic to understand and interpret the text.

The obtained models are very advantageous concerning the results of the validation phase. Based on the results of the exploratory stage, the researchers of the present study now conceive metacognitive listening strategies as having at least three and cognitive strategies and having five distinct, though related, factors which underlie the constructs of metacognitive and cognitive strategies, respectively. Additionally, based on the findings of the confirmatory stage, the current models and their parameters well fit the data gathered from the questionnaires. In brief, a model which exceeds minimum acceptance cut-off values for the indices can be regarded as a valid tool. The proposed models, therefore, can provide the required underpinning to measure the perceived use of cognitive and metacognitive listening strategies.

 

6. Conclusion

Regarding the importance of cognitive and metacognitive listening strategies and the absence of any valid scale in this field, the current study attempted to develop and validate a self-report questionnaire to gauge listeners' perceived use of cognitive and metacognitive strategies while listening to oral texts in L2. The results demonstrated that the CMLSQ appears to have acceptable psychometric properties as a measure of cognitive and metacognitive listening strategies.

The findings of this study embody two substantial implications: implications for language teaching pedagogy and implications for research purposes. First, second language learners can use the CMLSQ to assess their level of cognitive and metacognitive listening strategies knowledge and hence try to develop their strategy use so that, ultimately, they become skilled and self-regulated listeners. Second, instructors can use the CMLSQ as a diagnostic tool to determine the learners' weaknesses concerning strategy use and as a result, adjust the instruction to place greater emphasis on underused strategies. Finally, instructors and researchers can use the CMLSQ as a research tool. Researchers can use this instrument to assess students' knowledge and perceived use of cognitive and metacognitive listening strategies and to investigate the relationship between strategy use and listening test performance. Furthermore, this instrument can be used to determine the usefulness of listening strategy instruction and to monitor learners’ growing awareness of the processes underlying successful L2 listening.

 

مراجع

Bachman, L. F.(1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.

Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford: Oxford University Press.

Bacon, S. M. (1992). The relationship between gender, comprehension, processing strategies, and cognitive and affective response in foreign language listening. The Modern Language Journal, 76(2), 160-178. doi:10.1111/j.1540-4781.1992.tb01096.x

Bidabadi, F., & Yamat, H. (2011). The relationship between listening strategies used by Iranian EFL freshman university students and their listening proficiency levels. English Language Teaching, 4(1), 26-32. doi:10.5539/elt.v4n1p26

Brown, A. L. (1980). Metacognitive development and reading. In R. J. Spiro, B. C. Bruce, & W.F. Brewer (Eds.), Theoretical issues in reading comprehension: Perspectives from psychology, linguistics, artificial intelligence and education (pp. 453-482). Hillsdale: Erlbaum.

Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1-47. doi:10.1093/applin/i.1.1

Carrell, P. L. (1989). Metacognitive awareness and second language reading. Modern Language Journal, 73(2), 121-131. doi:10.2307/326568

Chamot, A. U. (2005). Language learning strategy instruction: Current issues and research. Annual Review of Applied Linguistics, 25, 112-130. doi:10.1017/s0267190505000061

Chamot, A. U., & O’Malley, J. M. (1987). The cognitive academic language learning approach: A bridge to the mainstream. TESOL Quarterly, 21 (2), 227-249. .doi:10.2307/3586733

Chang, C. A. (2008). Sources of listening anxiety in learning English as a foreign language. Perceptual and Motor Skills, 106(1), 21-34. doi:10.2466/pms.106.1.21-34

Chen, L., Zhang, R., & Liu, C. (2014). Listening strategy use and influential factors in Web-based computer assisted language learning. Journal of Computer-Assisted Learning, 30, 207-219. doi: 10.1111/jcal.12041

Cliff, N. (1987). Analyzing multivariate data. New York: Harcourt College Pub.

Dornyei, Z. (2003). Questionnaires in second language research: Construction, administration and processing. Mahwah: Erlbaum.

Flowerdew, J. H., & Miller, L. (2005). Second language listening: Theory and practice. New York: Cambridge University Press.

Floyd, F. J. & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286-299. doi:10.1037//1040-3590.7.3.286      

Gillham, B. (2000). Developing a questionnaire. London: Continuum.

Goh, C. (2002). Exploring listening comprehension tactics and their interaction patterns.  System, 30(2), 185-206. doi:10.1016/s0346-251x(02)00004-0

Goh, C. (1998). How learners with different listening abilities use comprehension strategies and tactics. Language Teaching Research, 2(2), 124-147. doi:10.1177/136216889800200203

Kline, R. B. (2011). Principles and practices of structural equation modeling (2^nd ed.). New York: Guilford

Liu, H. (2008). A study of the interrelationship between listening strategy use, listening

proficiency, and learning style. ARECLS, 8, 84-104. Retrieved from    https://research.ncl.ac.uk/media/sites/researchwebsites/arecls/liu_vol5.pdf

O’Malley, M. J., & Chamot, A. U. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press.

Oller, J. W. (1979). Language tests at school. London: Longman.

Oxford, R. (1990). Language learning strategies: What every teacher should know. Boston:  Heinle & Heinle.

Pallant, J. (2013). SPSS survival manual: a step by step guide to data analysis using IBM SPSS (5^th ed). Maidenhead: Open University Press.

Paris, S. G., & Jacobs, J. (1984). The benefits of informed instruction for children’s reading awareness and comprehension skills. Child Development, 55(6), 2083-2093. doi:10.1111/j.1467-8624.1984.tb03904.x

Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8(3), 293-316. doi:10.1016/0361476x(83)90018-8

Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading comprehension test performance. Language Testing, 20(1), 26-56. doi:10.1191/0265532203lt243oa

Phakiti, A. (2006). Modeling cognitive and metacognitive strategies and their relationships to EFL reading test performance. Melbourne Papers in Language Testing, 1, 53-95.

Phakiti, A. (2008). Construct validation of Bachman and Palmer’s (1996) strategic competence model over time in EFL reading tests. Language Testing, 25(2), 237-272. doi:10.1177/0265532207086783

Politzer, R. L., & McGroarty, M. (1985). An exploratory study of learning behaviors and their relationship to gains in linguistic and communicative competence. TESOL Quarterly, 19(1), 103-123. doi:10.2307/3586774

Purpura, J. E. (1997). An analysis of the relationships between test takers’ cognitive and metacognitive strategy use and second language test performance. Language Learning, 47(2), 289-294. doi:10.1111/0023-8333.91997009

Purpura, J. E.(1999). Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge: CambridgeUniversity Press.

Rashtchi, M., & Khani, P. (2010). Improving Iranian EFL learners’ oral proficiency through metacognitive strategies instruction. Journal of English Language Studies, 1(4), 137-156.

Richards, J. (1990). The language teaching matrix. Cambridge: Cambridge University Press.

Rubin, J. (1994). A review of second language listening comprehension research. The Modern Language Journal, 78(2), 199-217. doi:10.2307/329010

Sheoery, R., & Mokhtari, K. (2001). Differences in the metacognitive awareness of reading strategies among native and non-native readers. System, 29(4). 431-449. doi:10.1016/s0346-251x(01)00039-2

Teng, H. (1997). An investigation of EFL listening strategies by Taiwanese college students.

 Proceedings of the Sixth International Symposium on English Teaching. Taipei: The Crane Publishing Co., Ltd.

Vandergrift, L. (1997). The strategies of second language (French) listeners: A descriptive study. Foreign Language Annals, 30(3), 387-409. doi:10.1111/j.1944-9720.1997.tb02362.x

Vandergrift, L. (2003). Orchestrating strategy use: Toward a model of the skilled second language listener. Language Learning, 53(3), 463-496. doi:10.1111/1467-9922.00232

Vandergrift, L., & Goh, C. (2012). Teaching and learning second language listening: Metacognition in action. New York: Routledge.

Vandergrift, L., Goh, C., Mareschal, C. J., & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56(3), 431-462. doi:10.1111/j.1467-9922.2006.00373.x

Vogely, A. (1995). Perceived strategy use during performance on three authentic listening comprehension tasks. Modern Language Journal, 79(1), 41-56. doi:10.1111/j.15404781.1995.tb05414.x

Wenden, A.L. (1991). Learner strategies for learner autonomy. Englewood Cliffs: Prentice-Hall.

Wenden, A. L. (1998). Metacognitive knowledge and language learning. Applied Linguistics, 19(4), 515-537. doi:10.1093/applin/19.4.515

Zeng, Y. (2012). Metacognition and self-regulated learning (SRL) for Chinese EFL listening development. Unpublished doctoral dissertation, Nanyang Technological University, Singapore.

Zhang, L. J. (2010). A dynamic metacognitive systems account of Chinese university

students’ knowledge about EFL reading. TESOL Quarterly, 44, 320–353.

Zhang, L., Goh, C. M. C., & Kunnan, A. J. (2014). Analysis of test takers’ metacognitive and cognitive strategy use and EFL reading test performance: A multi-sample SEM approach. Language Assessment Quarterly, 11(1), 76-102. doi:10.1080/15434303.2013.853770

Zhang, L., & Zhang, L. J. (2013). Relationships between Chinese college test takers' strategy use and EFL reading test performance: A structural equation modeling approach. RELC Jornal, 44(1), 35-57. doi:10.1177/0033688212463272