The Effect of Dynamic Assessment on the EFL Learners’ Complaint Speech Act Developmental Trajectories: The Application of Rasch Model

Research in English Language Pedagogy

دوره 10، شماره 1 - شماره پیاپی 18، خرداد 2022، صفحه 76-95   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.30486/relp.2021.1931983.1280

نویسندگان

Zahra Rezaee¹؛ Parisa Daftarifard^{* 1}؛ Morvarid Lavasani²

¹English Language Research Center, South Tehran Branch, Islamic Azad University, Tehran, Iran

²Department of English Language, Yadegar-e- Imam Khomeini (RAH) Shahre Rey Branch, Islamic Azad University, Tehran, Iran

چکیده

Dynamic assessment (DA) has been recently used to dominate the realm of pragmatic research. This study investigated the effect of dynamic assessment on EFL learners’ complaint speech act performance, as the most face-threatening one.  Thirty-three participants were selected using the convenience method of sampling. They formed two groups of NDA (non-dynamic assessment) (n = 15) and DA (dynamic assessment) (n =18) and the participants were all lower-intermediate on the Oxford Placement Test. The learners in both groups took a discourse completion test (DCT) as the pretest and posttest. At intervals, the participants in the experimental group took an intervention within the DCT test over five sessions. The gradually constructed hints which were constructed using both literature and experts’ view were given upon the participants’ response. The results of the study using the Rasch model and Classical True Score Theory revealed that DA groups significantly outperformed NDA groups. Furthermore, most of the participants in DA group made considerable changes from the pretest to the posttest. Rasch model reveals that the learners had idiosyncratic zigzagged changes as was predicted by Vygotsky. The findings of the study can shed light on language teaching and materials development for the complaint speech act.

کلیدواژه‌ها

Complaint speech act؛ Developmental Trajectories؛ Dynamic assessment؛ Pragmatic؛ Rasch Model

اصل مقاله

1. Introduction

Dynamic Assessment (DA) has won the scholars’ attention in the realm of language in general and most recently in language pragmatics in particular (Alavi et al., 2020; Moradian et al., 2019). The importance of such dialectical interaction between assessment and teaching in this area has been hinged upon the notion of Zone of proximal development (ZPD). Vygotsky (1980) claims that learners’ potential ability occurs as the result of learners’ interaction with objects, self, and others. Such desired interaction is believed to emerge when the intervention is systematically embedded within the test in the form of DA (Nassaji et al., 2021). This feature has recently become prominent in the realm of pragmatic studies.

The aim of almost all research on DA pragmatics is twofold, firstly to promote pragmatic development more efficiently and secondly to assess the knowledge more precisely and validly. While the latter is mostly stated in the literature, the former is legitimately triggered for further inquiry. Different scholars (Tajeddin & Tayebipour, 2012) believe that L2 pragmatic competence cannot develop without instruction. Speech act acquisition is not an easy task for learners as it is highly cultural and context-dependent in the way that an appropriate way of stating or acting something in one language might be inappropriate in another language. In other words, second language learners need to acquire pragmatic competence by gaining cultural understanding and communication skills. Zhang (2001)believes that teaching L2 pragmatic norms in the classroom (either directly or indirectly) would help learners to become competent users of the target language.

However, there is less consensus on what type of implicit or explicit instruction might be more efficient. Takahashi (2010) contends that explicit and implicit instructions can be both effective, but explicit instruction leads to much better performance in learning the targeted features. Still, others believe that through DA calibrated hints, not only learners become autonomous, but also they acquire knowledge more efficiently. It seems that DA is the site where explicit-implicit instruction merges “not in terms of polarity but based on modality [thereby] making the instruction more learner-friendly and ZPD-sensitive” (Tajeddin & Tayebipour, 2012, p. 88).

One of the prevalently used and face-threatening speech act is a complaint. Not much research has been done to investigate the effect of DA on the acquisition of pragmatic (Moradian et al., 2019) especially complaint speech acts. According to Pakzadian and Tajeddin (2014), although dynamic assessment has been a promising strand of research, a few studies have ever been conducted on complaint speech act Interlanguage development from the perspective of DA except for few. One of the research, for instance, is done by Alavi et al. (2020) who have applied computerized dynamic assessment to investigate the EFL learners’ complaint speech act acquisition. Accordingly, the paired sample t-test revealed the EFL learners’ developments throughout the study. What has been done in their research was to use static prompts that were given to the test takers upon their failures. Therefore, the interventions were not idiosyncratic. Elsewhere, Tajeddin and Tayebipour (2012) using the sandwich model teaching the speech acts through conversation investigated the effects of dynamic self-assessment on the acquisition of several speech acts including complaint; they could not find the overall improvement in the experimental group.

Although a few studies have been conducted on the complaint speech act, no research has been done to investigate the Microgenesis of development of EFL intermediate language learners’ complaint speech act through the DA process. This is important for two reasons. The first reason is that as was mentioned before Complaint speech act is one of the most face-threatening ones which requires systematic ways of assessment and intervention. And the second reason is that it is claimed that ZPD based intervention can be more helpful than any other type of instruction (Tajeddin & Tayebipour, 2012). To test the latter, this study focused on pre-planned but idiosyncratic hints to engage the learners with pragmatic production problems during the process of assessment. Particularly, it aimed at investigating learners’ feasible development and gaining higher levels of cognitive growth by appropriate forms of mediation. Furthermore, following Daftarifard (2016), it applies the Rasch model to investigate the microgenesis of development more closely. According to Sijtsma (1993), for instance,  Classical True Score Theory (CTS) cannot account for dynamicity in DA studies. He suggested the use of Item Response Theory (IRT) to be used instead.

2. Literature Review

2.1 Complaint Speech Act

A complaint speech act is one of the most face-threatening speech acts which is frequently used. Olshtain and Weinbach (1993) articulate that in the action of complaining, the speaker (S) expresses her dissatisfaction about a socially unacceptable act (SUA) for which the hearer (H) is responsible. Schaefer and James (1982), using oral role-play identified nine components for the complaint speech act as “(1) Opener, (2) Orientation, (3) Act Statement, (4) Justification of Speaker, (5) Justification of Addressee, (6) Remedy, (7) Threat, (8) Closing, and (9) Valuation” (p. 44). Elsewhere, Piotrowska (1987) using the same data collection procedure with undergraduates EFL learners at the University of Hong Kong developed eight more components as societal justification, request for an explanation, blame, resignation, conciliation, persuasion, indirect disagreement, and request for agreement. Furthermore, Olshtain and Weinbach (1987) studied the complaint speech act production by native and non-native speakers of Hebrew and developed five strategies for the complaint speech act severity: (1) below the level of reproach, “No harm done, let’s meet some other time”, (2) disapproval, “It’s a shame that we have to work faster now”, (3) direct complaint, “You are always late and now we have less time to do this job”, (4) accusation and warning, “Next time do not expect me to sit here waiting for you”, and (5) threat, “If we do not finish the job today, I’ll have to discuss it with the boss” (p. 202). The results of the study revealed that both native and non-native speakers of Hebrew had a greater tendency to use disapproval, complaint, and accusation strategies.

DeCapua (as cited in Arent, 1996) also compared the production of native speakers of German and American English as well as EFL German speakers’ complaint speech acts using ‘an open-ended Discourse Completion Test’ (DCT). He identified five components to classify the complaint speech act namely as “(1) statement of problem, (2) request for repair, (3) justification, and (4) criticism” (p.127). He concluded that Germans were more indirect than Americans while speaking German and English. Elsewhere, Trosborg (1995) comparing the complaints produced by Danish learners of English with those of native speakers identified four categories: “(1) No Explicit Reproach, (2) Expression of Annoyance or Disapproval, (3) Accusation, and (4) Blame” (p. 315). Trosborg (1995) further classifies these into eight strategies: “(1) Hints, (2) Annoyance, (3) Consequences, (4) Indirect Accusation, (5) Direct Accusation, (6) Modified Blame, (7) Explicit Condemnation of the Accused’s Action, and (8) Explicit Condemnation of the Accused as a Person” (p. 315-320). Trosborg (1995) also developed some mitigating strategies to reduce the impact of the complaint speech act imposition. Mitigating strategies are of two major types: (a) internal modifiers and (b) external modifiers or supportive moves. The latter refers to “(1) Preparators, (2) Disarmers, (3) Providing Evidence, and (4) Substantiation” (p. 330-332). And finally, Murphy and Neu (1996)  investigated the American native speakers’ and Korean English learners’ production of complaint speech act. They found four semantic formulas in the speech act as “(1) an explanation of the purpose, (2) a complaint, (3) a justification, and (4) a candidate solution: request” (p. 200). Accordingly, although American native speakers and Korean English learners produced a similar explanation of the purpose, justification, and candidate solution components, they were different in the production of the second component (the complaint). The non-natives produced more criticism than complaints, which was socially unacceptable to the English native speakers.

Learning pragmatics is important to avoid miscommunication or any imposition caused by cultural differences. The concept of ‘politeness’ is directly linked to the speech act theory. According to Lakoff (1973), politeness has three main universal rules, namely “do not impose, give options, and be friendly” (p. 298). Brown and Levinson (1987) state that all members of a society tend to maintain a public self-image in communication with others which is called ‘face’ (either positive face or negative face). According to politeness theory, a positive face shows the desire to have one’s self-image approved by others, while a negative face is a part of the personality that requires not to be imposed upon. This public self-image is very fragile, and any threat to this image is considered to be “face-threatening and not acceptable” (Cutting & Fordyce, 2020, p. 43).  In the complaint speech act, the imposition is highly dependent on distance and power. According to Shahrokhi and Jan (2012), “the social distance between the interlocutors is an indication of how well the speaker and the hearer know one another. Social distance has a binary value of (+SD) where the interlocutors do not know one another well, and (-SD) where the interlocutors know one another well” (p.693). Furthermore, “the social power is the relative social dominance of one of the interlocutors on the other one; social dominance has a ternary value, namely (S>H) where the speaker dominates the hearer, (S=H) where the speaker and the hearer are equal, and (S<H) where the speaker is dominated by the hearer” (Shahrokhi & Jan, 2012, p. 693).

2.2  . Dynamic Assessment in Pragmatic Instruction

Nowadays, it is believed that interaction within the test (dynamic assessment) can give a better picture of the learners’ real competence (Lantolf & Poehner, 2008). Dynamic assessment (DA) is drawn from Vygotsky’s (1978) ‘Sociocultural Theory of Mind’ (SCT), a mixture of dialectical integration of instruction and assessment in a dynamic manner, which needs being sensitive to learners’ ‘Zone of Proximal Development’ (ZPD) during the process of assessment (Daftarifard, 2016) when instruction is embedded into assessment dialectically (Ableeva, 2010; Lantolf, 2009). For instance, Lantolf (2009) emphasizes that successful education includes the dialectical integration of instruction and assessment in a dynamic manner and must be sensitive to learners’ zone of proximal development. It is believed that these interactions are internalized and provide the basis for cognitive development (Lantolf & Poehner, 2008).

• Does the dynamic assessment have any significant effect on EFL intermediate learners’ complaint speech act production?
• To what extent, do dynamic interventions create qualitative differences in the EFL intermediate learners’ complaint speech act production?
• To what extent, is the EFL intermediate learners’ developmental trajectory steady or zigzagged?

 

3. Methodology

3.1. Design and Context of the Study

The design of this study is a mixed-method quasi-experimental developmental design (Daftarifard, 2016). It is mixed-method because, in the process of instrumentation, the interview was conducted thereby constructing the intervention within the DCT was performed. Furthermore, the study was quasi-experimental since manipulation was applied to see the effect of DA complaint DCT on the EFL learners’ pragmatic competence and relevant ZPD. And finally, the present research is developmental as in DA studies, idiosyncratic development is the main focus of attention.

 

3.2. Participants

Two groups of participants attended this study. The understudy group included 33 EFL Iranian learners who were selected from a pool of 77 participants in Islamic Azad University, South Tehran Branch in Tehran, Iran. All of them majored in Teaching English as a Foreign Language (TEFL), and their age ranged between 18 and 40 years old. The participants were all lower intermediate whose scores on the OPT were between 20 and 30 out of 40. The method of sampling was convenience method and the participants were all volunteers. They formed two groups of non-dynamic (NDA) (n = 15) and dynamic (DA) (n =18) assessment groups. The NDA participants enrolled in letter writing class, whereas the participants in the DA group had not taken letter writing course then. NDA group aimed to control the practice effect. Therefore, no intervention was taught to the NDA group although they were learning about different letter writing purposes through their letter writing course. The unequal sample size was due to the nature of convenience sampling and the fact that the present participants formed an intact group. The sample size was suitable for Rasch Analysis (Linacre, 2021, personal communication). Rasch person reliability index is 0.87 logit and Cronbach Alpha equals 0.85 which is acceptable. The second group included two expert raters and six novice language learners. The novice learners’ opinions were used to measure the response validity of the test. In other words, to ensure the operativeness of the responses were once checked with similar learners. The sampling size was determined based on data saturation. Two other expert raters rated the degree of explicitness and implicitness of the interventions.

Table 1.

Summary of 33 Measured Person

          TOTAL                         MODEL         INFIT        OUTFIT   

          SCORE     COUNT     MEASURE   ERROR      MNSQ   ZSTD   MNSQ   ZSTD

 MEAN      45.4      12.0        1.45     .44       .99     .0   1.00     .0

 S.D.       7.0        .0        1.37     .15       .51    1.0    .47     .9

 MAX.      59.0      12.0        5.36    1.04      2.99    2.6   2.74    2.5

 MIN.      33.0      12.0        -.28     .27       .32   -1.9    .36   -1.6

 REAL RMSE    .49 TRUE SD    1.28  SEPARATION  2.63  PERSON RELIABILITY  .87

 MODEL RMSE    .46 TRUE SD    1.29  SEPARATION  2.80  PERSON RELIABILITY  .89

 S.E. OF PERSON MEAN = .24                                                   

PERSON RAW SCORE-TO-MEASURE CORRELATION = .96

CRONBACH ALPHA (KR-20) PERSON RAW SCORE "TEST" RELIABILITY = .85

 

The second group included two experts as the raters for constructing the dynamic prompt tasks. All the raters held Ph.D. in Teaching English as a Foreign Language (TEFL), and they had sufficient competence in English sociopragmatics and pragmalinguistics to construct the dynamic prompt tasks. 

 

3.2. Instruments

Three instruments were used in this study. The first instrument was the ‘Oxford Placement Test’ (OPT) (Wistner et al., 2009), which was used to measure the participants’ language proficiency levels. To make our sampling fairly homogeneous in terms of their proficiency levels, the first 40 items on the OPT were administered. The OPT consisted of vocabulary and grammar items, which the participants answered them in 30 minutes. Based on the results of the OPT, Cronbach's Alpha was 0.74 and the mean was 25.97 (SD=4.997). The second instrument was two ‘Discourse Completion Test’ (DCT) on complaint speech act. The first DCT encompassed 6 situations of complaint speech act which were selected from Shahrokhi and Eshraghi (2016) and Zhang (2001) to which all learners were asked to write their responses. The second DCT also contained 6 other situations which were taken from the same source. The responses on both DCTs were rated by two raters. Kappa agreement was estimated as 90% which showed that the raters agreed on the scoring criteria. The ordinal raw scores given by the raters were then changed into a scaled score using the Rasch model for both pretest and posttest. According to Adams et al.  (2012), raw scores are ordinal in nature and should change into a scaled score to get a more valid result. Moreover, the nature of ZPD is dynamic thereby requires a dynamic measure. CTS is static in nature and cannot estimate idiosyncratic Z score for each individual; the attribute is observed when estimating scaled score. The test specifications are presented in Tables 2 and 3 below.

Table 2.

Item Statistics: Measure Order

 ENTRY   TOTAL  TOTAL           MODEL    INFIT     OUTFIT   PT-MEASURE  EXACT MATCH      

 NUMBER   SCORE  COUNT  MEASURE  S.E. MNSQ  ZSTD MNSQ  ZSTD CORR.  EXP.  OBS%  EXP% ITEM

 

     5    108     33     .83     .19 1.24    .9 1.12    .5   .65   .64  57.6  47.9  QPRE_5

     6    112     33     .68     .20 1.07    .3  .97    .0   .66   .63  54.5  48.3  QPRE_6

     4    114     33     .60     .20  .75   -.8 1.03    .2   .60   .62  51.5  49.5  QPRE_4

     1    119     33     .38     .22  .71   -.9  .66  -1.2   .68   .61  57.6  51.6  QPRE_1

     2    123     33     .18     .23  .92   -.1  .98    .0   .65   .60  45.5  54.1  QPRE_2

     7    127     33    -.05     .25 1.38   1.1 1.11    .5   .52   .59  54.5  55.2  QPOS_1

     3    128     33    -.11     .25  .99    .1  .89   -.3   .61   .59  63.6  55.2  QPRE_3

    10    130     33    -.24     .26  .80   -.5  .77   -.7   .67   .59  63.6  56.3  QPOS_4

    12    130     33    -.24     .26  .80   -.5  .75   -.8   .60   .59  60.6  56.3  QPOS_6

     8    132     33    -.38     .27 1.07    .3 1.03    .2   .49   .58  51.5  57.0  QPOS_2

    11    135     33    -.61     .28 1.01    .1  .94   -.1   .64   .57  51.5  59.4  QPOS_5

     9    140     33   -1.03     .30 1.39   1.3 1.71   1.9   .37   .55  54.5  63.7  QPOS_3

------------------------------------+----------+----------+-----------+-----------+-------

 MEAN   124.8   33.0     .00     .24 1.01    .1 1.00    .0              55.6  54.6       

 S.D.     9.4     .0     .53     .03  .22    .7 .25    .8               5.2   4.5      

 

Table 3.

Summary of Measured Item

          TOTAL                         MODEL         INFIT        OUTFIT   

         SCORE     COUNT     MEASURE   ERROR      MNSQ   ZSTD   MNSQ   ZSTD

MEAN     124.8      33.0         .00     .24      1.01     .1   1.00     .0

S.D.       9.4        .0         .53     .03       .22     .7    .25     .8

MAX.     140.0      33.0         .83     .30      1.39    1.3   1.71    1.9

MIN.     108.0      33.0       -1.03     .19       .71    -.9    .66   -1.2

REAL RMSE    .26 TRUE SD     .47   SEPARATION     1.81  ITEM   RELIABILITY  .77

MODEL RMSE    .24 TRUE SD     .47   SEPARATION    1.93  ITEM   RELIABILITY  .79

S.E. OF ITEM MEAN = .16                                                       

UMEAN=.0000 USCALE=1.0000

ITEM RAW SCORE-TO-MEASURE CORRELATION = -.99

As are shown in the Tables, the items do not have any misfit (including infit and outfit); all item indices were between 0.6 and 1.6. Furthermore, the Test Rasch reliability index was acceptable; it was 0.79 digits.

The third instrument which was only used in the DA group was the ‘intervention instrument’. Following the interventionist approach, the pre-planned hints were prepared to intervene the learners with pragmatic production problems during the treatment sessions. The only difference was that the hints order was selected idiosyncratically depending on the EFL learners’ mistakes. These hints were constructed by using the literature and the experts’ views. Basically, the hints were as follows:

 

Figure 1.

The most implicit to the most explicit interventions

 

 

 

 3.3. Data Collection Procedure

The present research occurred through the following stages. In the first stage, the gradually complex interventions were constructed using both literature based on frequency and experts’ views. The dynamic assisted DCT was validated using the expert raters and six novice language learners. The novice learners’ opinions were used to measure the response validity and operativeness of the gradually constructed interventions. In the second step, the participants were selected based on their language proficiency scores on the OPT.

In the next step, the learners in both DA and NDA groups took a discourse completion test (DCT) selected from Shahrokhi and Eshraghi (2016), as well as Zhang (2001), as the pretest. The learners were asked to write their answers to the complaint situations in the pretest. After the pretest, the learners in DA groups took a gradually mediated discourse completion test (DCT) as the treatment material for every one of five treatment sessions. The intervention is highly dependent on the type of errors they made. First, the learners in DA groups were asked to write their response to each situation within DA material. And then, the learners took the graduated prompt format of the DCT which was prepared in the phase of instrument construction. The dynamic version of the test was administered individually and all sessions were recorded. For instance, one of the situations in this study for DA groups is presented below:

Situation 3: One of your neighbors, Judy seldom tightly closes the door of the garbage can after she throws the garbage into it. You are the closest resident to this garbage can. The bad smell of the garbage and the flies bother you a lot. Today, she does that again when you see her near the garbage can. How will you complain? (-SD, S=H)

• Learner: Close the door of the garbage can tightly from tonight because the bad smell of the garbage bothers us. Do it, otherwise, I will speak in other way next time.
• Mediator: Try it again, please.
• Learner: Some neighbors and I are angry because of the bad smell of the garbage. I saw you did not close the door of the garbage can tightly. Do it from tonight.
• Mediator: State your complaint more indirectly, please.
• Learner: All the members of this apartment are annoyed because of the bad smell of the garbage. Please close the door of the garbage can tightly.
• Mediator: Why don’t you give more remedy at the end?
• Learner: All the members of this apartment are annoyed because of the bad smell of the garbage. Please close the door of the garbage can tightly or you can change the place of it.

The results in each stage were scored using the following 6-point Likert scale taken from Taguchi, et al., (2006). The scale ranged from 0 to 5 as follows:

0          shows no performance at all;

1          shows very poor performance (very difficult expressions to understand);

2          shows poor performance (difficult determination of correctness because of the interference from grammatical and sociolinguistic errors);

3          shows fair performance (somewhat correct and appropriate sentences);

4          shows good (mostly correct and appropriate expressions);

5          shows excellent performance (fully correct and appropriate expressions).

In the end, both DA and NDA group took the second DCT test as the posttest. The results were rated by two expert raters using the Taguchi, et al.’s scale. Any disagreements were discussed to reach a consensus.

 

 

3.4. Data Analysis Procedure

To answer the questions posed in this paper, both SPSS and Winsteps were used. Rasch repeated measure is used to investigate the idiosyncratic changes the individuals showed during the intervention procedure.

 

4. Results

4.1. Complaint Strategy Dynamic Assessment: Analysis of Covariance

To answer the first question of this study, “does dynamic assessment have any significant effect on EFL learners’ complaint speech act production”, one-way ANCOVA was run using SPSS 24. The assumptions were checked before running the analysis. The data was revealed to be normally distributed (indices of skewness and kurtosis were less than 2). Furthermore, the data revealed to have homogeneity of regression slope (Figure 2). Also, the groups were not significantly different in the pretest (t (31) = 2, p > 0.05). And finally, data showed homogeneity of variance both in the pretest (Flevene =1.52, p = ns) and post-test (Flevene =3, p = ns).

   Figure 2.

Homogeneity of two groups in pre and post-tests

 

 

 

Table 4.

The Effect of DA on Students’ Complaint Speech Act Performance Tests of between-subjects Effects

Source	Type III Sum of Squares	df	Mean Square	F	Sig.	Partial Eta Squared
Corrected Model	179.804a	2	89.902	10.872	.000	.420
Intercept	310.540	1	310.540	37.554	.000	.556
goroh	119.936	1	119.936	14.504	.001	.326
pre_koli	121.136	1	121.136	14.649	.001	.328
Error	248.075	30	8.269
Total	19532.000	33
Corrected Total	427.879	32
a. R Squared = .420 (Adjusted R Squared = .382)

 

As is shown in Table 4, the effect of dynamic assessment on EFL learners’ complaint speech act performance was significant; F (1, 30) = 14.504, p < 0.05. The eta square was 0.32 which showed a moderate effect size of the data. After controlling the effect of covariance (pretest) the adjusted mean for DA group was 25.6 (SEM = 0.76) which was more than that of NDA group (M = 22.46, SEM = 0.77). This is in line with Daftarifard’s (2016) research who found that DA was more informative than NDA assessment. Elsewhere, Spector (1992) working on phonemic awareness through graduated prompt DA found that the participants’ performance has considerably changed from the pretest and posttest. Furthermore, the author (2016) found that DA was efficient in improving EFL learners’ performance on the FCE, although these changes were idiosyncratic.

 

• Dynamic Assessment and EFL Developmental Trajectories

In the heart of Vygotsky’s theory and Dynamic assessment is the notion of Zone of Proximal Development or idiosyncratic changes that individuals may show during the intervention stage. To answer the second and third questions of this study, “To what extent, do dynamic interventions create qualitative differences in the EFL learners’ complaint speech act production?” and “To what extent, is the EFL learners’ developmental trajectory predictable from their performance in the pretest?”, the EFL learners’ production during the intervention stage were analyzed both descriptively and qualitatively.

The results of the descriptive analysis are shown in Figures 3 and 4. Figure 3 shows that the performance of the lower intermediate DA group from the pretest to the posttest was considerably different. Most of the participants except (participants 7, 8, 9, and 10) developed throughout the study.

 

Figure 3.

Developmental trends in lower intermediate DA group

 

Figure 4 shows that lower intermediate NDA group made no eye-catching developments from the pretest to the posttest.

 

Figure 4.

Developmental trends in lower intermediate NDA group

 

 

Also, to learn about the individual differences during the intervention procedure, the students’ performance was scored on a 5-point Likert scale. The participants had five opportunities to write the correct answers using the hints the researchers provided them with. Whenever the participants were able to get the complete core (i.e., 5), they would not receive any intervention on the task.  For the sake of measurement, the given score of 5 was considered as the score for the rest of remained intervention phase. The raw ordinal scores were then subjected to Winstep to change into the scaled scores which is an interval.  The estimated Rasch Person reliability was 0.92 and Rasch item reliability was estimated as 0.94 with a Cronbach alpha of 0.92. The first performance (the unassisted pretest) was kept as the baseline and other performances were anchored at the unassisted pretest. The results are shown in Figure 5.

 

Figure 5.

Developmental trends using the scaled score in DA group

 

As is shown in Figure 5, the baseline scores of participants were located when the test was unassisted. B, C, D, E, and F were five successive stages the participants received intervention upon their failure to fulfill the task successfully. In Figure 5, some of the participants (1, 7, 12, &13) scored the same in the pretest but developed differently through five-stage interventions; participants’ scores in the final attempt were 4.87 logits whereas participants 1 and 12 have got scores of 5.91 and 2.19 logits respectively. Moreover, some participants regressed considerably. For instance, participants 16 and 6 regressed from a scaled score of -0.46 and 0.45 to -0.55 and 0.39 logits respectively. Furthermore, as is shown in Figure 5, the following four developmental patterns can be observed in the data:

• Some participants’ ability levels in the complaint pragmatics were the same in the unassisted stage but improved differently (the participants number 11 and 15)
• Some of the participants’ ability level was different in the unassisted stage but reached the same logit score in stage two (for instances the participants number 15 and 4) or stage three (for instance, the participants 1 and 12) or stage four (for instance, the participants 9 and 11).
• Some of the participants were scored similarly in the unassisted pretest, developed differently in some stages but reached the same ability level in stage four (like participants 4 and 9)
• And some of the participants had the same ability in the first and last stages of the experience but developed differently within the intervention stage.

 

5. Discussion

The result of this study confirmed that mediational hints in the DA group made a significant change in learners’ complaint speech act production and also the DA group was more successful than NDA. This finding is in line with Daftarifard (2016) who found 7 patterns in the EFL performance on the FCE through DA different stages of interventions. Also, the finding is in line with what Vygotsky (1980) claimed about individuals’ different ZPD even when their unassisted performance is the same. Also, the zigzagged trajectory found in the data is in line with what Ableeva (2010) found in finding who found in his research on DA listening comprehension.

There have been some other studies that are in line with the findings of this research such as the one carried out by Hidri (2014). He investigated the effect of dynamic assessment on listening comprehension and used both static and dynamic assessment approaches. Indeed, both quantitative and qualitative analysis of data indicated better development in the learners’ listening comprehension in the DA group. The findings of the present study are also in line with findings of a study carried out by Sadeghi and Khanahmadi (2011), who assessed the effect of dynamic assessment on the development of Iranian EFL learners’ gramma. They concluded that dynamic assessment was significantly effective in developing the learning of L2 grammar. Both DA and NDA groups in the present research received instruction implicitly. However, intervention within the test method seems to arise learners’ noticing their gaps during the DA session. The finding is in line with Trenchs (1994) who investigated the EFL Catalan and American speakers’ ability to transfer the complaint speech act from their language to English using Bilingual Discourse Completion Questionnaire.  Trenchs (1994) concluded, “although both groups made use of similar semantic formulas, EFL speakers showed negative pragmatic transfer” (p. 281).

This is also true with Aljaafreh and Lantolf's (1994) study in which the impact of mediation on three EFL learners’ grammatical production in composition was examined. The results indicated that dynamic assessment was significantly effective in helping the learners to gain control over the grammatical structures. Elsewhere, Nassaji et al. (2000) followed Aljaafreh and Lantolfs's (1994) research and investigated the interaction between a mediator and two ESL learners to find which one of the mediation types (ZPD-sensitive one vs. random mediation type) led to development. The result of this study proved that the learner receiving ZPD-sensitive mediation improved more than the non-ZPD learner on the final composition task. In the present research, the DA group received ZPD sensitive feedback and mediation. This probably causes them to notice their gap. This finding is in line with Alavi et al.’s (2020) research who assessed the effect of dynamic assessment on improving EFL learners’ pragmatic knowledge. The results revealed that dynamic assessment was significantly effective in promoting the learners’ complaint speech act performance.  The difference between the present study and that of Alavi et al.’s study is that in the former the intervention was ZPD sensitive but in the latter, the mediation was constructed statically. To change computerized DA into a ZPD sensitive program, we suggest the use of artificial intelligence.

Another finding of this research supported the zigzagged nature of developmental trajectories. This is in line with the results of Poehner’s (2005) research which focused on the effect of dynamic assessment on the oral abilities of six advanced students of L2 French. According to Poehner (2005), “the findings suggest that DA is an effective means of understanding learners’ abilities and helping them to overcome linguistic problems” (p. iv) as the EFL learners’ development is Zigzagged. Elsewhere, Ableeva (2010) studied the effect of dynamic assessment on promoting listening comprehension of intermediate university students of L2 French and compared the findings of this study with those of traditional tests. Ableeva (2010) concluded that the learners’ listening comprehension development in the DA group was idiosyncratic.

 

6. Conclusion

The findings of the present study indicated that the performance of  DA group from the pretest to the posttest was significantly different. In other words, DA group in the posttest greatly outperformed NDA group in the pretest. Whereas, NDA group made no significant changes from the pretest to the posttest. Furthermore, this developmental trend was found to be idiosyncratic or ZPD based thereby helping learners to notice their gap. As finding an efficient way of teaching speech act to EFL learners seems necessary (Thijittang & Lê, 2010) ZPD based speech act instruction using DA can help learners to internalize their understanding about the correct use of language to express their complaints appropriately in various situations.

According to the findings drawn from the present research, DA-based, ZPD-oriented, and interactive activities can help EFL learners have better learning of L2 pragmatics. Thus, it is suggested that such activities can be applied to the EFL lessons. Furthermore, this study has provided an opportunity for testing the complaint speech act by using dynamic assessment. Therefore, it is recommended that the results of the present study can be applied as the baseline data for designing tests in EFL classrooms regarding different speech acts. The findings of this study can have effective implications in all educational contexts including foreign language learning and teaching especially curriculum designers to take useful steps for learners’ development. Further research can be conducted to study the effect of DA on different levels of language proficiency in different parts of foreign language learning or to endeavor to apply artificial intelligence to DA process to enable ZPD based or idiosyncratic intervention within the test speech act DA format.  

مراجع

Ableeva, R. (2010). Dynamic assessment of listening comprehension in L2 French (Unpublished doctoral dissertation), The Pennsylvania State University, United States.

Adams, R. J., Wu, M. L., & Wilson, M. (2012). The Rasch rating model and the disordered threshold controversy. Educational and Psychologial Measurement, 72(4), 547-573.

Alavi, S., Shahsavar, M., & Norouzi, M. (2020). Diagnosing EFL learners development of pragmatic competence implementing computerized dynamic assessment. Issues in  Language Teaching, 9(1), 117-149.

Aljaafreh, A., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. Modern Language Journal, 78(4), 465-483.

Arent, R. (1996). Sociopragmatic decisions regarding complaints by Chinese learners and NSs of American English. Hong Kong Journal of Applied Linguistics, 1(1), 125-147.

Brown, P., & Levinson, S. C. (1987). Politeness: Some universals in language usage: Cambridge university press.

Cutting, J., & Fordyce, K. (2020). Pragmatics: a resource book for students: Routledge.

Daftarifard, P. (2016). Reading comprehension as an interactional ability: Evidence from task-based and response-based interventions (unpublished doctoral dissertation), Science and Research Branch, Islamic Azad University, Tehran, Iran.

Eshraghi, A., & Shahrokhi, M. (2016). The realization of complaint strategies among Iranian female EFL learners and female native English speakers: A politeness perspective. International Journal of English Linguistics, 6(2), 8-20.

Hidri, S. (2014). Developing and evaluating a dynamic assessment of listening comprehension in an EFL context. Language Testing in Asia, 4(4), 1-19.

Lakoff, R. (1973). The logic of politeness. Comparative Literature Studies, 9, 292-305.

Lantolf, J. P. (2009). Dynamic assessment: The dialectic integration of instruction and assessment. Language Teaching, 42(3), 355-368.

Lantolf, J. P., & Poehner, M. E. (2008). Dynamic assessment. In Encyclopedia of language education (pp. 273-284). Cambridge University Press.

Moradian, M., Asadi, M., & Azadbakht, Z. (2019). Effects of concurrent group dynamic assessment on Iranian EFL learners’ pragmatic competence: A case of requests and refusals. Journal of Research in Applied Linguistics, 10(2), 106-135.

Murphy, B., & Neu, J. (1996). My grade is too low: The speech act of complaining. In S. M. Gass & J. Neu (Eds.), Speech acts across cultures: Challenges to communication in a second language (pp. 191-216): Berlin.

Nassaji, H., Kushki, A., & Rahimi, M. (2020). The differential diagnostic affordances of interventionist and interactionist dynamic assessment for L2 argumentative writing. Language Sociocultural Theory, 7(2), 151-175.

Nassaji, H., & Swain, M. (2000). A Vygotskian perspective on corrective feedback in L2: The effect of random versus negotiated help on the learning of English articles. Language awareness, 9(1), 34-51.

Olshtain, E., & Weinbach, L. (1987). Complaints: A study of speech act behavior among native  and non-native speakers of Hebrew. In J. Verschueren & M. Bertucelli-Papi (Eds.), The pragmatic perspective (pp. 195-208). John Benjamins. (Reprinted from: Eds.).

Olshtain, E., & Weinbach, L. (1993). Interlanguage features of the speech act of complaining. In G. Kasper & S. Blum-Kulka (Eds.), Inerlanguage pragmatics (pp. 108-122): Oxford University Press.

Pakzadian, S. S., & Tajeddin, Z. (2014). Dynamic self-assessment: Is it an effective task for pragmatic development and metapragmatic awareness? Procedia-Social Behavioral Sciences, 136, 283-287.

Piotrowska, M. (1987). An investigation into the sociolinguistic competence of Hong Kong university students, with specific reference to making complains. (Unpublished master’s thesis), Hong Kong University, Hong Kong.

Poehner, M. E. (2005). Dynamic assessment of oral proficiency among advanced L2 learners of French: The Pennsylvania State University.

Sadeghi, K., & Khanahmadi, F. (2011). Dynamic assessment of L2 grammar of Iranian EFL learners: The role of mediated learning experience. . International Journal of Academic Research, 3, 931-936.

Schaefer, J., & James, F. (1982). An examination of language as gesture in a play by Gertrude Stein. Text Performance Quarterly, 3(1), 1-14.

Shahrokhi, M., & Jan, J. M. (2012). The realization of apology strategies among Persian males. Procedia-Social Behavioral Sciences, 46, 692-700.

Sijtsma, K. (1993). Claasical and modern test theory with an eye toward learning potential testing. In J.H.M. Hamers, K. Sijtsma, & A. J. J. M. Ruijssenaars (Eds.), Learning potential assessment: Theoretical, methodological and practical issues 117-133: Swets & Zeitlinger.

Spector, J. E. (1992). Predicting progress in beginning reading: Dynamic assessment of phonemic awareness. Journal of educational psychology, 84(3), 353.

Taguchi, E., Gorsuch, G. J., & Sasamoto, E. (2006). Developing second and foreign language reading fluency and its effect on comprehension: A missing link. Reading Matrix: An International Online Journal, 6(2).

Tajeddin, Z., & Tayebipour, F. (2012). The effect of dynamic assessment on EFL learners’ acquisition of request and apology. The Journal of Teaching Language Skills, 4, 87-118.

Takahashi, S. (2010). Assessing learnability in second language pragmatics. In A. T. (Eds.) (Ed.), Pragmatics across languages and cultures (pp. 391-422): De Gruyter Mouton.

Thijittang, S., & Lê, T. (2010). Pragmatics Strategies of English of Thai University Students. Paper presented at the Australian Association for Research in Education, Retrieved March 31, 2013.

Trenchs, P., Mireia (1994). Complaining in Catalan, complaining in English: A comparative study of native and EFL speakers. Revista española de lingüística aplicada(10), 271-288.

Trosborg, A. (1995). Interlanguage pragmatics: Requests, complaints, and apologies: Mouton Gruyter.

Vygotsky, L. S. (1980). Mind in society: The development of higher psychological processes: Harvard university press.

Wistner, B., Sakai, H., & Abe, M. (2009). An analysis of the Oxford Placement Test and the Michigan English Placement Test as L2 proficiency tests. Bulletin of the Faculty of Letters, Hosei University, 58(2), 33-44.

Zhang, D. (2001). The speech act of complaining: A cross-cultural comparative study of Chinese and American English speakers: Iowa State University.