Requesting in an EFL Institutional Context: Comparing and Contrasting WDCT, ODCT, and Role-Play with Natural Method

Research in English Language Pedagogy

مقاله 8، دوره 7، شماره 2 - شماره پیاپی 13، مهر 2019، صفحه 363-385   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.30486/relp.2019.665953

نویسندگان

Rasoul Mohammad Hosseinpur^{* 1}؛ Reza Bagheri Nevisi¹؛ Abdolreza Lowni²

¹English Department, Faculty of Humanities, University of Qom, Qom, Iran

²University of Qom

چکیده

Pragmatic assessment has recently opened up a new line of inquiry for many interested researchers within the realm of L2 pragmatics. Accordingly, different methods have been proposed to assess pragmatic competence. Drawing on request speech act, this study aimed at comparing and contrasting Written Discourse Completion Task (WDCT), Oral Discourse Completion Task (ODCT), and Role-play with the natural method in terms of five dependent variables: length, repetitions, omissions, inversions, and exclamation particles. To this end, 27 intermediate level EFL learners were asked to make two requests with contextual features of low-status, low-imposition, and two requests with contextual features of high-status, low-imposition using each of the three elicitation techniques. Having recorded the natural talk-in interactions of all students and teachers over 15 weeks, the researchers transcribed the gathered data for further in-depth pragmatic analysis. To analyze the data, Chi-Square and binominal tests were run. The findings indicated that role-play yielded the data closest to the natural method in terms of the dependent variables and the differences between role-play and the natural method were less significant than those inherent in WDCT and ODCT. The study implies that more authentic and natural data can be elicited by incorporating features of the natural method into the other pragmatic data collection procedures, namely, WDCT and ODCT.

کلیدواژه‌ها

WDCT؛ ODCT؛ role-play؛ natural methodology؛ dependent variables

اصل مقاله

1. Introduction

Several studies conducted within the realm of second language (L2) pragmatics have attempted to delve into different aspects of L2 pragmatics, and have particularly concentrated on various approaches to L2 pragmatics teaching (Ellis, 2008; Nassaji & Fotos, 2007; Norris & Ortega, 2000; Taguchi 2018). The upsurge of interest in L2 pragmatics studies has coincided with a growing interest in the pragmatic assessment which has lately taken on a new turn (Bardovi-Harlig & Shin, 2014; Eslami-Rasekh & Mirzaei, 2014; Golato, 2003; Uso-Juan & Martinez-Flor, 2014; Yuan, 2001). Accordingly, such studies on pragmatic assessment have opened up a new line of inquiry for many interested researchers within the field of L2 pragmatics.

Pragmatic assessment, as an important area of pragmatic studies, puts its primary focus on evaluating learners' pragmatic competence. Pragmatic competence has been defined by Taguchi (2009) as the learners’ ability to appropriately use language in social contexts. Due to the complexity of the pragmatic dimension of language use and the presence of many intervening and overlapping variables during its use, pragmatic assessment can be regarded as a complicated phenomenon which plays a pivotal role in developing well-designed methods to elicit learners' pragmatic knowledge (Uso-Juan & Martinez-Flor, 2014). The pragmatic assessment has always been one of the main concerns of most interlanguage and cross-sectional pragmatic studies (Alcon-Soler & Safont, 2018).

Different methods have been proposed to assess pragmatic competence. These methods could be classified under two general rubrics of methodologies: The ones that can gather "natural discourse data" and methods which can extract "elicited data" (Felix-Brasdefer, 2007). In order to gather naturally-occurring data, some kind of observation, recording or video-taping/audio-taping will be needed. Field observations and recordings of naturally occurring talk-in-interaction are two methods that fall within this category. However, in order to elicit data in some methods such as discourse completion tasks, questionnaires, recall protocols, and role-plays, the researcher needs to tap learners' pragmatic competence (Golato, 2003). Each methodology has its own merits and demerits, and not all of them lend themselves to all contexts, that is, each will have to be applied in the appropriate setting. However, as Cohen (2008) has stated, problems and questions far outweigh the provided solutions so far and more thorough and in-depth analyses and investigations are deemed necessary to make future headways in the field of pragmatics in general and pragmatic assessment in particular.

To date, several studies have addressed the pragmatic assessment issue (e.g., Bardovi-Harlig & Shin, 2014; Eslami-Rasekh & Mirzaei, 2014; Golato, 2003; Taguchi, 2018; Yuan, 2001). However, few studies have incorporated several data collection procedures and compared and contrasted them with one another.

Contrary to most of the pragmatic studies that take advantage of every day conversational interactions or conversational discourse as their data (Bardovi-Harlig & Hartford, 2005), this study elicited and gathered learners’ request speech acts as data in institutional contexts. Bardovi-Harlig and Hartford (2005) have defined institutional talks as "talk that occurs in the course of carrying out an institution's business, usually between an institutional representative and a client" (p, 8). Therefore, institutional discourses are those interactions which commonly take place in various contexts, including courts of law, surgical rooms, and even educational contexts. Such educational settings demonstrate and represent common talk-in interactions and social negotiations of meanings in pedagogical contexts. Such contexts have been selectively picked by the researchers because they possess three important features. According to Bardovi-Harlig and Hartford (2005), institutional talks enjoy three essential characteristics of comparability (there is control over various variables to make comparison easier), interactivity (there are turn-taking patterns in the process of interaction), and consequentiality (there is a real life goal for communicating). Ellis (2008) describes the data entailing these three features as "ideally data".

Relying upon learners’ request speech act, the present study compared and contrasted Written Discourse Completion Task (WDCT), Oral Discourse Completion Task (ODCT), and Role-play data collection procedures with the Natural method in an EFL institutional context in terms of a number of dependent variables. The participants’ frequently-employed requests in the classroom context were analyzed in terms of length, the number of repetitions, omissions, inversions, and the exclamation particles commonly referred to as inherent characteristics of natural data (Yuan, 2001).

 

2. Literature Review

The growing interest in pragmatic studies has coincided with a growing interest in pragmatic assessment. Pragmatic measurement studies (e.g., Bardovi-Harlig & Shin, 2014; Eslami-Rasekh & Mirzaei, 2014; Golato, 2003; Yuan, 2001) try to seek through different methods of data collection in the pragmatic area to find the best way to assess pragmatic awareness and production of the participants. The Natural and elicitation methods such as WDCT, ODCT and role-play are popular data collection methods which are used to measure pragmatic production of language learners.

The natural method as one type of data collection method requires observers' observations through which the observer may either take notes online or record with audio/visual equipment the natural interactions of the intended participants (Félix-Brasdefer, 2007; Schauer, 2009). This measure of pragmatic competence has the merit of capturing the very essence of real-life, authentic interaction features (Golato, 2003; Schauer, 2009; Yuan, 2001).

Drawing upon conversation analysis and discourse analysis procedures, researchers can transcribe and analyze the recorded data through audio/visual equipment. Recently, in addition to audio/video equipment, the potentialities of modern digital technologies such as social networking and video conferencing have been employed by interested researchers to collect interactional data in the face-to-face conversations (Taguchi, 2018). Striking a logical balance between the allocated time and the outcome the researcher achieves is of paramount significance in this data collection method (House, 2018).   

Obviously, the most important advantage of this method of data collection is the naturalness. The collected data really reflect the true features of real life interactions because of their occurrences in natural interactions. On the other hand, in this method, the researcher has no control over the variables under the investigation, and this makes it a little difficult and broad to study a particular variable. One further drawback of this method is the negative impact of the presence of the observer or recording equipment in the environment. It can cause what Labov (1972) referred to as the "observer's paradox" which adversely affects the interactants' negotiations and talk-in interactions. Finally, due to the restrictions associated with the contexts within which the researchers can gather the data, the lack of generalizability is a serious drawback of this method (Taguchi, 2018).

WDCT is a kind of pragmatic measure which measures pragmatic production of the respondents through completing some written tasks featuring a description of the situation, social status of the participants, and the respondents' roles. Participants are required to write down their responses to the tasks in the provided blanks. Additionally, there is one specific type of WDCT which refers to the probable answers of the imaginary hearer called rejoinder (Schauer, 2009). Because of their overall practicality, they enjoy widespread popularity among many researchers. Their practicality allows researchers to exert high control over extraneous variables (Golato, 2003). They also make a collection of ample samples within a relatively short period of time possible. However, there are still controversies over the validity of using such measures to assess pragmatic competence, and some researchers question and doubt their validity since they believe natural data will not be elicited through the application of such measures (Hartford & Bardovi-Harlig, 1992). Although Discourse Completion Tasks (DCTs) fail to consider some discourse features such as paralinguistic and non-verbal elements in the learners’ speech, they suit to measure learners’ metapragmatic competence, the speakers’ awareness of the pragmatic functions of a speech act, and are usually associated with little variation (House, 2018).

Oral discourse task, as one type of completion task, requires learners to listen to the oral description of the tasks which entail important information about the setting, participants' roles, and interlocutors' status. Brown (2001) defines ODCT:

An oral discourse completion task (ODCT) is a pragmatic instrument that requires the students listening to a description of a situation (usually on a tape recorder) and to say aloud what they would say in that situation (typically into another tape recorder). (P. 302)

This form of the interlanguage pragmatic measure was developed in response to the drawbacks of WDCTs, and it was developed to make up for the inherent limitations of WDCTs. It was further claimed that WDCTs elicit written responses and these data cannot be a real representative of the natural conversations since the same features of oral responses will not be shared with them (Golato, 2003). Although ODCTs elicit oral responses which can alleviate this problem of WDCTs, they still suffer from similar drawbacks of WDCTs; it is stated that there is no real interaction among the interactants’ oral responses in ODCTs (Yuan, 2001).

Role-play is yet another method of eliciting data in interlanguage pragmatic studies. In this method of data elicitation, two parties of the conversation assume roles based upon predefined experimental conditions to provide the researcher with spoken data. Role-plays, which could be easily and systematically manipulated by the researcher, lend themselves well to tackle the learners’ linguistic-pragmatic knowledge (House, 2018). One of the advantages of Role-play over Natural method is that the researcher has some control over the variables, but the process of elicitation through this method is rather time-consuming (Eslami-Rasekh & Mirzaie, 2014). Nonetheless, the degree of the naturalness of the elicited data in this kind of method is still under question (Golato, 2003). Kasper (2000) also has pointed to yet another limitation of this method where the participants need to be actually capable of playing roles, and otherwise, this can be taxing for them. According to Taguchi (2018), this method along with naturalistic conversations and computer-mediated communication enjoys adaptability in interaction, through which the speaker can adjust his speech to the changes occurring in the process of interaction.

 

2.1. Studies Focusing on Pragmatic Measures

The importance of employing the most efficient measure of pragmatic knowledge to elicit and gather reliable data has led many researchers to compare existing measures to pinpoint the most useful ones. Sasaki (1998) tried to contrast Role-plays and WDCTs to determine their differences and similarities. She conducted her study in the Japanese EFL context and intended to elicit the participants’ requests and refusal responses. The findings were indicative of the fact that Role-plays elicited more extended data accompanied by more strategy use.

Yuan (2001) also essayed to compare and contrast pragmatic measures. As a matter of fact, her study was more comprehensive than that of Sasaki (1998) and incorporated more pragmatic measures, including ODCT, WDCT, field notes, and recorded conversations to display the differences between the elicited and gathered data. She compared elicited and gathered data through these four measures in terms of five dependent variables; namely, response length, the number of exclamation particles, the number of omissions, the number of repetitions, and the number of inversions. This study was carried out in a non-western context with complement and compliment responses as the speech act to be elicited and gathered. The findings revealed the proximity of the elicited data through ODCT to the naturally-occurring data, but concluded that ODCTs share some limitations of WDCTs as well.

Golato (2003) compared two types of pragmatic measures: Naturally occurring data and DCTs. She designed a DCT reflective of the context of the natural interactions to make the comparisons even more tangible and plausible. The natural interactions were gathered in advance and the speech act under examination in the study was a compliment. The data were elicited and gathered through these two techniques did not match, and based upon this discrepancy; it was concluded that investigating natural talk-in interactions could lead to the study of the language organization and the elicited data through DCTs are metapragmatic data.

Eslami-Rasekh and Mirzaie (2014) concentrated on WDCT and ODCT in a non-western context. They compared WDCT and ODCT in terms of response length, range and context of the expression, formality level, and spoken genres in contrast with written genres. Based on their findings, ODCTs tended to elicit longer, more elaborated, and more linguistically-oriented forms in comparison with WDCTs. They further added that WDCTs are not good candidates to elicit data in languages with different forms of spoken and written varieties in terms of stylistic variations. Therefore, their study indicated that WDCTs and ODCTs were different with regard to the kind of data they could elicit.

A review of previous studies on different methods of pragmatic knowledge assessment reveals that few studies have considered the institutional context for the data collection procedure in an EFL context. Moreover, most of the studies have only focused on comparing and contrasting two measures of pragmatic knowledge.

 

2.2. Dependent Variables

The participants’ gathered and elicited requests were analyzed in this study in terms of five dependent variables which are referred to as inherent characteristics of natural data (Yuan, 2001). These variables were the length, the number of repetitions, the number of omissions, the number of inversions, and the number of exclamation particles. Length is the number of request characters which are used by a requester, such as the following example which contains 5 characters:

1. Can you open the window?

Repetition is the number of repeated characters in a request. For example, if the requester makes a request like example 2, he has repeated "can you" in his request and there is one repetition in this request.

2. Can you can you open the window?

Omission simply refers to the deletion of one element of the request in which the requester leaves out one element of the request in the hope of retrieving that element by the requestee through context. As it is clear in example 3, the requester left out "the window".

3. Can you open?

Inversion means the promotion of one element of the request to the request-initial to put emphasis on the inverted element like "the window" in example 4 which was inverted to the sentence initial position.

4. The window, can you open?

Exclamation particles are some elements which express surprise, exclamation, or hesitation such as "oh, wow, gosh, my goodness, um, and ah".  As it is evident in example 5, the requester employs one exclamation particle in this request.

5. Oh, can you open the window?

Drawing upon the above-mentioned dependent variables, the present study intended to compare and contrast WDCT, ODCT, and Role-play with the natural method. Therefore, the following questions were formulated:

1. What patterns of data elicitation can be found in WDCT in terms of the dependent variables in comparison with the patterns of data gathering in the Natural method?
2. What patterns of data elicitation can be found in ODCT in terms of the dependent variables in comparison with the patterns of data gathering in the Natural method data?
3. What patterns of data elicitation can be found in Role-play in terms of the dependent variables in comparison with the patterns of data gathering in Natural method?

 

3. Methodology

3.1. Design and Context of the Study

The current study employed a quantitative approach which needed data collection at the first phase of the study in the natural context. After data gathering in the natural setting, WDCT, ODCT, and Role play were employed to elicit data, and the gathered and elicited data were analyzed quantitatively.

 

3.2. Participants

From among 56 language learners who initially participated in the study, 27 students met the required conditions to accompany the researchers in the course of this study. The selection of these participants as the sample of this study was done based on their successful production of at least two requests with the contextual features of low-status, low-imposition, and two requests with the contextual features of high-status, low-imposition in the institutional context of the 6 intact EFL language classrooms over a 15-week educational semester.

Therefore, the study was conducted with 27 male and female intermediate–level Iranian EFL learners whose age ranged from 19 to 28. They all were studying in an English language institute in Malayer and agreed to participate and cooperate in all phases of the study to the extent possible. Based on the obtained results from the Michigan Test of English Language Proficiency (MTELP) conducted prior to the study, it was recognized that the participants were all at level 2 or intermediate level. Basic demographic characteristics such as L1 and culture were common among the participants.

 

3.3. Data Collection Procedure

To elicit and gather data, four methods of pragmatic data collection procedures, WDCT, ODCT, Role-play, and Natural method, were utilized. Having recorded the natural interactions of all students and teachers in 6 classes over a 15-week educational semester, the researchers transcribed the gathered data for further in-depth pragmatic analysis. The gathered data were analyzed to find out the most prominent pragmatic features in the naturally-occurring data. The data gathered in the first three weeks were disregarded and dispensed with to avoid observer's paradox. Important features of the naturally-occurring data were identified in advance because the selection of the WDCT, ODCT, and Role-play's tasks should have been based upon the very same features extracted from the naturally-occurring data. In other words, the included tasks in WDCT, ODCT, and Role-play were all reflective of the contextual features of naturally-occurring data, and therefore making analogies and comparisons were even made more feasible and plausible. The in-depth and thorough analysis of the transcribed data indicated that an absolute majority of the selected participants' requests revolved around two contextual features, that is, low-status, low-imposition, and high-status, low-imposition. Social status refers to the social power of a speaker over a hearer, or vice versa. Following Schauer's (2009) study, two values of social status, high and equal or low, were considered in this study. In the same vein, the language used when requesting also depends on the type or imposition of the task we want the other person to do. When we are asking for a big favor, we need to be more formal. Imposition also has two values, high and low.

Accordingly, the researchers made their selection of the tasks of the Role-play, WDCT, and ODCT consistent and congruent with the above-stated contextual features. Social status simply refers to the social power of the interlocutors over one another and imposition is how heavy a burden request imposes upon the requestee. The included contextual features of the social status and imposition in the current study were high and low respectively.

The selected tasks, in line with the specified contextual features in natural data, for the Role-play, WDCT, and ODCT, were all taken from Schauer (2009) and Jalilifar (2009). Next, the participants were involved in the Role-plays. The printed tasks were distributed among participants who were paired in two and assumed roles. For the first two tasks, the roles were assumed by two students because the social status of the first two scenarios was low. However, as for the second two tasks, the teacher also played the role of the requestee because the social status of the second two requests was high. The participants' interactions were recorded and transcribed. In the following session, the recorded scenarios of the ODCTs were played for the participants. They listened to the scenarios and made their requests based upon the very same scenarios. Their responses were recorded and transcribed for later in-depth analysis.

Finally, one week later, the WDCTs were distributed among the participants. The WDCT scenarios were not the same as those of Role-play and ODCT, but shared the same contextual features of status and imposition: Two of the tasks were low-status, low-imposition, and the other two were high-status, low-imposition. The participants were asked to read the scenarios and write their answers in the provided blanks.

Each individual participant made four requests in natural data, four through Role-play, four through ODCT, and four by WDCT with the same contextual features of imposition and status. As a result, employing four different pragmatic measures, 16 requests were made in total by each participant.

 

3.4. Data Analysis Procedure

The transcribed data of the Natural method, Role-plays, and ODCTs along with the written data from the WDCTs were analyzed by two Ph.D. holders in TEFL to specify the dependent variables inherent in the data. Having determined all the dependent variables in the data pool, the researchers counted their frequency of occurrences. In other words, the numbers of the characters, repeated items, omitted elements, inverted elements, and exclamation particles were counted for each measure to be compared and contrasted with one another through Statistical Package for Social Sciences (SPSS). To compare these four methods, Binomial test was run.

 

4. Results

In this study, the Natural method was compared and contrasted with WDCT, ODCT, and Role-play in terms of five dependent variables. The selection of these variables was made based on the fact that they all constitute the building blocks of natural day-to-day conversations (Yuan, 2001). The included dependent variables were the length of the requests, the number of repetitions, omissions, inversions, and exclamation particles. Table 1 illustrates the frequency and percentages of the occurrences of these dependent variables.

 

Table 1.

Descriptive Statistics of Dependent Variables for pragmatic measures

Dependent Variables					Total
Length	Repetition	Omission	Inversion	Exclamation
Method	WDCT	Count	722	5	10	6	3	746
		% within Dependent	28.6%	1.0%	5.6%	20.7%	11.1%	23.0%
	ODCT	Count	710	142	26	6	3	887
		% within Dependent	28.1%	29.6%	14.4%	20.7%	11.1%	27.3%
	Role-play	Count	625	205	65	10	8	913
		% within Dependent	24.7%	42.7%	36.1%	34.5%	29.6%	28.1%
	Natural	Count	471	128	79	7	13	698
		% within Dependent	18.6%	26.7%	43.9%	24.1%	48.1%	21.5%
Total		Count	2528	480	180	29	27	3244
		% within Dependent	100.0%	100.0%	100.0%	100.0%	100.0%	100.0%

 

Table 1 shows that the length of the requests produced in the Natural method (471) are shorter than the one produced in the WDCT (722) and ODCT (710). The instances of repetition have been highest in Role-play (205) and ODCT (142). It could be seen from Table 1 that the most omission has occurred in the Natural method (79) and Role-play (65). Instances of inversion and exclamation have been somewhat the same and do not indicate any sharp contrast.

Binomial tests were run to display the similarities and differences between WDCT, ODCT, and Role-play and Natural method in terms of length, the number of repetitions, omissions, inversions, and exclamation particles. Table 2 illustrates the results of the binomial tests for the WDCT and the Natural data in terms of length, repetition, omission, inversion, and exclamation Particles (Dependent Variables).

 

Table 2.

Binomial Tests for the WDCT and Natural Data in Terms of Dependent Variables

Category	N	Observed Prop.	Test Prop.	Exact Sig. (2-tailed)
WDCT & Natural	Length	WDCT	722.00	722	.61	.50	.000
		Natural   Total	471.00	471   1193	.39   1.00
WDCT & Natural	repetition	WDCT	5.00	5	.04	.50	.000
		Natural   Total	128.00	128   133	.96   1.00
WDCT & Natural	Omission	WDCT	10.00	10	.11	.50	.000
		Natural   Total	79.00	79   89	.89   1.00
WDCT & Natural	Inversion	WDCT	6.00	6	.46	.50	1.000
		Natural   Total	7.00	7   13	.54   1.00
WDCT & Natural	Exclamation particles	WDCT     Natural   Total	3   13.00	3   13   16	.19   .81   1.00	.50	.021

 

Prop: Proportion

Table 2 reveals that the differences between the WDCT and the Natural method in terms of length, the number of repetitions, the number of omissions, and the number of exclamation particles are statistically meaningful. However, the number of inversions did not significantly differ in both methods. Therefore, it could be stated that WDCT does not elicit the same data as the Natural method in terms of length, the number of repetitions, omissions, and exclamation particles, whereas WDCT elicits the same data as the Natural method in terms of the number of inversions. Table 3 demonstrates the results of the binomial tests for the ODCT and the Natural data in terms of length, repetition, omission, inversion, and exclamation Particles (Dependent Variables).

 

Table 3.

Binomial Tests for the ODCT and Natural Data in Terms of Dependent Variables

 

 

Category	N	Observed Prop.	Test Prop.	Exact Sig. (2-tailed)
ODCT &Natural	Length	ODCT	710.00	710	.60	.50	.000
		Natural   Total	471.00	471   1181	.40   1.00
ODCT &Natural	repetition	ODCT	142.00	42	.53	.50	.429
		Natural   Total	128.00	128   270	.47   1.00
ODCT &Natural	Omission	ODCT	26.00	26	.25	.50	.000
		Natural   Total	79.00	79   105	.75   1.00
ODCT &Natural	Inversion	ODCT	6.00	6	.46	.50	1.000
		Natural   Total	7.00	7   13	.54   1.00
ODCT &Natural	Exclamation particles	ODCT     Natural   Total	3   13.00	3   13   16	.19   .81   1.00	.50	.021

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Prop: Proportion

 

Table 3 illustrates that noticeable differences exist between the ODCT and the Natural method in terms of length, the number of omissions, and the number of exclamation particles. The requests made in the ODCT are almost two times longer than those made through the Natural method. Omissions occur more frequently in the Natural data in comparison with the ones occurring in ODCT, almost three times more than those which transpire in ODCT. Similarly, exclamation particles are very much frequently in  the Natural method, approximately four times higher than the instances occurring in ODCT. Nonetheless, the two methods did not significantly differ in terms of the number of repetitions and the number of inversions. In other words, the participants employed different sets of the dependent variables in the ODCT and natural settings. The length, the number of omissions, and the number of exclamation particles' patterns vary in the ODCT and Natural method. Needless to say, the number of repetitions and inversions remained almost the same and were insignificant. Therefore, it can be concluded that ODCT does not elicit the same data as the Natural method in terms of length, the number of omissions, and exclamation particles, but elicits the same data as Natural method in terms of the number of repetitions and inversions.

Table 4 depicts the results of the binomial tests for the Role-play and Natural data in terms of length, repetition, omission, inversion, and exclamation Particles (Dependent Variables).

 

Table 4.

Binomial Tests for the Role-play and Natural Data in Terms of Dependent Variables

 

 

Category	N	Observed Prop.	Test Prop.	Exact Sig. (2-tailed)
Role-play & Natural	Length	Role-play	625.00	625	.57	.50	.000
		Natural   Total	471.00	471   1096	.43   1.00
Role-play & Natural	repetition	Role-play	205.00	205	.62	.50	.000
		Natural   Total	128.00	128   333	.38   1.00
Role-play & Natural	Omission	Role-play	65.00	65	.45	.50	.279
		Natural   Total	79.00	79   144	.55   1.00
Role-play & Natural	Inversion	Role-play	10.00	10	.59	.50	.629
		Natural   Total	7.00	7   17	.41   1.00
Role-play & Natural	Exclamation particles	Role-play   Natural   Total	8   13.00	3   13   21	.38   .62   1.00	.50	.383
Prop: Proportion

 

The results presented in Table 4 indicate that there were marked differences between the Role-play and the Natural method in terms of length and the number of repetitions. However, the differences between the Role-play and the Natural method in terms of the number of omissions, inversions, and exclamation particles remained almost insignificant.  In other words, the data elicited via the Role-plays and gathered through the Natural method turned out to be meaningfully different in terms of length and the number of repetitions, while their differences were rather unmarked in terms of the number of omissions, inversions, and exclamation particles. Therefore, it can be concluded that Role-play could elicit partly similar data to Natural method in terms of the number of omissions, inversions, and exclamation particles.

Figure 1 portrays the differences and similarities between the WDCT, ODCT, and Role-play with Natural method. As shown in Figure1, in terms of the length of the elicited requests, all the three measures were different from the Natural method. Concerning the number of the repeated characters in the requests, WDCT and Role-play differed from the Natural method. Finally, regarding exclamation particles and omissions, WDCT and ODCT did not elicit the same data as the Natural method. Therefore, it can be stated that Role-play could yield the data closest to  the Natural method in terms of the dependent variables.

 

Figure 1: Dependent variable use pattern in the WDCT, ODCT, Role-play, and Natural data.

 

The present study intended to compare and contrast WDCT, ODCT, and Role-play, as three common pragmatic data elicitation methods, with Natural method in classroom institutional context. Relying upon the request speech act, four measures of pragmatic knowledge were explored in terms of the length, the number of repetitions, omissions, inversions, and of exclamation particles in the EFL participants’ requests. These five dependent variables are considered, the building blocks of natural day-to-day conversations (Yuan, 2001).

The findings revealed that the gathered data through the Natural method shared the omission, exclamation, and inversion features with the Role-play, repetition and inversion features with the ODCT, and inversion feature with the WDCT. Therefore, the results indicated that, in terms of the five dependent variables, Role-play and, to some extent, ODCT approximated the Natural method.

In line with many other studies (e.g., Economidou-Kogetsidis, 2013; Golato, 2003; Hartford & Bardovi-Harlig, 1992; Turnbull, 2001; Yuan, 2001), the findings of this study suggested that WDCT did not elicit data similar to Natural method. Likewise, although ODCT and Natural method both shared the same mode of delivery, they differed in terms of length, omission, and exclamation particle features. This finding supports other studies (e.g., Turnbull, 2001) which, in spite of some similarities, highlight marked distinctions between these two measures.

The absence of appropriate situational prompts in the WDCT and ODCT might justify the existence of conspicuous differences between these two methods and the Natural method (Economidou-Kogetsidis, 2013). Lack of appropriate and proper context might make the respondents expand, elaborate, and expatiate on their requests to compensate for the loss and lack of context, and this would possibly have prompted the participants to employ lengthy requests and avoid some features such as omission. That was why some kind of exaggeration and overreaction were discerned in the DCTs highlighting that participants used lengthy requests to ensure that they had produced appropriate requests (Woodfield, 2012).

Another probable explanation for the marked differences between the two DCTs and the Natural method might lie in the nature of the interactions in these pragmatic measures (Golato, 2003). In real-life interactions, the only purpose of the interaction is communication and individuals pay little attention to the surface linguistic structures and features, and sometimes communicators leave several elements out, invert them, or even repeat them to add emphasis.  In the DCTs, the respondents mainly focus on the surface linguistic structures. While completing the DCTs, the participants of the study were primarily preoccupied and obsessed with grammatical correctness. Through lengthy requests, they tried to avoid omissions and incomplete sentences. This could be due to the fact that the majority of instructional materials in the EFL contexts have mostly adopted a formal instructional approach to teaching pragmatics (Martinez-Flor, 2012). Furthermore, as Neizgoda and Roever (2001) asserted, EFL learners are very much concerned about their grammatical accuracy than their pragmatic appropriateness. For example, in the following examples derived from the collected data pool from WDCTs and ODCTs, it can be seen that respondents stuck to grammatical rules and used every element of a sentence without leaving out elements of the sentence:

(1)   Requester: Excuse me! Can I borrow your pen? I don't find my pen and I need a pen to fill in this form.

(2)   Requester: Excuse me! Can I can I borrow your pen?

As it can be seen in these examples (1 from WDCT and 2 from ODCT data pool), the respondents tried to make their requests grammatically correct. However, in most situations in a natural setting, they did not make full questions. As in the following example (3), the requester asked for the pen by leaving the sentence incomplete by taking advantage of real life context and pointing to the pen.

(3)   Requester: Ali?

Requestee: Just looking at the requester.

Requester: Give me your………

The respondents were required to imagine the intended contexts in WDCT and ODCT elicitation methods, and this artificiality might have been another probable reason for the distinction and marked differences between them. Not all respondents are equally well-endowed and well-equipped with imagination power (Schauer, 2009).

The scarce employment of exclamation particles in the WDCTs and ODCTs can be attributed to the very nature of these measures. Such measures and techniques usually do not incorporate a surprise element. Contrary to Natural interactions, the participants mostly regard DCTs as a test-like and formal activity (Sasaki, 1998). In fact, the nature of these tasks requires the respondents to display their metapragmatic knowledge rather than what they actually say or linguistic competence (Golato, 2003).

Regardless of some noticeable differences, requests elicited through ODCT and the ones gathered by the

 Natural method displayed somewhat similar cases of repetitions and inversions. This might be due to the similar mode of delivery of both methods. Brown (2001) regarded the delivery mode of ODCTs as one advantage of this type of pragmatic measurement. He further added that ODCTs encourage oral production. In spoken language, it is natural that speakers have the inclination to repeat some elements for reasons such as mind distraction or adding emphasis. In the same vein, to show emphasis, speakers usually invert elements to the initial position, while this rarely happens in the writing genre. In example number 2, the requester has repeated the phrase can I twice in the request.

 The findings of the current study, parallel to many other studies (e.g., Golato, 2003; Kasper, 2000; Turnbull, 2001; Woodfield, 2012), corroborate the fact that the data yielded by Role-play approximate the data generated through the Natural method. Role-play shares the same mode of delivery and turn-taking procedures with this method. This type of data collection procedure allows participants to take turns, ask and answer questions orally and interactively and is very much similar to what happens in natural interactions and this can be regarded as a positive advantage of the Role-play over other elicitation techniques (Brown, 2001). In example number 4, this advantage of Role-play is clearly evident which allows interactants to have interactions and take turns during their conversations. This can suitably justify why the data elicited through Role-play and gathered through the Natural method displayed similar patterns.

(4)   Requester: Excuse me!

Requestee: Yes

Requester: Your pen! I may use it!

Requestee: Sure.

 

Omitting and doing away with some surface structures and superficial linguistic elements can be considered part of the nature of the spoken genre, and the interlocutors heavily rely on contextual cues to deliver their intended meanings. Therefore, having constant access to the appropriate and proper context allows the learners to do away and dispense with some unnecessary and redundant parts which can be readily retrieved from the context. This might account for the fact that both methods displayed similar patterns of omission. Additionally, the inverted forms in the spoken genre happen more frequently than those the written genre, and that is why somewhat the same pattern of inversions could be observed in both Role-play and the Natural method. However, the differences between the Role-play and   the Natural method can be ascribed to the different nature of such measures as well. During natural interactions, there was no need to imagine the situation, whereas, during Role-plays, the role players were required to use their imagination power. The fact of the matter is this imagination power will have a direct bearing on the type of the data which Role-plays elicit because the role players are involved in pretending to be in real-life contexts and not all the participants are equally capable of creating imaginary and artificial contexts with the same degree of success (Schauer, 2009). Although the participants were engaged in real-life authentic interactions by watching the interlocutors and were able to benefit from their assistance in the flow of interaction and negotiation, it seems that the impact of the nature of the task was so evident that even the Role-players could not avoid making lengthy requests to display their deference.

 

6. Conclusion

The findings of the current study revealed that WDCT and ODCT potentially cannot, in terms of the number of repetitions, omissions, inversions, exclamation particles and the length of requests elicit the same data as the Natural method can. Role-play approximated the Natural method and born similarities of some included elements within the dependent variables.

The purpose of the present study was not to call into question the established pragmatic measures (WDCT, ODCT, and Role-play), but rather to pinpoint the similarities and differences of such measures to be employed for various purposes and across different contexts. Data collection methods should be chosen based on researchers' objectives and research questions (Yuan, 2001). In the WDCTs and ODCTs, for instance, the participants of this study cared more about surface pragmalinguistic features. Therefore, despite the lack of control over some discourse features such as paralinguistic and non-verbal elements, it could be claimed that DCTs lend themselves well to elicit pragmalinguistic and metapragmatic information (House, 2018; Martinez-Flor &Uso-Juan, 2011).

Some important points should be considered when interpreting the findings of the current study. Relying upon request speech act, the current study was conducted in an EFL context and the discourse of this study was institutional. Moreover, in recording naturally occurring data, the presence of the recorder in the classroom might have prompted students to show something other than their real self in their discourses.

The findings of this study can provide suitable support for teachers, practitioners, and educators to apply each measure in the right place and in the right time while accounting for different needs, purposes, and functions. Contextual information can also be another possible line of inquiry within this realm to be thoroughly probed into and deeply investigated. Schauer (2009) asserted that contextual information and cues play a pivotal role during the data collection process. Finally, it seems that the inclusion of matched modality tasks (Bardovi-Harlig, 2018), the adoption of mixed method approaches (House, 2018), and the combination of authentic and elicited data procedures (Portoles & Safont, 2018) are more promising areas of investigation for interested researchers to be able to properly tackle learners’ pragmatic competence.

مراجع

Alcon-Soler, E., & Safont, P. (2018). Editors' introduction to mixed methods approaches in investigating pragmatic learning, System, 75, 1-3.  doi: 10.1016/j.system.2018.03.013

Bardovi-Harlig, K. (2018). Matching modality in L2 pragmatics research design, System, 75, 13-22. doi: 10.1016/j.system.2018.03.007

Bardovi-Harlig, K. & Shin, S. (2014). Expanding traditional testing measures with tasks from L2 pragamtics research. Iranian Journal of Language Testing, 4 (1), 26-49.

Bardovi-Harlig, K. & Hartford, B. (2005) Institutional discourse and interlanguage pragmatics research. In K. Bardovi-Harlig & B. Hartford (Eds.), Interlanguage pragmatics: Exploring institutional talk (pp. 7-36). Mahwah, N.J.: Lawrence Erlbaum.

Brown, J. D. (2001). Pragmatics test: Different purposes, different tests. In R. Rose, & G. Kasper (Eds.), Pragmatics in language teaching (pp. 301-325). Cambridge: Cambridge university press.

Cohen, A. D. (2008). Teaching and assessing L2 pragmatics: What can we expect fromlearners? Language Teaching, 41(02), 213-235.

Economidou-Kogetsidis, M. (2013). Strategies, modification, and perspective in native speakers’ requests: A comparison of WDCT and naturally occurring requests. Journal of Pragmatics, 53, 21-38.

Ellis, R. (2008). The study of second language acquisition (2^nded.). Oxford: Oxford University Press.

Eslami, Z. R., & Mirzaei, A. (2014). Speech acts data collection in a non-western context: Oral and written DCTs in the Persian language. Iranian Journal of Language Testing, 4(1), 137-154.

Félix-Brasdefer, J.C. (2007). Natural speech vs. elicited data: A comparison of natural and role play requests in Mexican Spanish. Spanish in Context, 4(2), 159-185.

Golato, A. (2003). Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied Linguistics, 24(1), 90-121.

Hartford, B. & Bardovi-Harlig, K. (1992). Experimental and observational data in the study of interlanguage pragmatics. In L. Bouton and Y. Kachru (Eds.), Pragmatics and Language Learning, Monography 3 (pp. 33-520). Urbana-Champaign, IL: DEIL.

House, J. (2018). Authentic vs. elicited data and qualitative vs. quantitative research methods in pragmatics: Overcoming two non-fruitful dichotomies, System, 75, 4-12. doi: 10.1016/j.system.2018.03.014

Jalilifar, A. (2009). Request strategies: Cross-sectional study of Iranian EFL learners and Australian native speakers. English Language Teaching, 2(1), 46-61.

Kasper, G. (2000). Data collection in pragmatics research. In H. Spencery-Oatey (Eds.), Culturallyspeakin: Managing Rapport through Talk across Cultures (pp. 316-41). London and New York: Continuum.

Labov, W. (1972). Sociolinguistic patterns. Philadelphia: University of Philadelphia.

Martinez-Flor, A. (2012). Examining EFL learners’ long-term instructional effects when mitigating requests. In M. Economidou-Kogetsidis & H. Woodfield (Eds.), Interlanguage request modification (pp. 243-274). Amsterdam: John Benjamins Publishing Company.

Martínez-Flor, A. & Usó-Juan, E. (2011). Research methodologies in pragmatics: Eliciting refusals to requests. ELIA, 11, 47-87.

Nassaji, H. & Fotos, S. (2007). Issues in form-focused instruction and teacher education. In S. Fotos & H. Nassaji (Eds.), Form-focused instruction and teacher education: Studies in honor of Rod Ellis (pp. 7-15). Oxford: Oxford University Press.

Niezgoda, K., & Rover, C. (2001). Pragmatic and grammatical awareness. In K. R. Rose & G. Kasper (Eds.), Pragmatics in language teaching (pp. 63-79). Cambridge: Cambridge University Press.

Norris, J. & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta analysis. Language Learning, 50 (3), 417-528.

Portoles, L., & Safont, P. (2018). Examining authentic and elicited data from a multilingual perspective. The real picture of child requestive behavior in the L3 classroom, System, 75, 81-92.  doi: 10.1016/j.system.2018.03.012

Sasaki, M. (1998). Investigating EFL students’ production of speech acts: A comparison of production questionnaires and role plays. Journal of Pragmatics, 30, 457-484.

Schauer, G. A. (2009). Interlanguage pragmatic development: The study abroad context. London, UK: Continuum.

Taguchi, N. (Eds.). (2009). Pragmatic competence. Berlin/ New York: Mouton  deGruyter.

Taguchi, N. (2018). Data collection and analysis in developmental L2 pragmatics research: Discourse completion test, role play, and naturalistic recording. In A. Gudmestad & A. Edmonds (eds.), Critical reflections on data in second language acquisition, (pp. 7-32). Amsterdam, Netherlands: John Benjamins.

Turnbull, W. (2001). An appraisal of pragmatic elicitation techniques for the social psychological study of talk: The case of request refusals. Pragmatics, 11(1), 31-61.

Uso-Juan, E. & Martinez-Flor, A. (2014). Reorienting the assessment of the conventional expressions of complaining and apologizing: From the single response to interactive DCTs. Iranian Journal of Language Testing, 4(1), 113-136.

Woodfield, H. (2012). I think maybe I want to lend the notes from you: Development of request modification in graduate learners. In M. Economidou- Kogetsidis& H. Woodfield (Eds.), Interlanguage request modification (pp. 9- 49). Amsterdam: John Benjamins.

Yuan, Y. (2001). An inquiry into empirical pragmatic data gathering methods: Written DCTs, oral DCTs, and natural conversations. Journal of pragmatics, 33, 271-92.