American English File Series Evaluation Based on Littlejohn’s Evaluative Framework

International Journal of Foreign Language Teaching and Research

مقاله 10، دوره 1، شماره 1، خرداد 2013، صفحه 106-120   

نوع مقاله: Research Papers

نویسندگان

Hamid Reza Haghverdi¹؛ Behnam Ghasemi²

¹Assistant Professor Islamic Azad University, Khorasgan Branch, Isfahan, Iran

²M.A. Student, Islamic Azad University, Khorasgan Branch, Isfahan, Iran

چکیده

Textbooks play a pivotal role in language learning classrooms. The problem is, among a wide range of textbooks available, which one is more appropriate for a specific classroom and a group of learners. In order to evaluate ELT textbooks, theorists and writers have offered different kinds of evaluative frameworks based on a number of principles and criteria. This study evaluates one example of such a series of ELT textbooks, namely, “American English File” using Littlejohn’s (1998) evaluative framework to see what explicit features of the book are, and what pedagogic values it has. Littlejohn believes that we should evaluate a textbook based on its own pedagogic values and we should see what is in it not what the teacher and evaluators think must exist in it. Consequently his framework is claimed to be devoid of any impressionistic ideas and it is in-depth and objective rather than being subjective. Nine ELT experts and ten ELT teachers helped the researcher rate the evaluative checklists. The results of the study show that although a number of shortcomings and drawbacks were found in American English File, it stood up reasonably well to a detailed and in-depth analysis and that its pedagogic values and positive attributes far out-weighed its shortcomings. The internal consistency between ratings was computed via the statistical tool of Cronbach’s alpha that indicated a desirable inter-rater reliability

کلیدواژه‌ها

ELT textbook؛ Littlejohn’s framework؛ American English File series؛ evaluation

اصل مقاله

Textbooks are really crucial in today’s language teaching and learning. Although there might be some disagreements on using textbooks in ELT classes, most of the people dealing with education, teaching, and learning substantiate the importance of textbooks in ELT classes. For instance, according to Toms (2004), supporters of text books argue that it is the most effective way of presenting materials, it helps to reach a format and skeleton through which learners will be able to achieve a sense of system, cohesion and progress, and teacher preparation. Moreover, the selection of a particular core textbook signals an educational decision in which there is considerable professional, financial, and even political investment (Sheldon, 1988). Ansary & Babaii (2002) list the following arguments for the textbooks:

• a textbook is a framework which regulates and times the programs,
• in the eyes of learners, no textbook means no purpose,
• without a textbook, learners think their learning is not taken seriously,
• in many situations, a textbook can serve as a syllabus,
• a textbook provides ready-made teaching texts and learning tasks,
• a textbook is a cheap way of providing learning materials,
• a learner without a textbook is out of focus and teacher-dependent, and perhaps of  utmost importance and,
• for novice teachers, a textbook means security, guidance, and support.

 

Graves (2007) lists the following as some of the advantages of using a textbook:

1. It provides a syllabus for the course
2. It provides security for the students because they have a kind of road map of the course
3. It provides a set of visual, activities, readings, etc., and so saves the teacher time in finding or developing such materials
4. It provides teachers with a basis for assessing students’ learning

In the literature of textbook evaluation, many different schemes and checklists have been offered by different writers and evaluators. Azizifar, et al (2010) examined an evaluation of two series of ELT textbooks used for teaching English language in Iranian high schools from 1965 to the present. For this purpose, Tucker’s (1975) textbook evaluation model was employed. The results suggested that one of the main factors for the students’ achievement in English language is the ELT textbooks. The researchers suggested that in the textbooks, there should be enough opportunity for the learners to practice the language they are learning communicatively.

In this vein, this paper is concerned with carrying out an evaluation on a series of ELT materials, namely American English File using Littlejohn’s framework (1998). I have chosen Littlejohn’s framework in my paper because he believes what is required is a framework which separates assumptions about what is suitable from an analysis of the materials. His framework is new and seeks to evaluate the selected textbook irrespective of how it is used in the classroom. He mentions it as analyzing the materials as it is, a set of materials can be used quite differently in different contexts based on the knowledge, abilities, and preferences of different teachers. This framework, thus, is claimed to be devoid of impressionistic criteria about what is desirable in a set of materials. By analyzing the individual activities (tasks) in detail and by studying the important features of the book, this framework is claimed to be in-depth and objective rather than subjective.

 

Research Questions

This study will try to answer the following questions with reference to American English File series:

1. What are the explicit features of American English File series?

2. What pedagogic values does American English File series have?

 

Methodology

 

Selected Framework

This study will theoretically be based on Littlejohn’s (1998) framework. As he claims, the framework tries to be in-depth and objective rather than subjective by analyzing the individual activities in detail and based on important features. Thus, he has considered following questions as the basis of his framework:

1. What aspects of materials should we examine?
2. How can we examine the materials?
3. How can we relate our findings to our own teaching context?

In this part Littlejohn states that his framework consists of two main sections namely publication and design to focus on “methodology” of the material and their “context”. As he puts it, publication relates to the “tangible” or physical aspects of the materials and how they appear as a complete set or book. The second section in the framework, design (following Richards and Rodgers proposal 1986) relates to the thinking underlying the materials. Here we deal with such issues as the aims, principles of selection, principle of sequencing, subject matter and focus of subject matter in which they draw on the learner’s process competence (knowledge, affect, abilities, skills)), participation (who does what with whom), learner roles, teacher roles and finally role of materials as a whole.

    

Participants

Participants of this study consisted of 19 persons including nine ELT experts and ten EFL teachers. ELT experts were asked to help in this study to complete task analysis sheet (TAS) checklist. Teachers were asked to work on one of the checklists developed for evaluating the design of selected textbook. Each of these teachers had at least one year experience of teaching textbooks under analysis.

 

Materials

The materials of the study were American English File series, a five-level English course for adults and young adults. 12% of the total materials was extracted as a sample. Littlejohn argues that “I have found it is useful to analyze about 10% to 15% of the total material”. (p. 196). This sample included five units, the middle unit of each book. Each unit of American English File series is composed of four sections plus two-page practical English and writing, and a two-page review & check section. The sample contained 522 tasks. Each task was labeled by a number that was applied in task analysis sheets.

 

Instruments

A checklist was developed in order to examine the explicit features of American English File series. This checklist was based on Littlejohn’s framework (Appendix A).

There was another checklist based on Littlejohn’s framework (Appendix B) under the title of Task Analysis Sheet. This checklist examines the activities and tasks in one typical unit of each textbook based on what the learner is expected to do, with whom, and with what content. Each section has also several subsections.

The first section examines the materials according to three aspects of turn take (initiate, response, and/or not required); focus on (language system, meaning, and/or meaning-system relationship); and mental operation (retrieve form long-term memory, select information, draw on prior knowledge, relate sound to objects, compare, etc.)

In the second section three possible situations are checked: learner to class, learner individually simultaneously, and learners in pairs/groups.

The third section is also divided to input to learner, expected output from learners, source, and nature.

There was also a third checklist (Appendix C) developed based on claims declared by the authors of American English File series and Richards and Rodgers notion of design, as is mentioned in Littlejohn’s framework, to seek participants’ ideas about 30 statements. These statements were selected based on the authors’ claims and then categorized according to nine sections of design part in Littlejohn’s framework.

 

Data Collection

The checklist developed to examine the explicit features of American English File series was worked on by the researcher himself because as Littlejohn puts it, this checklist is dealt with tangible or physical aspects of the materials and there will not be any disagreement among different evaluators about the results provided by this checklist.

The second checklist (TAS) was offered to nine ELT experts as raters. It is worth mentioning here that this checklist, proposed by Littlejohn, is developed in a way that can be worked on by even one rater. Here in order to enhance the reliability of evaluation these nine experts were asked to help to evaluate the materials. Due to the wide range of tasks it was impossible to expect each rater analyzes all 522 tasks lonely. Therefore, the tasks were divided to three categories each category involved 174 tasks which was given to three ELT experts; altogether there were nine raters and 522 tasks. Each group of three raters was provided with a sample of 174 tasks and checklists to work on. Of course the organization, materials, and sequence of activities in American English File series textbooks are in such a way that they repeat in every unit. In this regard there seemed not to be any critical differences considering the organization and sequence of activities between total amount of materials and our sample.

The third checklist then was handed to ten teachers with experience in teaching the series. Teachers helped us at this part because of their experience of teaching these books they had an idea about the whole material, not just the sample.

 

Data Analysis

For the first checklist, a report on explicit features of American English File series is provided in the results section. Regarding TAS checklist after analyzing each expert’s ratings, using statistical procedures and SPSS, frequency counts and percentage indexes were reported for individual features listed in task analysis sheets. In addition, in order to examine the internal consistency between the nine ELT experts’ ratings, Cronbach’s alpha was utilized and indicated the inter-rater reliability is desirable between ratings (0.89).

The third checklist (Design) was analyzed in order to find an agreed view on each of the sections mentioned in the Littlejohn’s framework. Also the inter-rater reliability between the ten EFL teachers’ ratings was computed through Cronbach’s alpha and showed a desirable consistency (0.87).

 

Results

 

The first part of the checklist deals with Turn-take. Turn-take, as Littlejohn (1998) puts it, refers to the kind of participation which the learner should have when accomplishing the learning task. When a learner participates in a learning task, he/she may initiate using the language i.e. he/she is expected to express what he/she wishes to express without a script of any kind, or he/she may just respond i.e. the kind of language he/she is expected is narrowly defined. Besides, the learner may not be expected to use language at all, for instance, he/she may just be demanded to listen to the listening in this case no initiation and no response is expected from the learner.

As Table 3.1 and Figure 3.1 show, the learners were expected to “respond” to nearly half (53.65%) of the tasks in American English File. Rests of the tasks include 32.15% “initiation” and14.3% “the tasks that do not require learners to initiate or respond”. These results show that the American English File tasks encourage students to use the language and more importantly they often require them to express themselves rather than be a listener. 

 

Table 3.1. Frequency and percentage for Turn-Take

Turn take	Frequency	Percentage
Initiate	538	32.15%
Respond	896	53.55%
Not required	239	14.3%
Total	1673	100%

 

Figure 3.1.Turn-Take

This section of TAS, examines the tasks to see where the learner is expected to concentrate his/her attention when he/she participates in learning tasks presented by American English File. Based on the presented task by the textbook the learner may be required to focus on “language system” i.e. the rules or form of the language, “the meaning” i.e. the message of the language being used, or “the meaning-system relationship” i.e. the relationship between language form and its meaning.

According to Table 3.2 and Figure 3.2, it can be concluded that tasks in American English File mainly focus on meaning considering that about half of the tasks just focus on meaning and 27.36% of them focus on both form and meaning. This can also be concluded that American English File tries to enhance comprehension by using the tasks that draw students’ attention more to the meaning of the language than its form.

 

Table 3.2. Frequency and percentage for Focus

Focus on	Frequency	Percentage
Language System (rules and form)	436	25.44%
Meaning	809	47.2%
Meaning/ System relationship	469	27.36%
Total	1714	100%

 

 

Figure 3.2. Focus

 

In this section of TAS, ten items of mental operation were examined, namely “Retrieve from long term memory” i.e. a mental operation through which the learner is expected to recall items from previous units to the present task, “Build text” by which the learner produces rather a long stretch of spoken or written discourse (more than 50 words), “Draw on prior knowledge” in which the learner needs to draw on his/her prior linguistic knowledge in order to carry out a task, “Relate sounds to objects” by the use of which the learners listen and then point to the mentioned object or mark an object in their textbook, “Compare” via which the learners compare two or more sets of language data on the basis of meaning or form, “Decode semantic meaning” in which the learners are expected to deduce the surface meaning of the given language or what the text proposes, “Select information” through which the learners extract pieces of information from a text especially to answer reading comprehension questions, “Repeat with expansion” in which the learners are provided by an outline to be used as a frame to produce lengthier sentences, “Deduce language rules” in which the learners are supposed to infer language rules based on the ample examples provided to him/her, and finally “Apply language rule” in which the student is given the language rule and is demanded to produce a piece of language or respond according to the proper use of the rule.

As Table 3.3 and Figure 3.3 show, “Decode semantic meaning” is used slightly more than other operations (17.1%). It can be seen from the table that American English File provides learners with activities that give them the opportunity to infer the meaning of the texts. American English File pays less attention to repetition with expansion that reveals the fact that it does not require students to learn English through a lot of repetition.

 

Table 3.3. Frequency and percentage for Mental Operation

Mental operation	Frequency	Percentage
Retrieve from LT memory	333	13.3%
Built text	215	8.6%
Draw on prior knowledge	309	12.4%
Relate sounds to objects	126	5.1%
Compare	158	6.3%
Decode semantic meaning	428	17.1%
Select information	233	9.3%
Repeat with expansion	118	4.7%
Deduce language rule	210	8.4%
Apply language rule	372	14.8%
Total	2502	100%

 

 

Figure 3.3.  Mental Operation

 

This section of TAS, seeks to find the kind of interaction occurring between the students and class or teacher while they are accomplishing a task. Three kinds of interaction are considered in this part as “Learner to class” in which one student is supposed to give his/her reports to the others in class, “Learners individually simultaneously” in which each individual student performs the required task but not in collaboration with other learners, and “Learners in pairs/groups” in which the learners are required to interact with each other in pairs or groups in order to carry out the task.

As can be seen in Table 3.4 and Figure 3.4, activities which require students to accomplish a task individually simultaneously compose 52.4% of total tasks in the series. In conclusion we can see that the authors of American English File in spite of requiring learners to work in pairs in the process of learning have paid much attention to tasks that involve learners into activities that should be done individually simultaneously.

 

Table 3.4. Frequency and percentage for Who with?

Who with	Frequency	Percentage
Learner to class	420	25.47%
Learner individually simultaneously	864	52.4%
Learner in pairs/groups	365	22.13%
Total	1649	100%

 

 

Figure 3.4. Who with?

 

 

This part examines the channel and/or the length of the input provided for the learners. The input is whether in the form of “Graphic” e.g. pictures, illustrations, diagrams, etc., “Oral words/phrases”, “Oral extended discourse”, “Written words/phrases”, “Written extended discourse”, or “Sound/music”.

As Table 3.5 and Figure 3.5 show, written words/phrases have dominant source of input (31.80%) while graphic source of input which is as important as writing in encouraging and motivating learners has the least proportion of the whole tasks.

 

Table 3.5.  Frequency and percentage for Input to learners

Input to learners	Frequency	Percentage
Graphic	208	10.10%
Oral word/phrases	336	16.30%
Oral extended discourse	259	12.50%
Written word/phrases	658	31.80%
Written extended discourse	315	15.20%
Sound/music	290	14.10%
Total	2066	100%

 

 

Figure 3.5. Input to learners

 

This kind of expected output from learners can be either written or oral. In other words it can be in words or phrases length or in extended length form. In Littlejohn’s framework discourse which is more than 50 words is considered as an extended form.

As the table 3.6 and figure 3.6 show, the most attention has been paid to oral words and phrases (40.70%) while written extended discourse has the least proportion of output in the tasks (11.10%). In conclusion, we can understand that the series pays more attention to oral form of output which is used in every day conversations and communications. For the written form of output it mostly focuses on structures at words and phrases level rather than extended form.

 

Table 3.6. Frequency and percentage for Expected output from learners

Expected output from learners	Frequency	Percentage
Oral word/phrases	718	40.70%
Oral extended discourse	370	21.00%
Written word/phrases	479	27.20%
Written extended discourse	196	11.10%
Total	1763	100%

 

 

 

Figure 3.6. Expected output from learners

 

TAS examines three possible sources of the content for lessons and their activities. The content of lessons is provided whether by the materials, by the learners, or by the teachers. As table 3.7 and figure 3.7 show the textbook provides a great amount of materials and contents for lessons itself (64.10%). It seems the series demands teachers to be abided by their guide book and other components.  

 

 

Table 3.7. Frequency and percentage for Source

Source	Frequency	Percentage
Materials	1081	64.10%
Teacher	117	6.90%
learner	489	29.00%
Total	1687	100%

 

 

Figure 3.7. Source

 

In this section of analysis, the type of content which is the focus of the learning activity is studied. The content which the learners and the teachers are required to work with may have different natures. It could demand personal opinion or it may require involvement of facts, fictions, or personal information. Some activities also may need metalinguistic knowledge of learners for being done.

As table 3.8 and figure 3.8 show, Fact is more frequent in total tasks (40.20%), but fiction has the least frequency in nature tasks (3.70%). In fact, in the series it is endeavored to design tasks in a way that expose students in authentic contexts through using their own information an opinions.

Table 3.8. Frequency and percentage for Nature

Nature	Frequency	Percentage
Personal opinion	317	16.40%
Fact	776	40.20%
Fiction	71	3.70%
Personal information	393	20.40%
Metalinguistic knowledge	372	19.30%
Total	1929	100%

 

 

Figure 3.8. Nature

Results

 

The second section in Littlejohn’s framework, Design, relates to the thinking underlying the materials. This part involves consideration of areas such as the apparent aims of the materials, how the tasks, language and content in the materials are selected and sequenced. The results of the study of the design can help evaluators to see to what extent the materials developers have been successful in achieving their intended goals.

As Table 3.9 and Figure 3.9 show, the selected textbook obtained 60% of the optimum score (90 out of 150) for aims and objectives part. Principles of selection acquired 58% of the optimum score (87 out of 200). Principle of sequencing received 41.6% of optimum score (83.2 out of 150). The obtained score for subject matter and focus of subject matter was 62% of the optimum score (144 out of 200) which seem to be rather high. Types of teaching/learning activities obtained 32.4% of optimum scores (81 out of 250). Participation received 35.2% of optimum scores (52.8 out of 150). The next score was obtained by classroom roles of teachers and learners which was 40.4% of optimum proportion (80.8 out of 200). Learner role in learning obtained 46% of optimum scores (92 out of 200) and finally the role of materials as a whole received 41.6% of optimum scores (124.8 out of 300). In conclusion, it can be seen that the scores of the statements related to the authors’ claims are high which can be a sign of success for the authors of the series.

 

Table 3.9. Total score and proportion percentage for Design

Design	Total	Percentage
1.Aims & Objectives	90	60%
2.Principles of selection	87	58%
3. Principle of sequence	83.2	41.6%
4. Subject matter and focus of subject matter	144	62%
5. Types of teaching/ learning activities	81	32.4%
6. Participation	52.8	35.2%
7. Classroom roles of teachers and learners	80.8	40.4%
8. Learner roles in learning	92	46%
9. Role of the materials as a whole	124.8	41.6%

 

 

Figure 3.9. Design

 

Discussion

 

In this study Littlejohn’s (1998) framework was applied. By using this framework, the researcher attempted to answer to the following questions:

Question one: What are the explicit features of American English File series?

As mentioned before, there is no disagreement among researchers and evaluators about the explicit features of a textbook because, as Littlejohn stated, they are related to the tangible or physical aspects of the materials and how they appear as a complete set or book. Thus the explicit features of American English File are those mentioned in result section based on the results of applying Littlejohn’s checklists.

Question Two: What pedagogical value does American English File series have?

American English File’s pedagogical values are as follow:

I. According to the results of the study (Table 3.1) it was revealed that the tasks in American English File series more often encourage the students to use the language and focuses more on involvement of the learners in the classroom activities.

II. The results of this study (Table 3.2) revealed that American English File series mostly tries to draw on meaning as the basis for the learning task which is adjusted with the authors’ claims.

III. Based on the results of the study (Table 3.3) it is revealed that American English File provides students with tasks that give the opportunity to infer the meaning of the texts that accompany applying language rule, recalling previous learning, and using prior linguistic knowledge which can be related and applied in new tasks and activities. These characteristics give the textbook more consistency and continuity.

IV. According to (Table 3.4) American English File pays more attention to tasks that involves learners into activities that should be done individually simultaneously rather than group activities.

V. The study also shows that (Table 3.5) American English File has paid more attention to written word/phrases form of input.

VI. One of the pedagogic values of the series according to the results of this study (Table 3.6) is its attention to the oral word/phrases form of expected output from the learners.

VII. The results of the study also revealed that (Table 3.7) for the majority of tasks and activities the textbook itself specifies its own texts as the source of content. It contains systematic practice of practical language such as: how to ask for directions, how to request services at a hotel, airport etc. Furthermore, American English File provides content to help students develop a cultural fluency by creating and awareness of the varied rules across cultures for issues like politeness, greetings and introductions, etc.

VIII. According to the results of this study (Table 3.8) it was understood that fact is more frequent in total tasks while fiction has the least frequency in nature tasks. The tasks in the series are of various natures. Metalinguistic knowledge helps learners arrive at knowledge of forms structure and other aspects of language through reflecting and analyzing the language.

 

Conclusions

 

Although a number of shortcomings and drawbacks were found in American English File series, the results of the study seemed to reveal that this particular ELT textbook stood up reasonably well to a detailed and in-depth analysis and that its pedagogic values and positive attributes far outweighed the demerits. Both experts and teachers evaluation results showed that American English File series are in line with the goals set by its authors. Of course teachers of English language who teach the series are suggested to consider the shortcomings and try to alleviate or compensate for these drawbacks by supplementing, modifying and adapting problematic aspects of the textbook.

 

Implications

The findings of this study can incorporate in practical areas of second or foreign language learning and teaching and also material production. English teachers are expected to benefit from this study because it gives them an idea about the characteristics, positive attributes and shortcomings of the textbook they may want to teach. Publishers, authors and materials developers can also benefit from this study because they could consider the merits and demerits of the textbook under analysis to develop new materials with more positive attributes.

 

Limitations of the Study

There are different kinds of frameworks for textbook evaluation. Each of these frameworks examines the selected textbook based on a number of theories and assumptions. Some analyze a textbook to see to what extent pragmatical consideration of materials presentation is applied in it. Others may attempt to examine the critical aspects of content presentation in a given book such as gender representation. As such, this study cannot claim to be exhaustive and the researcher cannot assert that every possible aspect of the selected textbook has been analyzed. There are still a host of other issues to be studied in American English File series.

 

Suggestions for Further Study

Different aspects of American English File series are open to further evaluation and analysis. Supplementary study is needed to extract psycholinguistic analysis of the American English File series. Pragmatic aspects of the content of this book can also be examined in a study. Since there are a lot of ELT textbooks in the market, some evaluations can also be carried out on other widely used textbooks.

 

مراجع

Azizfar, A., Kusha, M., & Lotfi, A. R. (2010). An analytical evaluation of Iranian high school ELT textbooks from 1970 to the present. Procedia Social and Behavioral Sciences, 3, 36–44.

Ansary, H., & Babaii, E. (2002). Universal characteristics of EFL/ESL textbook: A step towards systematic textbook evaluation. The Internet TESL Journal, 2, 1-8.

Graves, K. (2007). Designing Language Courses. Canada: Newbury House.

Littlejohn, A. (1998). The analysis of Language teaching materials: Inside the Trojan horse. In Tomlinson, B. (Ed.), Materials development in Language teaching (pp. 190-216). Cambridge: Cambridge University Press.

Richards, J.C. (1986). The role of textbooks in a language program. Retrieved December 10, 2004 from: http://www.professorjackrichards.com/pdfs/role-of-textbooks.

Sheldon, L. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42(4), 237-246.

Tom, C. (2004). General English coursebooks and their place in an ESAP program. ELT journal 6(1), Article 9.

Tucker, C. (1975). Evaluating beginning textbooks. English Teaching Forum, 13, 335-361.    

 

 

 

 

 

 

Appendices

Appendix A: Checklist for explicit nature of a set of materials

Title:

Publisher:

BOOK AS A WHOLE

1. Type:

   2. Intended audience

   3. Extent

    a. Components

     b. Total estimated time

   4. Design and Layout

   5. Distribution

     a. Material

      b. Access

   6. Route through material

   7. Subdivision

B. OVERVIEW OF AN EXTRACT

   1. Length

   2. Sequence of activity