Analyzing the Readability Level of the English Textbooks Used in Jordan

Research in English Language Pedagogy

مقاله 6، دوره 8، شماره 2 - شماره پیاپی 15، آذر 2020، صفحه 334-348   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.30486/relp.2020.1895290.1191

نویسندگان

Rami Abu^* ؛ Abdullah Khataybeh

Department of Curriculum and Instruction, Yarmouk University, Jordan

چکیده

Because of the significance of the textbooks in the teaching and learning processes, it is essential to revise them permanently. This study investigated the readability level of the 12^th- grade English textbooks and also examined if the reading texts used in Action Pack 12 were suitable linguistically for the 12^th -grade students. This study adopted the qualitative descriptive design by using the Fry Graph to calculate the readability level. The researchers chose randomly 20 texts out of 36 texts as a sample of the study. The results revealed that 80% out of the 20 texts were under the level of the 12^th-grade students in Jordan. Also, the results indicated that 15 % of the texts were invalid and only 5% of the texts were within the 12^th-grade students’ level. The findings also indicated that Action Pack 12 was suitable linguistically for 12^th-grade students. Several recommendations are put forth.

کلیدواژه‌ها

Action Pack؛ English textbooks؛ Fry Graph؛ Grade 12؛ Jordan؛ Readability

اصل مقاله

1. Introduction

There is no doubt that the written texts are essential because they can enter the reader in a state of pleasure and knowledge. This process constitutes the most significant burden in designing and constructing the school curriculum, which aims to build the learner’s personality, through logical dialogue between the students and the book reaching to an improvement in their cognitive construction (Abu Sa'aleek, 2018).

Hence, the textbook is crucial because it is considered as one of the essential sources of knowledge. Many students today derive their knowledge and information from the textbook. It is also a source of learning, research and entertainment if it is built on pedagogical implications, and it is contained useful educational material interestingly and legibly (Al-Khalidi, 2013).

Designing textbooks in a way that suits students’ levels and abilities is a vital characteristic of these books because they are a fundamental resource for both teacher and student (Burns, 2006; Ivey, 2010).

Moreover, Eisner (1994) confirmed that the school textbook is the primary tool for the teaching and learning process. He also points out the importance of the textbook and states that the textbook provides an appropriate level of content and organizes the content logically on some topics in order to ensure the arrangement of the material for educational goals. Also, the textbook provides students with a kind of safety because it controls the process of learning, and it presents questions to the students in order to answer them.

The textbook used in English as a Foreign Language (EFL) plays an essential role in presenting materials that are beneficial for teachers and students in the educational process. The textbook has an advanced rank in EFL contexts.

The textbook is developed when the analysis and evaluation processes taking into account in any new textbooks. Evaluating and analyzing the existing textbooks provides an excellent base and direction for the development of new teaching materials when they are presented to students (Lee, 2013).

The textbook is one of the foremost necessary components that can be used in learning. Therefore, it should usually be written, organized, and substantially edited (Pamungkas, 2010). Also, the textbook in the teaching and learning processes can be seen as an indispensable guide for teachers and students (Hornby, 1995). The use of the textbook is required in the course of any language because it provides a principle to follow in a systematic way (Ur, 1996). Course books defined as a way to achieve the aims and objectives that take into account learner’s need (Cunningsworth, 1984). The importance of the textbook is evident in the learning process. So, it is required to evaluate it in order to detect the weaknesses and strengths of it (Tanner & Tanner, 1998).

The Jordanian Ministry of Education (JMoE) has given a noticeable interest in the textbooks that are used in teaching students in public schools and permanently attempted to develop them in both primary and secondary stages. In the Jordanian context, English is taught as a foreign language starting from the first to twelfth grade. In grade 12, students are taught English four times a week through Action Pack series, which is the official textbook for teaching English for grade 12. This book is a new edition with new topics started teaching it since 2015. Therefore, it is significant to evaluate the texts used in Action Pack 12 because they have played a role in students’ academic achievement and their motivation towards learning English.

This study tends to be significant because it, to the researchers’ best knowledge, is the first study in Jordan that evaluated the readability level of the textbook Action Pack 12, namely in its new edition. Also, it stems its significant from introducing to the other researchers and educators how to evaluate the texts via using the Fry formula. Optimistically, the results in this study may encourage the JMoE to revise the textbooks that are used in teaching the English language through the use of the Fry formula.

 

2. Literature Review

The term ‘readability’ used in evaluating and analyzing the linguistic level of communication materials. Readability is as a result of interaction between the reader and the material. There is no one specific definition of readability among the researchers, but they agree that readability is suitable for evaluating the reading material in terms of ease or difficulty of the text for the different age group. (Al-Awamleh, Al-Swailemeen, & Al-Sheik, 2010).

According to Richards, Platt and Platt (1992), readability means “how easily written materials can be read and understood. This process depends on several factors, including the average length of sentences, the number of new words contained, and the grammatical complexity of the language used in a passage” (p. 306). Also, McLaughlin (1969) defined readability as to how the students have natural reading materials. In another definition, Dale and Chall (1949) defined the readability as the whole communications of each one of those components in a given part of the written word that influences the readers’ achievement.

The readability formula is a logical way to predict the readability level of the texts (Kondur, 2006). Many different formulas can be used within the first and second language context. Such formulas may include the Flesch Reading Ease formula, the Fog Scale, the Fry Graph, the Flesch-Kincaid Grade Level, the Gunning Index, and the Gunning-Fog Index. In the current study, the Fry Graph is used in to identify the readability level of 12^th-grade students’ textbook Action Pack in Jordan. The researchers choose the Fry Graph scale as it is simple to use and reliable, as well as take a short time to count the score.

The Fry Graph has become known in 1968 as a useful effective tool (Gunning, 2003).  Edward Fry produced the formula in 1968 while he was working as an EFL teacher (DuBay, 2007). It is considered the best tool that has been applicable and easy to use with grades 1-17 (Crook, 1977; Fry, 1968; Paolo, 1977). The graph was first used to determine the readability for high schools, and later it as used in other grades (e.g., primary grade). The Fry Graph was used through selected three 100-word passages from a book. To get the results, the average number of sentences is put on the graph in order to determine the grade level. This graph is focused on sentences within the analyzed passage (DuBay, 2007).

There are a plethora of studies that carried out to analyze and evaluate the readability level. In this regard, Gyawaji, Saddhono, and Sulistyo (2017) aimed to analyze the readability of the 22 texts in the two tenth grade textbooks in Indonesia. The Fog index was used to calculate the readability level. The results showed that the two textbooks of the 10^th- grade students were easy to understand, which means that the texts were suitable for grade 10.

Nurhamsih (2017) examined the readability level of the English reading texts in Indonesian textbooks for grade 9. Thirty-eight reading texts were analyzed by using Raygor Readability Estimate. The results showed that the textbooks were not suitable linguistically for the third-grade students because the readability level of the textbooks were 11. Most of the readability level was over level 9. Namely, 26 reading texts were not linguistically suitable for the students.  Moreover, among 38 reading texts, there was only one reading text that its readability level is invalid. This text was not considered as a readable text. It means that this text cannot be read and learned.

In the Jordanian context, Khataybeh and Sakal (2017) analyzed the readability level of reading texts in the English textbook Action Pack Series for the seventh, ninth and 10^th grade. Different instruments were used in collecting the data, such as Fry Graph, Smog Formula and Flesch Chain. The results revealed that these instruments cannot determine the level of the texts whether these texts were suitable for grade 7, 9, and 10 or not.

Rohmatillah (2015) conducted a study to measure the readability level of reading texts in the English textbook for 10^th grade. The researcher used content analysis to collect the required data through the use of the Flesch readability formula. The findings of the analysis revealed that five texts from sixteen texts were relevant to the students at grade ten which means that the reading texts are ideal for the tenth grade.

Gutierrez (2014) evaluated the readability level of the English and Filipino languages for 548 passages from elementary and high textbooks. The instruments used in the study were Fry and Smog readability formula. The results showed that the analyzed passages in the English and Filipino languages did not suit for students. The results also showed that most of the passages were higher than the reader’s level according to the Smog formula.

Begeny and Greene (2014) evaluated the level of reading passages to determine the readability of the reading passages and whether these passages were readable for students. Findings showed that the readability level was not fit to students’ ability in reading.

Isnaeni (2013) carried out a study to evaluate the readability level of English written Indonesia materials. The descriptive quantitative approach was adopted by the researcher to achieve the purpose of the study. The Fog index used to analyze whether the English written materials, mainly the passages, were readable or not. The findings of the study indicated that written materials were at the natural level based on the Fog index.

 Gallagher, Fazio, and Gunning (2012) evaluated the reading levels for the elementary science students. Different formulas were used to determine the level of the text. The finding of the study showed that science texts were fit to the students’ level.

Geçit (2010) analyzed the readability of geography textbooks in Turkey. To collect the data, the Fog, Flesch, and Smog are used to determine the readability level of the textbook, and if these books are appropriate for grade 9 and 11. The findings of the study indicated that these books were not suitable for students based on Fog and Smog formulas, and were not suitable based on Close and Flesh tests.

Chiang, Englebrecht, Phillips, and Wang (2008) analyzed the readability of the seven accounting textbooks in Iran. Flesch Reading, Fog, and the Smog procedures were used. The findings showed that the readability level among the textbooks differs, but readability within textbooks was generally consistent.

It seems from the previous studies that most studies conducted to measure the readability in different textbooks. Different readability formulas were used, such as Fog, SMOG, Flesch, Raygor, and Fry Graph. This study seeks to analyze the readability level of English secondary textbook based on the Fry Graph. This study tries to investigate the readability level of textbooks for grade 12. Studies (e.g. Chiang et al., 2008; Geçit, 2010; Gyawaji et al., 2017; Isnaeni, 2013) used the Fog formula to identify the readability level. Also, studies (Khataybeh & Sakal, 2017; Rohmatillah, 2015) used the Flesch readability formula. The Fry formula was used in studies of (e.g. Gutierrez, 2014 ;  Khataybeh & Sakal, 2017).

This study is different from the previous studies because it evaluated the readability level of the English secondary textbooks in Jordan, and it uses the Fry formula to identify the readability level of these books. The present study is like the study of Khataybeh and Sakal (2017), which investigated the readability level of English books of Action Pack series for grade 7, 9, and 10. The study of Khataybeh and Sakal (2017) and the present study used the Fry formula to identify the readability level of the English texts book used in teaching English for the Jordanian students.

Action Pack 12 is published to teach the 12^th -grade students in the secondary stage. It is a new edition that the Jordanian Ministry of Education has developed in 2015/2016. Measuring the readability of textbooks derives from the importance of textbook in the educational process for both student and teacher. This readability provides the teachers and educators of what is status of the textbook which is taught now for the students in the secondary stage (i.e. grade 12). The readability studies have significant importance for publishers which can be used to identify the characteristics of the language based on the reader’s stage. Based on the previous reasons, it is important to measure the readability level of the texts used in Action Pack 12. Therefore, this study seeks to investigate the readability level of 12th grades’ textbook in Jordan. The researchers aim to answer the following questions:

1. What is the readability level of the textbook Action Pack 12 for the 12^th grade in Jordan?
2. Are the reading texts used in Action Pack 12 suitable linguistically for the 12^th-grade students?

 

3. Methodology

3.1. Design and Context of the Study

Mainly, the descriptive quantitative design of the study was used to establish the readability level of the two English secondary textbooks for the 12^th grade in Jordan. The Fry Graph (1968) was used as an instrument for the study to measure the readability level of the selected texts.

 

3.2. Population and Sample of the Study

The population of this study was based on the English language textbook for grade 12 used in the academic year 2017/2018. This book consisted of 10 units with 36 texts used in it. Several reading texts, totaling 20 texts were chosen randomly as a sample of the study. These texts were taken from the two textbooks: Action Pack 12, i.e., Student’s Book and Activity Book. The distribution of texts within the two textbooks is in Table 1. 

 

Table1.

Distribution of the Texts in Action Pack 12

No.	Textbooks	Number of Texts
1	Student’s Book	26
2	Activity Book	10
Total		36

 

The sample of this research was 20 texts out of the total population of the 36 texts. The researchers analyzed 20 texts out of 36 (56%) from the two textbooks used by the teacher to teach English to 12^th grade. These texts were chosen and subjected to Fry readability formulae, which also served as the significant evaluation instruments of the study. The sample texts were selected randomly to use for the Fry Graph. These were encoded, and the internet-based programs of Fry were run to determine the readability level of the texts. The results later tabulated and compared with the target level of the books.

 

 

3.3. Instrument

The instrument of the current research was the Fry Graph Readability Formula used for collecting data. This formula was used to identify the level of the reading texts. Fry Graph was one of the most popular readability formulas that are used as a graph. It was suitable for all ages from early reader to college. There were some directions for the Fry Graph formula as follows:

a. Select three samples of the text, but this study is selected 20 samples. Each sample is more than 100 words.

b. The number of syllables for each sample should calculate to get the average.

c. The number of sentences for each sample should calculate to get the average.

e. After the average of syllables and sentences are calculated, plot the average on the graph to get the readability level.

f. When the average number of sentences and syllables met, this indicates the grade reading level of the text. 

 

 

 

Figure 1. Fry Graph Scale

 

 

3.4. Data Collection Procedures

To establish the validity of the procedures used in this study, a group of experts reviewed the procedures used by the researchers. Their comments were taken into consideration before beginning the process of analyzing. To find out the reliability, the researchers first calculated the reliability by using the following two methods: First, the researchers analyzed the choosing texts as a first analysis. After two weeks, another analysis of the same texts was made to compare the two results of the analysis. The finding showed that there was a high agreement between the two analysis in total 0. 88. This result means that this result accepted for the study. Second, 10 texts were chosen randomly to analyze them by another analyst after training them how to deal with the Fry Graph Scale in order to be able to interpret its results. The results showed that there is a high agreement between the analysis reaching to 0.90. These results confirmed that the results of the analysis were reliable. To check the validity of the Fry Graph scale, it was handed out to a jury of experts to judge on its validity. They said that the Fry Graph reliability has a high validity that can be used to analyze the texts in EFL contexts. The researchers asked them to analyze a sample of the texts to identify the readability of the texts. They should analyze the texts based on the following link http://www.readabilityformulas.com/free-fry-graph-test.php. The researchers also asked them to copy at least 100 words and paste it on the link to determine the readability level.

 

3.5. Data Analysis Procedures

The sample of the study consisted of 20 out of 36 texts.  All these texts were evaluated to identify the readability level of the 12^th grade. The selected texts were downloaded and analyzed by using the following link http://www.readabilityformulas.com/ free-fry-graph-test.php.

This website was based on the number of sentences and the number of syllables to analyze the texts. The downloaded texts should be at least 100 words of a text to run our readability assessment test. Different steps have been taken to evaluate the texts. The first step is to choose a sample of 100 words of a text. The second step is to count the number of sentences in the text, estimating the fraction of the last sentence to the nearest one tenth. The third step is to count the number of syllables in the text. The fourth step is to enter the graph with an average sentence length and number of syllables. Plot dot where the two lines intersect.

To interpret the readability level, the researchers used the following scale:

1. Easy if the score points out a grade, below grade 12. This means that the text is easy for students.
2. Meet the Grade if the score points out grade 12. This means that the texts meet 12^th -grade students.
3. Difficult if the score out a grade above grade 12. This means that the analyzed texts are difficult. 
4.  Invalid if the score points out the dark area on the Fry Graph (long sentence and word areas).

 

4. Results

Finding of the study indicated that 16 texts out of 20 are in the easy classification. It means that 80% of 20 texts are easy for 12^th-grade students who learn the English language for twelve years. They spoke the Arabic language as a mother tongue. They learn English the language as a foreign language as one of the primary school subjects in the educational Jordanian context. Moreover, they learn the language four times a week by using the two books (i.e. Student’s Book and Activity Book).

It seems that there are 16 texts in the range of reading grade level. Table 2 also implies that there is one text that meets the students level. It means that 5% of the texts meet the demanded reading grade level for twelfth-grade students. In other words, there is one text on the reading grade level 12. Table 2 also implies that three texts are determined to be invalid texts. It means that 15% of the texts were invalid for twelfth-grade students. In other words, there were three texts did not have a reading grade level, according to the Fry readability formula. Results are presented in Table 2.

 

 Table 2

Readability Level of the Reading Texts in Action Pack 12

No.	Text Title	Number of Sentences	Number of Syllables	Readability Level
1	The history of computers	7.6	156	9
2	Using technology in class	6.2	165	12
3	Complementary medicine	4.9	181	Invalid
4	Get Moving!	5.2	151	9
5	The King Hussein Cancer Center	6	155	10
6	Accident victim tests first artificial limb	5.8	142	8
7	Masdar City	6.4	181	Invalid
8	A founding father of farming	4.8	147	9
9	The arts in Jordan	4.4	154	11
10	The time we spend at school	4.9	153	10
11	After school	8.3	144	7
12	Learning a foreign language	5	174	Invalid
13	Learn English fast	6.7	141	7
14	Speaking with signs	5.6	155	10
15	What are they talking about?	6.9	151	9
16	Doing business in China	9	143	7
17	How to make a sales pitch?	5.6	139	8
18	My job as an interpreter	8	147	7
19	Stepping into the business world	8.1	146	7
20	Curriculum Vitae	7.4	155	9
	Total	126.8	3080
	Average	6.34	154	9

 

 

 

First, dealing with the number of sentences, the reading text entitled (After school) get the highest number of sentences with 8.3 sentences. In contrast, the lowest text is the reading text entitled (The arts in Jordan) with only 4.4 sentences. The average number of all sentences is 6.3.4

Second, dealing with the number of syllables, two reading texts have the most syllables, which are entitled (Complementary medicine) and (Masdar City). The number of syllables of them is 181. Conversely, the text with the lowest number of syllables is the one entitled (How to make a sales pitch?) with 139 syllables. The average number of syllables is 154.

Third, there is one reading text which gets the highest readability level. Its readability level is at level 12. The lowest one is in level 7. However, there are three reading texts that their readability level is in an invalid level.

 

5. Discussion

Overall, the results of the research indicate that 80% out of 20 texts (16 texts) is not difficult to be read by the 12^th -grade students. Besides, there are only 15% of the texts (three texts) possess invalid the reading grade level. This means that those texts (i.e. three texts) are not decided if they are appropriate for grade 12.  Moreover, only 5% of the texts (one text) is within the level of grade 12. This result points out that only one text touches the students’ level. The findings in this study are consistent with studies of (e.g., Gyawaji et al., 2017; Nurhamsih, 2017), which show that the reading texts are easy and understanding for the students in grade 12. Based on the findings, the 16 reading texts are under grade 12. Linguistically, those texts are too easy for the 12^th -grade students according to the Fry Graph formula. It is claimed that the best readability level for adults is located between the level 8-12.

The findings also indicated that there is only one reading text, whose readability level is 12. The suitable readability level for the 12^th grade is 12 because most of the students have been studying English for more than 11 years. Also, the researchers may interpret this result to the fact that the text which entitled (Using technology in class) has contained contemporary issues (e.g., social media) that have been taken place in students’ lives today. It can be concluded that one reading text is suitable linguistically for grade 12. This text is considered as a readable text for students who are in grade 12.

Thus, based on the findings of the study, the researchers found that three reading texts are in invalid level. Based on the Fry Readability formula, these reading texts are not considered as readable texts. This means that the students may get difficulties in understanding these texts.

The findings of the present study are consistent with the findings of studies of (Gallagher et al., 2012; Gyawaji et al., 2017; Isnaeni, 2013) which showed that the analyzed texts are suitable for students’ level. In contrast, the findings of the present study are inconsistent with the findings of studies of (Begeny & Greene, 2014; Chiang et al., 2008; Geçit, 2010; Gutierrez, 2014; Nurhamsih, 2017; Rohmatillah, 2015),  which showed that the textbooks that have been analyzed in their study are not suitable for students’ level.

 

6. Conclusion

Several conclusions can draw from the findings. First, the readability of the textbook of Action Pack 12 is nine. This means the textbook is suitable linguistically for the 12^th-grade students. This result means that the 12^th -grade students should not find any difficulty in dealing with the texts which are taught for them in this grade. This result also means that there is no difficulty level of the texts that are taught for grade 12. Second, there are 16 reading texts that their readability level are below level 12. Third, there is one reading text that is considered as readable text with the readability level 12. Fourth, among 20 reading texts, there are only three reading texts that their readability level is in an invalid level. This text is not considered a readable text. This means that this text cannot be read and learned by students in grade 12.

There are some implications of these results. First, the texts which are identified as a readable grade level 12 should use in teaching reading for the students in grade 12 in Jordan. Second, the texts which are less than grade 12 should be improved and revised. Third, teachers may find or develop other texts and analyze them with readability measures to get appropriate texts that can be used for grade 12. Based on the findings proposed in this study, some recommendations are presented:

1. Using the readability within the specifications of textbooks.
2. English textbook publishers should be more aware of the readability texts in order to take into account students’ needs and their abilities.

مراجع

Abu Sa’aleek, R. A. (2018). The involvement of the twelfth grade English textbook in Jordan: An analytical study. Indonesian Research Journal in Education, 2(2), 75-85. https://doi.org/10.22437/irje.v2i2.5909.

Al-Awamleh, A., Al-Swailemeen, M., & Al-Sheikh, A. (2010). Readability level of science textbook among the 7th-grade students in Jordan. Journal of Islamic University, 18(2), 805-823.

Al-Khalidi, J. (2013). Readability level of Islamic education textbooks in Jordan. Journal of Al-Azhar University – Gaza, Humanities, 15(1), 1-22.

 Begeny, J.C.,& Greene, D.J. (2014). Can readability formulas be used to successfully gauge the difficulty of reading materials? Psychology in the Schools, 51, 198-215.

Burns, B. (2006). I don't have to count syllables on my fingers anymore: Easier ways to find readability and level books. Illinois Reading Council Journal,  34(1), 34-40.

Chiang, W., Englebrecht, T., Phillips, T., & Wang, Y. (2008). Readability of financial accounting principles textbooks. The Accounting Educators’ Journal, 18, 47-80.

Crook, N. M.  (1977). A Study of the validity of the Fry readability graph. Journal of Social Studies Research, 1, 53-59.

Cunningsworth, A. (1984). Evaluating and selecting EFL teaching materials. London: Heinemann.

Dale, E., & Chall,  J. S. (1949). The concept of readability. Elementary English, 26(1), 19-26.

DuBay, W. (2007). Smart language: Readers, readability, and the grading of text. Charleston, SC: BookSurge Publishing.

Eisner, E. W. (1994). The educational imagination. New York: Macmillan Publishing Co. Inc.

Fry, E. (1968). A readability formula that saves time. Journal of Reading, 11, 513-516, 575-578.

Gallagher R. L., Fazio X., & Gunning, T. G. (2012). Varying readability of science-based text in elementary readers: Challenges for teachers. Reading Improvement, 49, 92-112.

Geçit, Y. (2010). The evaluation of high school geography 9 and high school geography 11 text books with some formulas of readability. Educational Sciences: Theory & Practice,  10(4), 2205-2220.

Gunning, T. G. (2003). The role of readability in todayʼs classrooms. Topics in Language Disorders, 23(3), 175-189.

Gutierrez, M. (2014). The suitability of the fry and smog readability formulae in determining the readability of Filipino texts. Normal Lights, 8(1), 35-47.

Gyawaji, F., Saddhono, K., & Sulistyo, E.(2017). Fog index on textbooks of Indonesian subject for class X of senior high school in standard based curriculum (Sbc) and curriculum 2013 (A Study of Legibility and Feasibility Textbooks). Proceeding of 2nd International Conference of Arts Language And Culture, 511-522.

Hornby, A. S. (1995). Oxford advanced learner's dictionary of current English. Oxford University Press.

Isnaeni, N. (2013). Readability of English written materials. Journal of English and Literature, 1(1), 179-192.

Ivey, G. (2010). To create lifelong readers, we need to give students reading materials that leave them wanting to know more. Educational Leadership, 67(6), 18-23.

Khataybeh, A., &  Sakal, R. (2017). The readability of Action Pack for 7th, 10th and 9th grades. Journal of Education, Society and Behavioural Science, 22(2), 1-13.

Kondur, J. (2006). Using part of speech structure of the text in the prediction of its readability [Unpublished master’s thesis]. University of Texas, Arlington, U.S.

Lee, S. (2013). The development of evaluation theories for foreign language textbooks. Journal of Pan-Pacific Association of Applied Linguistics, 17(2), 69-89.

McLaughlin, G. H. (1969). SMOG-grading: A new readability formula. Journal of Reading, 12(8), 639-646.

Nurhamsih, Y. (2017). The analysis of the readability levels of the reading texts in textbooks entitled fast tract to English for the third year students of SMA based on raygor readability estimate. International Journal of English Language and Teaching, 1(1), 50-75.

Pamungkas, D. A. (2010). The quality of the English textbook used by international standard Junior High school [Unpublished master’s thesis]. The State University of Malang.

Paolo, M. F. (1977). A comparison of readability graph scores and oral reading errors on trade books for beginning reading [Unpublished master’s thesis]. Rutgers University, New Brunswick, New Jersey.

Pelteret, C., Kilbey, L., & Greet, J. (2015). Action Pack 12 twelfth grade: Student’s book. London: York Press.

Pelteret, C., Kilbey, L., & Greet, J. (2015). Action Pack 12 twelfth grade: Activity book. London: York Press.

Richards, J. C.,  Platt, J., & Platt, H. (1992). Longman dictionary of language teaching and applied linguistics. London: Longman.

Rohmatillah, R. (2015). Readability level of reading texts in the English textbook entitled English alive for senior high school grade X. English education. Tadris English, 7(1), 81-104.

Tanner, D., & Tanner, L. N. (1998). Curriculum development theory. New York: Macmillan Book Company.

Ur, P. (1996). A course in language teaching: Practice and theory. Cambridge University Press.