suggested a checklist that was designed to evaluate coursebooks which are employed in ESL programs in a local setting. The checklist consists of two sections: Practical Considerations and Language Related Considerations. The former concentrates on the accessibility of the coursebooks for teachers and learners, layout’s clarity, physical features of the coursebook, and cultural elements, whether it is culture-biased or not. On the other hand, Language Related Considerations are the demonstration of skills based on the language and cognitive demonstration of language, which is if there is authenticity, variety, recycling and sequencing in the language, presentation of exercises concerning the balance between free and controlled exercises, whether they promote communication in a meaningful way and allow for negotiation, and finally the clarity of user definition” (p. 4).He also stated that
It is necessary for teachers to be well-equipped with the skills to evaluate materials to ensure that students are using the highest quality texts possible and that their language learning experience is enhanced, not hindered, by the books used in their classrooms (p.5).
Three criteria are also offered by Ellis and Ellis (1987) in the assessment of EFL textbooks which are relevance, accessibility and cohesion. They stated that finding out relevance in the area of coursebook development has to concern “the signposts, (headlines, titles, photographs, etc.) audience, (availability of sufficient variety of design to intent the learner, the level of cartoon. and photograph for the learner, the level of density and variety of text for the learner), color, and mimesis” (p. 92).
On the other hand, accessibility has to be investigated in relation to the clarity of the reading providing that “accessible material will have a clear reading path, possess obvious quality of production both in text presentation and layout and in choice and use of visual support” (Ellis & Ellis, 1987, p. 94). In addition, the quality of a coursebook development is considered. Thirdly, cohesion is taken into account with visual demonstration of the content. It is offered that overall coherence has to gain various recognizable patterns. Therefore, Ellis and Ellis (1987) drew attention to the points such as “uniformity of page allocations to unit, use of color, typographical and design conventions adopted within the materials” (p. 97).
Hutchinson and Waters (1987) also proposed a model for coursebook evaluation. In their model, they claimed that coursebook assessment is a matching process that consists of four steps: defining criteria, subjective analysis, objective analysis and matching. They developed a checklist of criteria for objective analysis that is the analysis of coursebooks to be evaluated and subjective analysis that is the analysis of specific teaching/learning situations in relation to coursebooks’ requirement. The checklist consists of items which are related to the audience, aims, content, methodology, and other criteria. The first section of checklist includes questions which are related to learners’ information like their gender, age, nationality, study/work specialism, language background and interests. The next section consists of some questions about the objectives of the coursebook. The third section includes questions related to language materials to be covered, skills, text types, subject matter, organization of content, and the sequence of content. The fourth section examines methodology which consists of questions about learning theory, learners’ attitudes, tasks and exercises, aids, and flexibility. The last section involves questions about costs and accessibility of coursebooks.
Turkish Ministry of National Education, Department of Educational Research and Development (1993) developed a material design model, which includes criteria for material design. The main criteria are as follow:
1) Relevancy of the material to the needs and interests of the learners.
2) Adequacy of the material to meet and support the development of the objectives
3) Sequence and continuity in the materials and the link between the material and the students’ subject of study.
4) Contribution of the material in encouraging the learners to gain different points of views.
5) Appropriateness of the time specified in the material.
6) Clarity of instructions.
7) Opportunities for self-evaluation (cited in Yumuk, 1998, p. 68)
As mentioned earlier, the most common method of conducting a coursebook evaluation is making use of an appropriate checklist that is developed by well-known scholars. The next part deals with checklists and introduces the checklist that has been used in the present study.
While the subject of coursebook evaluation has not been extensively investigated, several authors have offered methods of assisting teachers to be more accurate in their evaluation by demonstrating evaluation checklists based on criteria which can be used by teachers and learners in various situations. Soori, Kafipour, and Soury (2011) defined checklist as an instrument which provides the evaluator with a list of features of successful teaching/learning materials.
Checklists are quantitative or qualitative. Quantitative checklists allow an objective assessment of a given coursebook through Likert style rating scales (Skierso, 1991).On the other hand, in qualitative checklists, open-ended questions are used to get subjective information about textbooks’ quality. Qualitative scales have capability of an in-depth assessment of coursebooks, but quantitative ones are considered more reliable and convenient to work with.
Several scholars have conducted a detailed investigation of a textbook’s language content that has managed to the production of large amounts of evaluation checklists. One of these checklists is Cunningsworth’s (1984) checklist in which he touches upon the significance of relating coursebooks to course goals and the learners’ needs and processes. Another one is Sheldon’s (1988) checklist which includes a large variety of components and tries to evaluate all aspects of content such as graphics and physical characteristics to authenticity and flexibility.
In spite of Sheldon’s (1988) suggestion that no list of criteria can really be used in all ESL/EFL settings without modification, most of these checklists include similar parts which can be applied as useful starting points for ELT teachers in various situations. Several scholars in the field of coursebook design like Sheldon (1988), Brown (1995), and Cunningsworth (1995) all assented that evaluation checklists ought to have some criteria related to the physical features of coursebook like layout, organizational, and logistical features as well. Other significant criteria which should be included are the criteria to evaluate a coursebook’s methodology, objectives, approaches and the extent to which particular materials are not only teachable, but also are appropriate for both teacher’s approach and organization’s curriculum. In addition, criteria have to analyze the particular language functions, grammar, and skills which are covered by a specific coursebook and the relevance of linguistic elements to the socio-cultural context. Finally, coursebook evaluations have to contain criteria which are related to presentation of culture and gender as well as the degree to which the linguistic items, subjects, content, and topics fit the learner’s personalities, backgrounds, needs, and interests as well as those of the teacher and/or institution.
A review of the literature indicates that most evaluation checklists have common characteristics. For example, Skierso’s (1991) checklist contains features pertaining to bibliographical data, aims and goals, subject matter, vocabulary and structures, exercises and activities, and layout and physical makeup. These characteristics are mostly consistent with those in Cunningsworth’s (1995) checklist which considers aims and approaches, design and organization, language content, skills, topic, methodology, and practical considerations. Even though the topic of the parts in the two checklists seem different, an investigation of the items will display that they are approximately similar. For instance, Skierso (1991) considers the cost-effectiveness of the coursebook in the section named ‘bibliographical data’, while Cunningsworth (1995) refers to it in the ‘practical consideration’ section. Daoud and Celce-Murcia (1979) proposed a checklist for coursebook evaluation which includes five main parts: subject matter, vocabulary and structures, exercises, illustrations, and physical make-up. Every part consists of detailed strategies that can be used in assessing and analyzing every coursebook.
The checklist used in the present study was a coursebook evaluation checklist which was primarily designed by Cunningsworth (1995) and specifically modified for this study. Cunningsworth (1995) stated that since “different criteria will apply in different circumstances” (p. 2), teachers and researchers ought to identify their own priorities and develop their own checklists. The checklist used in this study consists of fifty three criteria in fourteen sections; content, grammar, vocabulary, phonology, language skills, methodology, study skills, visuals, practice and testing, supplementary material, objectives, content selection, gradation, and culture. Moreover, the checklist has a four-point multiple-choice Likert scale format.
2.4 Studies on coursebook evaluation
Many studies have been conducted on coursebook evaluation in different settings. Different coursebook evaluation checklists have been employed by different researchers to assess different coursebooks. For instance, Morgan (2003) assessed IELTS preparation books and indicated that these books do not give the learners enough knowledge which is