recurrent evaluation process:
•The evaluator meets with clients, staff, and audiences to gain a sense of their perspectives on and intentions regarding the evaluation.
•The evaluator draws on such discussions and the analysis of anydocuments to determine the scope of the evaluation project.
•The evaluator observes the program closely to get a sense of its operation and to note any unintended deviations from announced intents.
•The evaluator discovers the stated and real purposes of the project and the concerns that various audiences have about it and the evaluation.
•The evaluator identifies the issues and problems with which the evaluation should be concerned. For each issue and problem, the evaluator develops an evaluation design, specifying the kinds of data needed.
•The evaluator selects the means needed to acquire the data desired. Most often, the means will be human observers or judges.
•The evaluator implements the data-collection procedures.
•The evaluator organizes the information into themes and prepares “portrayals” that communicate in natural ways the thematic reports. The portrayals may involve videotapes, artifacts, case studies, or other “faithful representations.”
•By again being sensitive to the concerns of the stakeholders, the evaluator decides which audiences require which reports and chooses formats most appropriate for given audiences. (as cited by Glatthorn, 1987, pp. 275–276).
2.7.5. Eisner’s Model(1979)
Elliot Eisner (1979) drew from his background in aesthetics and art education in developing his “connoisseurship” model, an approach to evaluation that emphasizes qualitative appreciation. The Eisner model is built on two closely related constructs: connoisseurship and criticism. Connoisseurship, in Eisner’s terms, is the art of appreciation—recognizing and appreciating through perceptual memory, drawing from experience to appreciate what is significant. It is the ability both to perceive the particulars of educational life and to understand how those particulars form part of a classroom structure. Criticism, to Eisner, is the art of disclosing qualities of an entity that connoisseurship perceives. In such a disclosure, the educational critic is more likely to use what Eisner calls “nondiscursive”—a language that is metaphorical, connotative, and symbolic. It uses linguistic forms to present, rather than represent, conception or feeling.
Educational criticism, in Eisner’s formulation, has three aspects. The descriptive aspect is an attempt to characterize and portray the relevant qualities of educational life—the rules, the regularities, the underlying architecture. The interpretive aspect uses ideas from the social sciences to explore meanings and develop alternative explanations—to explicate social phenomena. The evaluative aspect makes judgments to improve the educational processes and provides grounds for the value choices made so that others might better disagree.
2.7.6. Oliva’s Model (1992)
Oliva’s (1992) curriculum modelconceptualized four main components – curriculum goals, curriculumobjectives, organization and implementation of the curriculum, andevaluation of the curriculum.Oliva (1992) points out that to consider students’ achievement intheir cognitive, affective and psychomotor learning as the effectiveness ofthe curriculum is not accurate. This is because, according to Oliva (1992), the primary purpose of curriculum evaluation is to determine whether thecurriculum goals and objectives are being successfully carried out or not. Inaddition, Oliva (1992) asserts that in the course of the instructional process,there are other questions curriculum planners would like to know, too. Questions suggested by Olivia (1992:479) are:
•whether the curriculum is functioning while in operation
• if the best material is being used and following the bestmethods
• whether the programs are cost-effective – whether we aregetting the most for the money spent
Once adeveloped curriculum is implemented in schools, appropriate evaluationprocedures shall be devised to examine the effectiveness of the curriculumin achieving the aims, goals and objectives of the curriculum.
Curriculum evaluation is an indispensable and central facet of any national education system. It provides accurate sources for curriculum policy decision makers, for constant curriculum adjustments and processes of curriculum completion. To do so, Dessler (1997) believed that curriculum evaluation is by and large based on participants’ conception of the courses, learning, behavior, results and assessment. Therefore, this study was to evaluate the success of translation studies at the level of M.A. regarding instructors and students conceptions based on the evaluation model provided by Tyler (1949).
This chapter illustrated the general design of the current research including participants, data collection instruments, and data collection procedure and data analysis. In this evaluation study, both qualitative and quantitative data were collected. Qualitative data were collected via interviews with the instructors. Also, 4 researcher-designed questionnaires based on Tyler model were employed to collect data from both students and instructors. Descriptive and inferential statistics were used to analyze the data collected through the questionnaires.
Curriculum Evaluation can be either a small-scale task involving a very limited number of participants if it is classroom based, or a massive large-scale task involving a number of universities, professionals and some community members. The participants of this study were 120 students graduated in M.A in English Translation from 2009-2013 and 30 instructors from both State and Azad universities of Iran including Allameh Tabatabaei university, Islamic Azad university of central and south Tehran, Science and Research Branch of Azad university in Tehran, and Ferdowsi university of Mashhad. They were between 23 and 35 years old, and all of them were native speakers of Persian. Fifty students were male and seventy students were female. Having in-depth findings and information around the topic and supplementary analysis(McNamara,1999), 10 instructors out of 30 were interviewed in their own rooms and they truly contributed to the study by giving their opinions with regard to the program implemented based on given questionnaire.
The success of any study highly depends on how the samples were selected and the different population components were represented in the samples (Elder, 2009). Among the four most commonly used sampling methods, which are simple random sampling, systematic sampling, stratified sampling, and cluster sampling, the researcher used simple random sampling because it best fits the purpose of the present study. In simple random sampling, every subject has an equal chance of being selected for the study. The subjects were selected randomly from both State and Azad universities. It should be mentioned that the random selection was among graduated students of English Translation at M.A. level. Since gender of the participants was not considered as a moderator variable, there was no control for sex variable. In the same way, the selection of instructors for conducting interview was done randomly from both State and Azad universities.
Questionnaires and interview with instructors were applied to collect data in this study. To check the validity and appropriateness of students and instructors’ questionnaires, comments were gathered from 6 university instructors. In addition to this, an attempt was made to make sure the face validity of students
and instructors’ questionnaire items. Accordingly 25 students and 6 instructors were concerned to complete this job. The purpose of conducting the face validity was to ensure that every item is appropriate to the target groups, the questions or items are not vague, and instructions are clear, questions or items do not yield ineffectual data to remove them and finally to be attentive to likely problems which may occur in the process of collecting data. The findings of this activity were found to be highly significant in the process of designing the final version of the questionnaires and making preparation for the main study.
The data collection instrument was designed by the current researcher. The researcher made all her attempts to carry out a comprehensive investigation about the problematic points through the study of the respective papers, books, journals and M.A. theses and even doctorial dissertations conducted not only in Iran but also in abroad. Accordingly, regarding the relevant obtained literature and the model of evaluation provided by Tyler (1949), the researcher designed a self-reported questionnaire consisting of 4 sections (See Appendix ). In fact, considering the research questions of the study, these questionnaires were provided. Along with gathering students’ demographic information, the questionnaire designed by the researcher based on Tyler model (1949) Basic Principles of Curriculum and Instruction was used purposely to find out the students and instructors’ perception of the following fundamental criteria:
1. Students and instructors’ conceptions of the instructional objectives of the current English Translation M.A. Program. It consisted of five point scale items. The values ranged from 1-5 indicating 5 for strongly disagree, 4 for agree, 3 for uncertain, 2 for agree and 1 for strongly agree.
2. Students and instructors’ conception of compulsory and optional
courses in terms of allocated time and credits of the M.A. English
Translation Curriculum. It consisted of five point scale items. The
values ranged from 1-5 indicating 5 for completely insufficient, 4
for not sufficient, 3 for somehow sufficient, 2 for sufficient and 1
for quite sufficient.
3. Students and instructors conception of the arrangement of the courses. It consisted of five point scale items. The values ranged from 1-5 indicating 5 for strongly disagree, 4 for agree, 3 for uncertain, 2 for agree and 1 for strongly agree.
4. Students and instructors’ conception of assessment. It consisted of five point scale items. The values ranged from 1-5 indicating 5 for strongly disagree, 4 for agree, 3 for uncertain, 2 for agree and 1 for strongly agree.
Interview with selected instructors was undertaken person-to-person on various issues put forth by questionnaires. The major advantage of the interview was its flexibility in controlling the response situation, setting up a communally convenient time and place, and controlling the succession and pacing of the questions asked. On the other hand, the Interview helped the current researcher look deeply into instructors’ beliefs, attitudes and inner experiences by following up with questions to obtain more information that might not have been revealed by using other data collection instruments.
3.4. Data Collection Procedure
Among the variety of methods of data collection, the researcher selected the one which correspond with research questions. In order to investigate the research questions, multiple choices questionnaire and an open-ended interview in single and/or multiple sessions used to collect the required data from the students and instructors.
As Nunan (1993) states there can be “disparities between what teachers believe happens in class and what actually happens”. This study was both qualitative and quantitative research. It consisted of two rounds. In the first round, multiple choices questionnaire was used to collect the required data from both students and instructors.
The questionnaires were administered to 120 students and 30 instructors in