Misplaced Pages

#789210

52-449: A rating is an evaluation or assessment of something, in terms of quality, quantity, or some combination of both. Rating or rating system may also refer to: Evaluation The primary purpose of evaluation, in addition to gaining insight into prior or existing initiatives, is to enable reflection and assist in the identification of future change. Evaluation is often used to characterize and appraise subjects of interest in

104-402: A common ideology entitled liberal democracy . Important principles of this ideology include freedom of choice, the uniqueness of the individual and empirical inquiry grounded in objectivity . He also contends that they are all based on subjectivist ethics, in which ethical conduct is based on the subjective or intuitive experience of an individual or group. One form of subjectivist ethics

156-543: A concept or proposal, project or organization, with the intention of improving the value or effectiveness of the proposal, project, or organization . It can also be summative , drawing lessons from a completed action or project or an organization at a later point in time or circumstance. Evaluation is inherently a theoretically informed approach (whether explicitly or not), and consequently any particular definition of evaluation would have been tailored to its context – the theory, needs, purpose, and methodology of

208-405: A full list of types of evaluations would be difficult to compile. This is because evaluation is not part of a unified theoretical framework, drawing on a number of disciplines, which include management and organizational theory , policy analysis , education , sociology , social anthropology , and social change . However, the strict adherence to a set of methodological assumptions may make

260-441: A lack of a definition of evaluation but are rather due to evaluators attempting to impose predisposed notions and definitions of evaluations on clients. The central reason for the poor utilization of evaluations is arguably due to the lack of tailoring of evaluations to suit the needs of the client, due to a predefined idea (or definition) of what an evaluation is rather than what the client needs are (House, 1980). The development of

312-481: A mass perspective. Seven true evaluation approaches are included. Two approaches, decision-oriented and policy studies, are based on an objectivist epistemology from an elite perspective. Consumer-oriented studies are based on an objectivist epistemology from a mass perspective. Two approaches—accreditation/certification and connoisseur studies—are based on a subjectivist epistemology from an elite perspective. Finally, adversary and client-centered studies are based on

364-665: A methodical approach is the scientific method , in which a person will produce a hypothesis based on what they believe to be true, then construct experiments in order to prove that hypothesis wrong. This method, when followed correctly, helps to prevent against circular reasoning and other fallacies which frequently plague conclusions within academia. Other disciplines, such as philosophy and mathematics, employ their own structures to ensure intellectual rigour. Each method requires close attention to criteria for logical consistency, as well as to all relevant evidence and possible differences of interpretation. At an institutional level, peer review

416-487: A particular assessment. General professional codes of conduct , as determined by the employing organization, usually cover three broad aspects of behavioral standards, and include inter- collegial relations (such as respect for diversity and privacy ), operational issues (due competence , documentation accuracy and appropriate use of resources), and conflicts of interest ( nepotism , accepting gifts and other kinds of favoritism). However, specific guidelines particular to

468-458: A point, some point is within an angle, and figures can be superimposed on each other). This was contrary to the idea of rigorous proof where all assumptions need to be stated and nothing can be left implicit. New foundations were developed using the axiomatic method to address this gap in rigour found in the Elements (e.g., Hilbert's axioms , Birkhoff's axioms , Tarski's axioms ). During

520-548: A principled approach; and intellectual rigour can seem to be defeated. This defines a judge 's problem with uncodified law . Codified law poses a different problem, of interpretation and adaptation of definite principles without losing the point; here applying the letter of the law, with all due rigour, may on occasion seem to undermine the principled approach . Mathematical rigour can apply to methods of mathematical proof and to methods of mathematical practice (thus relating to other interpretations of rigour). Mathematical rigour

572-420: A principled position from which to advance or argue. An opportunistic tendency to use any argument at hand is not very rigorous, although very common in politics , for example. Arguing one way one day, and another later, can be defended by casuistry , i.e. by saying the cases are different. In the legal context, for practical purposes, the facts of cases do always differ. Case law can therefore be at odds with

SECTION 10

#1732790778790

624-412: A project since each may have a different definition of 'merit'. The core of the problem is thus about defining what is of value." From this perspective, evaluation "is a contested term", as "evaluators" use the term evaluation to describe an assessment, or investigation of a program whilst others simply understand evaluation as being synonymous with applied research. There are two functions considering to

676-462: A situation or constraint either chosen or experienced passively. For example, the title of the book Theologia Moralis Inter Rigorem et Laxitatem Medi roughly translates as "mediating theological morality between rigour and laxness". The book details, for the clergy , situations in which they are obligated to follow church law exactly, and in which situations they can be more forgiving yet still considered moral. Rigor mortis translates directly as

728-469: A standard methodology for evaluation will require arriving at applicable ways of asking and stating the results of questions about ethics such as agent-principal, privacy, stakeholder definition, limited liability; and could-the-money-be-spent-more-wisely issues. Depending on the topic of interest, there are professional groups that review the quality and rigor of evaluation processes. Evaluating programs and projects, regarding their value and impact within

780-405: A standard of rigour, but are written in a mixture of symbolic and natural language. In this sense, written mathematical discourse is a prototype of formal proof. Often, a written proof is accepted as rigorous although it might not be formalised as yet. The reason often cited by mathematicians for writing informally is that completely formal proofs tend to be longer and more unwieldy, thereby obscuring

832-471: A subjectivist epistemology from a mass perspective. The following table is used to summarize each approach in terms of four attributes —organizer, purpose, strengths, and weaknesses. The organizer represents the main considerations or cues practitioners use to organize a study. The purpose represents the desired outcome for a study at a very general level. Strengths and weaknesses represent other attributes that should be considered when deciding whether to use

884-411: A topic or case is dealt with in a rigorous way, it typically means that it is dealt with in a comprehensive, thorough and complete way, leaving no room for inconsistencies. Scholarly method describes the different approaches or methods which may be taken to apply intellectual rigour on an institutional level to ensure the quality of information published. An example of intellectual rigour assisted by

936-403: A transparent, proportionate, and persuasive link between findings and recommendations. Thus evaluators are required to delimit their findings to evidence. A mechanism to ensure impartiality is external and internal review. Such review is required of significant (determined in terms of cost or sensitivity) evaluations. The review is based on quality of work and the degree to which a demonstrable link

988-400: A wide range of human enterprises, including the start of a period of months Evaluation is the structured interpretation and giving of meaning to predicted or actual impacts of proposals or results. It looks at original objectives, and at what is either predicted or what was accomplished and how it was accomplished. So evaluation can be formative , that is taking place during the development of

1040-471: Is utilitarian , in which " the good " is determined by what maximizes a single, explicit interpretation of happiness for society as a whole. Another form of subjectivist ethics is intuitionist / pluralist , in which no single interpretation of "the good" is assumed and such interpretations need not be explicitly stated nor justified. These ethical positions have corresponding epistemologies — philosophies for obtaining knowledge . The objectivist epistemology

1092-786: Is also an evaluation group within the OECD-DAC, which endeavors to improve development evaluation standards. The independent evaluation units of the major multinational development banks (MDBs) have also created the Evaluation Cooperation Group to strengthen the use of evaluation for greater MDB effectiveness and accountability, share lessons from MDB evaluations, and promote evaluation harmonization and collaboration. The word "evaluation" has various connotations for different people, raising issues related to this process that include; what type of evaluation should be conducted; why there should be an evaluation process and how

SECTION 20

#1732790778790

1144-977: Is and might be—they call this pseudo-evaluation . The questions orientation includes approaches that might or might not provide answers specifically related to the value of an object—they call this quasi -evaluation. The values orientation includes approaches primarily intended to determine the value of an object—they call this true evaluation. When the above concepts are considered simultaneously, fifteen evaluation approaches can be identified in terms of epistemology, major perspective (from House), and orientation. Two pseudo-evaluation approaches, politically controlled and public relations studies, are represented. They are based on an objectivist epistemology from an elite perspective. Six quasi-evaluation approaches use an objectivist epistemology. Five of them— experimental research, management information systems , testing programs, objectives-based studies, and content analysis —take an elite perspective. Accountability takes

1196-594: Is associated with the utilitarian ethic; in general, it is used to acquire knowledge that can be externally verified (intersubjective agreement) through publicly exposed methods and data . The subjectivist epistemology is associated with the intuitionist/pluralist ethic and is used to acquire new knowledge based on existing personal knowledge, as well as experiences that are (explicit) or are not (tacit) available for public inspection. House then divides each epistemological approach into two main political perspectives. Firstly, approaches can take an elite perspective, focusing on

1248-423: Is attained through ensuring independence of judgment is upheld such that evaluation conclusions are not influenced or pressured by another party, and avoidance of conflict of interest, such that the evaluator does not have a stake in a particular conclusion. Conflict of interest is at issue particularly where funding of evaluations is provided by particular bodies with a stake in conclusions of the evaluation, and this

1300-451: Is often cited as a kind of gold standard for mathematical proof . Its history traces back to Greek mathematics , especially to Euclid 's Elements . Until the 19th century, Euclid's Elements was seen as extremely rigorous and profound, but in the late 19th century, Hilbert (among others) realized that the work left certain assumptions implicit—assumptions that could not be proved from Euclid's Axioms (e.g. two circles can intersect in

1352-425: Is possible for a situation to be encountered, in which the process could not be considered advisable; for instance, in the event of a program being unpredictable, or unsound. This would include it lacking a consistent routine; or the concerned parties unable to reach an agreement regarding the purpose of the program. In addition, an influencer, or manager, refusing to incorporate relevant, important central issues within

1404-409: Is possible to doubt whether complete intellectual honesty exists—on the grounds that no one can entirely master his or her own presuppositions—without doubting that certain kinds of intellectual rigour are potentially available. The distinction certainly matters greatly in debate , if one wishes to say that an argument is flawed in its premises . The setting for intellectual rigour does tend to assume

1456-452: Is provided between findings and recommendations. Transparency requires that stakeholders are aware of the reason for the evaluation, the criteria by which evaluation occurs and the purposes to which the findings will be applied. Access to the evaluation document should be facilitated through findings being easily readable, with clear explanations of evaluation methodologies, approaches, sources of information, and costs incurred. Furthermore,

1508-855: Is released or withheld to meet the special interests of the holder, whereas public relations information creates a positive image of an object regardless of the actual situation. Despite the application of both studies in real scenarios, neither of these approaches is acceptable evaluation practice. As a group, these five approaches represent a highly respected collection of disciplined inquiry approaches. They are considered quasi-evaluation approaches because particular studies legitimately can focus only on questions of knowledge without addressing any questions of value. Such studies are, by definition, not evaluations. These approaches can produce characterizations without producing appraisals, although specific studies can produce both. Each of these approaches serves its intended purpose well. They are discussed roughly in order of

1560-448: Is required to be maintained against any pressures brought to bear on evaluators, for example, by project funders wishing to modify evaluations such that the project appears more effective than findings can verify. Impartiality pertains to findings being a fair and thorough assessment of strengths and weaknesses of a project or program. This requires taking due input from all stakeholders involved and findings presented without bias and with

1612-420: Is seen as potentially compromising the independence of the evaluator. Whilst it is acknowledged that evaluators may be familiar with agencies or projects that they are required to evaluate, independence requires that they not have been involved in the planning or implementation of the project. A declaration of interest should be made where any benefits or association with project are stated. Independence of judgment

Rating - Misplaced Pages Continue

1664-448: Is used to validate intellectual rigour. Intellectual rigour is a subset of intellectual honesty —a practice of thought in which ones convictions are kept in proportion to valid evidence . Intellectual honesty is an unbiased approach to the acquisition, analysis, and transmission of ideas. A person is being intellectually honest when he or she, knowing the truth, states that truth, regardless of outside social/environmental pressures. It

1716-555: The arithmetization of analysis . Starting in the 1870s, the term gradually came to be associated with Cantorian set theory . Mathematical rigour can be modelled as amenability to algorithmic proof checking . Indeed, with the aid of computers, it is possible to check some proofs mechanically. Formal rigour is the introduction of high degrees of completeness by means of a formal language where such proofs can be codified using set theories such as ZFC (see automated theorem proving ). Published mathematical arguments have to conform to

1768-415: The 19th century, the term "rigorous" began to be used to describe increasing levels of abstraction when dealing with calculus which eventually became known as mathematical analysis . The works of Cauchy added rigour to the older works of Euler and Gauss . The works of Riemann added rigour to the works of Cauchy. The works of Weierstrass added rigour to the works of Riemann, eventually culminating in

1820-527: The Joint Committee. They provide guidelines about basing value judgments on systematic inquiry, evaluator competence and integrity, respect for people, and regard for the general and public welfare. The American Evaluation Association has created a set of Guiding Principles for evaluators. The order of these principles does not imply priority among them; priority will vary by situation and evaluator role. The principles run as follows: Independence

1872-450: The approach for a particular study. The following narrative highlights differences between approaches grouped together. Politically controlled and public relations studies are based on an objectivist epistemology from an elite perspective. Although both of these approaches seek to misrepresent value interpretations about an object, they function differently from each other. Information obtained through politically controlled studies

1924-429: The context they are implemented, can be ethically challenging. Evaluators may encounter complex, culturally specific systems resistant to external evaluation. Furthermore, the project organization or other stakeholders may be invested in a particular evaluation outcome. Finally, evaluators themselves may encounter " conflict of interest (COI) " issues, or experience interference or pressure to present findings that support

1976-560: The evaluation There exist several conceptually distinct ways of thinking about, designing, and conducting evaluation efforts. Many of the evaluation approaches in use today make truly unique contributions to solving important problems, while others refine existing approaches in some way. Two classifications of evaluation approaches by House and Stufflebeam and Webster can be combined into a manageable number of approaches in terms of their unique and important underlying principles. House considers all major evaluation approaches to be based on

2028-503: The evaluation is integrated into a program, for the purpose of gaining greater knowledge and awareness? There are also various factors inherent in the evaluation process, for example; to critically examine influences within a program that involve the gathering and analyzing of relative information about a program. Michael Quinn Patton motivated the concept that the evaluation procedure should be directed towards: Founded on another perspective of evaluation by Thomson and Hoffman in 2003, it

2080-524: The evaluation process itself. Having said this, evaluation has been defined as: The main purpose of a program evaluation can be to "determine the quality of a program by formulating a judgment" Marthe Hurteau, Sylvain Houle, Stéphanie Mongiat (2009). An alternative view is that "projects, evaluators, and other stakeholders (including funders) will all have potentially different ideas about how best to evaluate

2132-424: The evaluation purpose. Formative Evaluations provide the information on improving a product or a process. Summative Evaluations provide information of short-term effectiveness or long-term impact for deciding the adoption of a product or process. Not all evaluations serve the same purpose some evaluations serve a monitoring function rather than focusing solely on measurable program outcomes or evaluation findings and

Rating - Misplaced Pages Continue

2184-456: The evaluator's role that can be utilized in the management of unique ethical challenges are required. The Joint Committee on Standards for Educational Evaluation has developed standards for program, personnel, and student evaluation. The Joint Committee standards are broken into four sections: Utility, Feasibility, Propriety, and Accuracy. Various European institutions have also prepared their own standards, more or less related to those produced by

2236-622: The extent to which they approach the objectivist ideal. Evaluation is methodologically diverse. Methods may be qualitative or quantitative , and include case studies , survey research , statistical analysis , model building, and many more such as: Rigour Rigour ( British English ) or rigor ( American English ; see spelling differences ) describes a condition of stiffness or strictness. These constraints may be environmentally imposed, such as "the rigours of famine "; logically imposed, such as mathematical proofs which must maintain consistent answers; or socially imposed, such as

2288-423: The field of evaluation more acceptable to a mainstream audience but this adherence will work towards preventing evaluators from developing new strategies for dealing with the myriad problems that programs face. It is claimed that only a minority of evaluation reports are used by the evaluand (client) (Data, 2006). One justification of this is that "when evaluation findings are challenged or utilization has failed, it

2340-494: The formalisation of proof does improve the mathematical rigour by disclosing gaps or flaws in informal written discourse. When the correctness of a proof is disputed, formalisation is a way to settle such a dispute as it helps to reduce misinterpretations or ambiguity. The role of mathematical rigour in relation to physics is twofold: Both aspects of mathematical rigour in physics have attracted considerable attention in philosophy of science (see, for example, ref. and ref. and

2392-411: The interests of managers and professionals; or they also can take a mass perspective, focusing on consumers and participatory approaches. Stufflebeam and Webster place approaches into one of three groups, according to their orientation toward the role of values and ethical consideration. The political orientation promotes a positive or negative view of an object regardless of what its value actually

2444-561: The international organizations such as the I.M.F. and the World Bank have independent evaluation functions. The various funds, programmes, and agencies of the United Nations has a mix of independent, semi-independent and self-evaluation functions, which have organized themselves as a system-wide UN Evaluation Group (UNEG), that works together to strengthen the function, and to establish UN norms and standards for evaluation. There

2496-570: The line of argument. An argument that appears obvious to human intuition may in fact require fairly long formal derivations from the axioms. A particularly well-known example is how in Principia Mathematica , Whitehead and Russell have to expend a number of lines of rather opaque effort in order to establish that, indeed, it is sensical to say: "1+1=2". In short, comprehensibility is favoured over formality in written discourse. Still, advocates of automated theorem provers may argue that

2548-479: The process of defining ethics and law . "Rigour" comes to English through old French (13th c., Modern French rigueur ) meaning "stiffness", which itself is based on the Latin rigorem (nominative rigor ) "numbness, stiffness, hardness, firmness; roughness, rudeness", from the verb rigere "to be stiff". The noun was frequently used to describe a condition of strictness or stiffness, which arises from

2600-421: The stiffness ( rigor ) of death ( mortis ), again describing a condition which arises from a certain constraint (death). Intellectual rigour is a process of thought which is consistent, does not contain self-contradiction, and takes into account the entire scope of available knowledge on the topic. It actively avoids logical fallacy . Furthermore, it requires a sceptical assessment of the available knowledge. If

2652-536: The works quoted therein). Rigour in the classroom is a hotly debated topic amongst educators. Even the semantic meaning of the word is contested. Generally speaking, classroom rigour consists of multi-faceted, challenging instruction and correct placement of the student. Students excelling in formal operational thought tend to excel in classes for gifted students. Students who have not reached that final stage of cognitive development , according to developmental psychologist Jean Piaget , can build upon those skills with

SECTION 50

#1732790778790

2704-454: Was because stakeholders and clients found the inferences weak or the warrants unconvincing" (Fournier and Smith, 1993). Some reasons for this situation may be the failure of the evaluator to establish a set of shared aims with the evaluand, or creating overly ambitious aims, as well as failing to compromise and incorporate the cultural differences of individuals and programs within the evaluation aims and process. None of these problems are due to

#789210