Program evaluation - Wikipedia. Program evaluation is a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs. In both the public and private sectors, stakeholders often want to know whether the programs they are funding, implementing, voting for, receiving or objecting to are producing the intended effect. While program evaluation first focuses around this definition, important considerations often include how much the program costs per participant, how the program could be improved, whether the program is worthwhile, whether there are better alternatives, if there are unintended outcomes, and whether the program goals are appropriate and useful. However, planned social evaluation has been documented as dating as far back as 2. BC. People who do program evaluation come from many different backgrounds, such as sociology, psychology, economics, social work, and public policy. Some graduate schools also have specific training programs for program evaluation. Doing an evaluation. Each of these stages raises different questions to be answered by the evaluator, and correspondingly different evaluation approaches are needed. Rossi, Lipsey and Freeman (2. Assessment of the need for the program. Module 1: Program Evaluation Overview. Guide to Program Evaluation. An Overview of Education Evaluation. Process evaluation focuses on program implementation and operation. Assessment of program design and logic/theory. Assessment of how the program is being implemented (i. Are the program's processes maximizing possible outcomes?)Assessment of the program's outcome or impact (i. Assessment of the program's cost and efficiency. Assessing needs. This includes identifying and diagnosing the actual problem the program is trying to address, who or what is affected by the problem, how widespread the problem is, and what are the measurable effects that are caused by the problem. For example, for a housing program aimed at mitigating homelessness, a program evaluator may want to find out how many people are homeless in a given geographic area and what their demographics are. Rossi, Lipsey and Freeman (2. Needs assessment involves the processes or methods used by evaluators to describe and diagnose social needs. Programs that do not do a needs assessment can have the illusion that they have eradicated the problem/need when in fact there was no need in the first place. Needs assessment involves research and regular consultation with community stakeholders and with the people that will benefit from the project before the program can be developed and implemented. Hence it should be a bottom- up approach. In this way potential problems can be realized early because the process would have involved the community in identifying the need and thereby allowed the opportunity to identify potential barriers. The important task of a program evaluator is thus to: First, construct a precise definition of what the problem is. This is most effectively done by collaboratively including all possible stakeholders, i. Including buy- in early on in the process reduces potential for push- back, miscommunication, and incomplete information later on. Selecting an Appropriate Design for the Evaluation. Selecting an Appropriate Design for the. Program Planning and Evaluation is a. PROGRAM EVALUATION MODEL 9-STEP PROCESS. Evaluation Process—Overview Effective program evaluation is a carefully planned and. Specify the Evaluation Design. An Overview of Program Evaluation. Program evaluation represents an adaptation of social research. What type of evaluation design do I need? Study Designs for Program Evaluation. Second, assess the extent of the problem. They need to answer the . Evaluators need to work out where the problem is located and how big it is. Pointing out that a problem exists is much easier than having to specify where it is located and how rife it is. Rossi, Lipsey & Freeman (2. But indicating how many children it affects and where it is located geographically and socially would require knowledge about abused children, the characteristics of perpetrators and the impact of the problem throughout the political authority in question. This can be difficult considering that child abuse is not a public behavior, also keeping in mind that estimates of the rates on private behavior are usually not possible because of factors like unreported cases. In this case evaluators would have to use data from several sources and apply different approaches in order to estimate incidence rates. There are two more questions that need to be answered. Having identified the need and having familiarized oneself with the community evaluators should conduct a performance analysis to identify whether the proposed plan in the program will actually be able to eliminate the need. For example, whether the job performance standards are set by an organization or whether some governmental rules need to be considered when undertaking the task. There are three units of the population: population at risk, population in need and population in demand. The difference or the gap between the two situations will help identify the need, purpose and aims of the program. Identify priorities and importance. In the first step above, evaluators would have identified a number of interventions that could potentially address the need e. These must now be examined in view of their significance to the program. This must be done by considering the following factors: cost effectiveness (consider the budget of the program, assess cost/benefit ratio), executive pressure (whether top management expects a solution) and population (whether many key people are involved). Identify causes of performance problems and/or opportunities. When the needs have been prioritized the next step is to identify specific problem areas within the need to be addressed. And to also assess the skills of the people that will be carrying out the interventions. Identify possible solutions and growth opportunities. Compare the consequences of the interventions if it was to be implemented or not. Needs analysis is hence a very crucial step in evaluating programs because the effectiveness of a program cannot be assessed unless we know what the problem was in the first place. Assessing program theory. This 'logic model' is often not stated explicitly by people who run programs, it is simply assumed, and so an evaluator will need to draw out from the program staff how exactly the program is supposed to achieve its aims and assess whether this logic is plausible. For example, in an HIV prevention program, it may be assumed that educating people about HIV/AIDS transmission, risk and safe sex practices will result in safer sex being practiced. However, research in South Africa increasingly shows that in spite of increased education and knowledge, people still often do not practice safe sex. This is why it is important to read research that has been done in the area. Explicating this logic can also reveal unintended or unforeseen consequences of a program, both positive and negative. The program theory drives the hypotheses to test for impact evaluation. Developing a logic model can also build common understanding amongst program staff and stakeholders about what the program is actually supposed to do and how it is supposed to do it, which is often lacking (see Participatory impact pathways analysis). Of course, it is also possible that during the process of trying to elicit the logic model behind a program the evaluators may discover that such a model is either incompletely developed, internally contradictory, or (in worst cases) essentially nonexisistent. This decidedly limits the effectiveness of the evaluation, although it does not necessarily reduce or eliminate the program. An evaluator should create a logic model with input from many different stake holders. Logic Models have 5 major components: Resources or Inputs, Activities, Outputs, Short- term outcomes, and Long- term outcomes . Looking at the different components of a program in relation to the overall short- term and long- term goals allows for illumination of potential misalignments. Creating an actual logic model is particularly important because it helps clarify for all stakeholders: the definition of the problem, the overarching goals, and the capacity and outputs of the program. If the program theory fails to address the needs of the target population it will be rendered ineffective even when if it is well implemented. Rutman (1. 98. 0), Smith (1. Wholly (1. 99. 4) suggested the questions listed below to assist with the review process. The evaluator can assess whether the program theory is congruent with research evidence and practical experiences of programs with similar concepts. This evaluation determines whether the components identified as critical to the success of the program are being implemented. The evaluation determines whether target populations are being reached, people are receiving the intended services, staff are adequately qualified. Process evaluation is an ongoing process in which repeated measures may be used to evaluate whether the program is being implemented effectively. This problem is particularly critical because many innovations, particularly in areas like education and public policy, consist of fairly complex chains of action. Many of which these elements rely on the prior correct implementation of other elements, and will fail if the prior implementation was not done correctly. This was conclusively demonstrated by Gene V. Glass and many others during the 1. Since incorrect or ineffective implementation will produce the same kind of neutral or negative results that would be produced by correct implementation of a poor innovation, it is essential that evaluation research assess the implementation process itself. This involves trying to measure if the program has achieved its intended outcomes, i. Thus the concept of an outcome does not necessarily mean that the program targets have actually changed or that the program has caused them to change in any way. It further helps you to clarify your understanding of your program. But the most important reason for undertaking the effort is to understand the impacts of your work on the people you serve. More information about impact evaluation is found under the heading 'Determining Causation'. Assessing efficiency. Evaluators outline the benefits and cost of the program for comparison. An efficient program has a lower cost- benefit ratio. Determining causation.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
December 2016
Categories |