In this new ‘Demystifying Evaluation’ blog series CES Project Specialists Karl McGrath and Dearbháile Slane will share CES’ learning and experience to help you get the most from your evaluation.
Over 5 instalments we will explore:
In the first of our series, we start with (seemingly) simple questions that are fundamental to understanding the evaluation process. We will explore:
Our definition of ‘evaluation’ is:
‘A planned investigation of an intervention, according to specific questions of interest. It is carried out in a systematic and robust way, using reliable social scientific methods, to determine an intervention’s value, merit or worth.’
An evaluation is an investigation. It examines or studies an aspect or aspects of an intervention in detail. In CES’ work, an intervention usually refers to a service, project, programme or policy. Essentially, an intervention can be anything people do to try change a problem or situation.
So, an evaluation investigates an intervention.
This depends on the specific questions of interest. In our work, the commissioner of an evaluation typically identifies the evaluation questions they want answered. But if a commissioner is unsure about what questions to ask or how to frame them, the evaluator can work with the commissioner to help develop or refine the questions.
Evaluation questions should be specific and clearly direct the evaluator towards the information you want to find out. It is often useful to consult with key evaluation stakeholders for feedback and revise your evaluation questions if required.
Our definition of an evaluation gives a couple of clues here. First, it is planned. Evaluators will do a lot of planning and preparation at the beginning to decide which methods will help them to answer the evaluation questions, at what stage they should be carried out, and with whom.
This allows the evaluation to be carried out in a systematic and robust way, using reliable social scientific methods. In other words, evaluators don’t decide on their methods as they go along. They want to provide trustworthy answers to the evaluation questions. Following a structured, pre-planned approach with methods that are suitable to the evaluation questions will help them to do that.
Evaluation is a tool that should serve you! There are a range of reasons why you might wish to evaluate an intervention, service, policy, or practice. Evaluations can be used to:
The findings that emerge from your evaluation can be used to assess the intervention’s effectiveness, value for money and identify areas of improvement’ - it's value, merit or worth. Certain types of evaluation approaches can also help you to uncover why and how something works and in what contexts – allowing you to develop an evidence base and use that learning to inform future projects or initiatives.
It is never too early to think about evaluation. In our experience, it is useful to think about how you will monitor and evaluate an intervention before it is implemented, to ensure systems can be put in place to collect the data you will need to measure change and evidence your outcomes from the start.
With that said, it’s also never too late to evaluate and even when there are few or no existing forms of monitoring data, there is always something that can be done to retrospectively evaluate an intervention.
Yes, there are many! In our next blog we will explore the various types of evaluations CES has undertaken and consider when and why each might be most useful.
