This guidance has highlighted the importance of establishing a theory of change (see evaluation practice 1.2) including outputs, outcomes and impact. It is important to identify the outputs, outcomes and impact of your initiative and how these are different from the learning outcomes defined for the module or programme, and then how these relate to the student learning experience. For example, if the initiative is around ‘feedback literacy’ are you trying to develop certain knowledge and skills or perhaps attitudes that you consider to be important in enhancing ‘feedback literacy’? These might include knowledge of the application and use of assessment criteria but also, perhaps, personal motivation, organisation and time management to make use of the feedback and a questioning mind-set / capacity to reflect on own work. Such desired outcomes often go beyond the stated learning outcomes for the module and may be inspired or motivated by a desire, for example, to reduce or eliminate an identified awarding gap.
No matter what you are trying to achieve with your intervention, you should explore and scrutinise the literature with two purposes in mind:
- To help you develop your practice (planned intervention)
- To help you to determine your evaluation scheme.
Planning your evaluation before you commence your intervention / initiative will also enable you, or certainly give you the opportunity to, identify more clearly the purposes of the planned change to practice and exploring the literature will also support this. There are some key questions to ask yourself when taking a scholarly / evidence-based approach to planning the intervention:
- Has someone solved this problem before?
- Can you apply their solution to your context?
- Will you do something new? (applying / integrating existing scholarship or discovering new knowledge).
Once the rationale and evidence base for the intervention is established then the approach to evaluation can be informed by that same literature and evidence base and other linking research. For example, in the ‘feedback literacy’ example highlighted in evaluation practice 1.1 and practice 1.2 the intervention adopted might draw on the work of Carless and Boud (2018) amongst others, and in doing so perhaps draw on the framework proposed by Carless and Boud, both to design the intervention and plan the approach to evaluation (this is just an example to illustrate a point and is not intended to be comprehensive coverage of the topic and associated literature).