Principle 4: Adopt authentic assessment

GenAI has prompted the sector to evaluate the extent to which existing modes of assessment are valid and sustainable in a GenAI-augmented world. Initial reactions to GenAI often focused on academic practice and integrity (Jisc, 2023). Since then, the debates have become much broader and deeper. For example, the essay has been pronounced ‘dead’ (Marche, 2022); and concerns about student agency have been raised (Wallbank, 2023). Three core certainties have emerged. 

  1. We must embrace and adapt to GenAI. ‘Digital and information literacy’ is a Brookes Graduate Attribute and future scholars, workers and entrepreneurs need to be able to use digital tools effectively. 
  2. Writing is a core component of assessment.
  3. In the face of artefacts being generated from prompts and created predictively via algorithms, there is an increasing emphasis upon placing human traits and thinking processes at the forefront of teaching and assessment. 

If we rely on traditional models of assessment then we have to recognise that students may be able to use (and possibly misuse) GenAI at every stage of the process, as illustrated by the following example of a written essay or report on a tutor-generated topic.

Students should know which of these applications of GenAI are acceptable or not acceptable for every one of their summative assessments.

An example: stages in a simple written assignment which could involve GenAI

Assessment components

1.  Assessment brief, students could:

  1. Define key terms (ask GenAI for definitions/expectations)
  2. Generate/plan structure (ask GenAI for an outline)
  3. Check assessment criteria (GenAI uses them to evaluate the completed assignment)
  4. Suggest schedule/time management (ask GenAI to propose schedule)

2.  Research and planning, students need to:

  1. Find relevant research papers (search using relevant paper analysing GenAI tool)
  2. Select the most important sources (search using relevant GenAI tool)
  3. Read and understand key texts (use a GenAI Large language Model (LLM) to summarise and clarify)
  4. Summarise key ideas (use GenAI to summarise and clarify)

3.  Writing, revising and presentation, students need to:

  1. Use appropriate structure (ask a LLM to suggest outline with appropriate sections)
  2. Prepare draft (ask relevant GenAI tool to propose outline)
  3. Design and prepare visual aids (use a image creating GenAI tool to prepare diagrams and images)
  4. Critique and revise (ask a LLM to evaluate using criteria)
  5. Final review and proof-read (LLM to proofread and comment on style)

The following synthesises current thinking and pedagogic principles related to inclusive, authentic assessments within the framework of Brookes’ Strategy 2035, Brookes’ Graduate Attributes, and the IDEAS Inclusive Curriculum Model. The focus is on assessment that either:

  1. incorporates GenAI or,
  2. creates opportunities for assessing students and fostering learning independently of AI.

Ideally changes to modules or assessments should be discussed at the programme level. This will ensure that the approach to assessment in response to GenAI is strategic and that decisions are made with consideration of both programme and level outcomes.

One useful response to the problem of deciding which uses of GenAI are appropriate in a given assignment is the AI Assessment Scale proposed by Mike Perkins and colleagues. There is also a useful short introduction to the development of this scale from Leon Furze. They propose a 5-point scale which ranges from ‘No AI’ where the assessment has to be “completed entirely without AI assistance” through to ‘Full AI’ where students can use AI fully without declaring what they have used. The intermediate steps allow students to use AI at some points and ask for different reporting of what has been used and how.

Changes to Assessment that do not require Quality Assurance/SRS Approval

Some changes to a module assessment do not require quality assurance or SRS approval. 

However, making your assessment more resilient to inappropriate GenAI use tends to involve much thought and effort in assessment design. What follows below is an outline of the quicker types of activities that can be implemented in order to support the integrity of teaching and assessment. While none of these can offer 100% GenAI proofing (nor is that the goal) they offer an approach to assessment that can facilitate the development of assessment literacy and offer a richer diet of authentic assessment:

  • anchoring assessment (where possible) to personal, local, or institutional contexts (where there is little to no web data available) can make it more difficult for the AI to produce high-quality texts that are likely to meet the specifications of the marking criteria. 
  • grounding the assessment in the content of an in-person lecture or seminar discussion where key themes are presented and contested (by teaching staff or peers) and then responded to in the discursive assignment. 
  • developing the marking criteria to articulate the level of specificity and analysis expected may mean that an AI-generated text is less likely to successfully meet the brief. This is because Large Language Models (LLMs) like ChatGPT and other text-generating AIs use web scraping to gather data for texts that they produce.
  • ensure you follow Brookes’ default settings for submissions to Turnitin to include the reference lists as this is a means of checking that the sources are genuine and in circulation if matched. In addition, instruct students that all references must have working links and accurate DOIs which make them traceable by the marker.
  • consider the weightings and granularity of different marking criteria. Add criteria or descriptors that include marks for responding to formative feedback, marks for synthesis with reference to their own knowledge or experience, or marks for reflection.
  • where there is an oral component to the assessment, consider ‘tying it’ to a written component and adding a critical rationale of the approach and sources used in the written assessment and therefore providing evidence of learning. 
  • think about ways that you might ask students to evidence the writing process. This may mean developing draft paragraphs in class, submitting the assignment in sections, or tracking the development through audit logs in Word/Google Doc or Moodle.

Potential uses for GenAI

Using GenAI for reflective writing

Embracing AI in learning and teaching (simulation)

Demystifying oral assessment

For more information on Talking Teaching across the Globe please see the OCAED website.