Category: computing

A lot of column-inches and a bazillion frantic tweets have been bashed out recently about the AI tool ChatGPT: the public, text interface to a Large Language Model (LLM) created by the OpenAI consortium. Originally a not-for-profit body which boasted Elon Musk as one of its original investors, OpenAI is now unashamedly for-profit and in November 2022 launched ChatGPT, a language model built on GPT3, the third iteration of their “generative, pre-trained transformer” software. This tool can process natural language text and respond with natural-sounding text back. It also remembers conversations, hence the “chat” element, and this is what makes it more powerful than previous iterations: you can refine your query over several inputs to get better results.

AI services like ChatGPT join a long line of technologies to have been described as both “dangerous”, downsides from its use as a class cheat’s superpower, to a phishing and identity fraud weapon. it’s “the end of assessment as we know it’ because “many of the problems we set in secondary school can now be solved by apps… It is not a good sign that we still teach and test mathematical material in such a routine way that free off-the-shelf systems like these can handle lots of it with ease’ – economist Daniel Susskind in his book A World Without Work.

But we’ve been here before. The internet was going to spell the end of academic assessment in the 90s. In truth it didn’t change much, except for democratising information so you didn’t need to be in school to learn. If we’re honest with ourselves, outside of controlled conditions such as the exam hall, there are a myriad ways to cheat already: copying from others, searching online or using an online service to do your homework for you, sometimes called an essay mill. If a piece of work is important (such as assessed coursework or “controlled assessment” work) then the teacher should already have some skill in plagiarism-checking. Online services such as Turnitin are widely used, but I’ve always found simply asking a student to explain their work, called a “viva voce” interview in academia, does the trick. You may not need to do this with 100% of submissions, just a 10% check might be sufficient to deter serious plagiarism.

And if you absolutely must have confidence the submission is the students own work, then conduct a test in controlled conditions with no devices allowed. But only a small number of pieces of work (often just a summative test of required knowledge to progress to the next stage, e.g. the GCSE’s and A-levels in the UK or the college-entrance-assisting AP tests in the US, and the final exams of a degree course) over a student’s lifetime should require this level of scrutiny. Everything else should be treated as formative and afforded a lesser degree of validity and therefore require less strict control.

Most of my students work is either self- or peer-assessed. A mixture of online self-assessment using platforms like Quizlet (most subjects) or SmartRevise (Computing and Business only at the moment) get the bulk of the feedback done cost-free, and the rest is largely done by the students with lots of guidance from me. I’m glad the UK never adopted the American high-school system of grade-point average (GPA) scoring, not least because it penalises poor early performance which is unfair to immigrants and those with health issues, and is linked with self-esteem issues, but because it makes every piece of work high-stakes and high-cost to the teacher. When both teacher and student are stretched to the max by tests every semester, there is no space to relax and enjoy the journey. And pity the student who gets a C during the grief of a bereavement which prevents them getting the required GPA for their college of choice no matter what they do next. (If you’re in the UK, thinking “glad we don’t have the GPA system here”, count how many controlled tests and data drops you must do each year, and ponder a moment).

Let’s remember the purpose of assessing work. All assessment is a surrogate for what we want to know: what is in their heads. Assessment is not an end in itself, the mark should reflect some measure of achievement that helps both teacher and student understand how to make progress. Let’s not forget that what we want to achieve is an improvement in learning, what’s in their head when they leave school, not what they wrote in a paper when they were eleven or fifteen. As Tom Sherrington writes:

If testing is going to have an effect on the learning process, it needs to have an outcome that will help students to develop a sense of themselves as learners and an awareness of what else there is left to learn. 

Tom Sherrington’s TeacherHead blog, link

As Daisy Christodoulou writes, the struggle, not the end product, is the point:

If a student struggles for an hour over an extended piece of writing and then finds that a computer has surpassed it in seconds, it is entirely possible they will feel demotivated. What they need to hear from adults is don’t worry, your work is of value, you’re on a journey and you are developing your own writing skills. 

Daisy Christodoulou’s No More Marking blog, link

Design your assessments so they create actionable feedback, not just test scores. Furnish the students with marking rubrics ahead of the assignment, and get them to mark themselves against the rubrics before handing in. If they’re using ChatGPT at home to write essays, they might be short-circuiting part of the process, so have the class critique each-others essays in class afterwards. Create model answers or “what a good one looks like” WAGOLLs they can mark themselves against, or choose a student’s answer that is high quality and work with the class to determine what makes it so. Joe Kirby’s seminal 2015 blog post “Marking is a Hornet, Feedback is a Butterfly” is still my go-to article for in-class feedback ideas that can be re-purposed in the ChatGPT age, even to make the most of so-called “plagiarised” work.

Back to ChatGPT and the “plagiarism panic”. Too often we forget the upsides of a new technology in all the swirling panic about its dangers. For LLMs like ChatGPT these include levelling the playing-field for people with disabilities or assisting people for whom English is an additional language. Make sure your EAL students have access to it and know how to use it. Discuss with your SENCO how students might use it to overcome learning difficulties like dyslexia and dyspraxia. As this article explains, it’s already helping a landscaper with low literacy write professional-sounding emails to customers (see image above), and writing assertive letters to a landlord on behalf of a shy tenant regarding a water leak (the leak was fixed in 3 days). We demonise this technology at our peril.

And with any luck, ChatGPT might bring down the GPA system and its pale imitations in the UK, with all the inequities those systems perpetuate. Which can’t be a bad thing.

If you enjoy my blog, why not buy me a coffee? And I talk much more about AI in the context of the Computer Science GCSE in my book.