Deepfakes in Schools: Why Protecting Students Is Only Half the Job

In September, a 20 year old South Australian man pleaded guilty to creating sexually explicit deepfakes of a teenage girl and sharing them online. He is the first person charged under the federal deepfake laws introduced in 2024, and he faces up to seven years when he is sentenced in October.

The detail schools should pay attention to is where the source image came from. It was her graduation photo.

Nothing about that photo was careless. It was the ordinary record of an ordinary milestone, the kind of image every school in the country publishes several times a term. The young woman, now 19, told the court that the images of her were fake but the impact they had was painfully real.

So schools are right to be reviewing what they publish and how. I would simply ask that we finish the job. Controlling images is the easier half of this problem, and the half that will not fix it.

The safeguards are worth doing

The eSafety Commissioner’s advice to schools is sensible and worth following. Take a risk-based approach to what you publish, get genuine consent rather than a form signed in Year 7, review your privacy and security settings each year, and think hard about publishing the combination that makes a student findable, which is a full name, a uniform and a location in the same post.

Most schools I work with are already somewhere down this path, and the ones that aren’t usually get there after a scare. My suggestion is to put a review of your image policy on a Term 4 agenda rather than waiting for the scare, and to include your communications staff in it, because they are the people making these decisions weekly.

There is a limit though. Safeguards reduce what a stranger can scrape from your website. They do nothing about the phone in a Year 10 pocket at Saturday sport, the photo a friend posted on their public Instagram account, or the student who has known the victim since Year 7. Between January and March this year, eSafety received more than 100 reports of anonymous accounts targeting schools with AI-manipulated content. You cannot lock the front door on this. It’s already inside.

The half that gets skipped

Students will often raise this with me before staff do. In sessions on consent and online respect, it comes up as a rumour about an app, or a story about a school down the road, and it is pretty much always delivered with the particular casualness teenagers use when they are testing whether an adult is going to handle the topic properly.

That casualness is the whole problem and the whole opportunity. Julie Inman Grant has said her office is seeing deepfake image-based abuse in Australian schools at least once a week, with reports doubling over eighteen months. These are not criminal masterminds. They are 15 year olds with an app, a grievance or a group chat, and almost no idea what they are actually doing.

What young people need to understand

Three things, and none of them are about technology.

The first is that the image being fake does nothing to reduce the harm. That young woman’s sentence about fake images and painfully real impacts is the most useful teaching line produced in this country so far, and I would put it in front of students directly. A fabricated image is still published, still seen by people who know you, still there when you walk into homeroom.

The second is that the law is not confused about this, even when teenagers are. Creating or sharing sexual material of a real person without consent is now a federal offence, and where the person depicted is under 18, Australian law can treat AI-generated material the same way it treats any other child abuse material. I’m not a lawyer and this isn’t legal advice, but young people should hear plainly that “I made it on my phone in ten minutes” is a description of the offence rather than a defence to it.

The third is the one needs to be said again, and again, and again. If you make one of these, you have not pulled a prank. You have produced sexual material of a person who never agreed to it, you have done something the courts describe as insidious, and you have handed yourself a record that will follow you into every job application and every police check you ever complete. The girl in the image is the victim. You will also have detonated your own life, and you will have done it in an afternoon, for a laugh, in front of an audience who will delete the group chat and deny everything.

How to have the conversation

Start with the victim’s words rather than the technology, because the technology will change by Easter and the principle will not. Read her sentence out, then ask the room what “painfully real” means for someone who has to come to school the next day.

Then ask the bystander question, which is where most of the real work is. Almost no student in your care will create one of these. A great many will be shown one. Ask them what they would do if a mate opened a phone and said “look at this”, and let them argue about it. A student who has rehearsed that moment in a classroom has a chance of handling it in a bedroom at 11pm.

For parents, I would encourage a lighter touch than the one instinct demands. Asking your teenager what apps are going around at the moment will get you further than announcing a new rule about phones. You know your own child, and this conversation will look different in your house than in mine.

And I appreciate this is being added to an already impossible list. Nobody became a teacher or a parent to become an expert in generative AI. The point is not expertise. The point is that a young person hears an adult treat this as serious, specific and survivable, before the group chat teaches them otherwise.

The girl in that court case did nothing wrong, and neither did her school when it published her graduation photo. Both hands matter here. Protect the images properly, and teach the people who might misuse them properly, because only one of those two jobs changes what a young person decides to do at 11pm.

If your school is working out how to talk to students about consent, image-based abuse and online respect, that work is what our Empowering Relationships programs do from upper primary through to Year 12, and we’re an endorsed Trusted eSafety Provider. Please do reach out if it would help to talk it through.