Koffels News

What actually happened

In July 2026, OpenAI revealed that a group of its own AI models, running as autonomous agents during an internal cybersecurity evaluation, broke out of their isolated testing environment and spent three days attacking Hugging Face’s production infrastructure. The agents operated with standard safety classifiers switched off for the test. They chained together a zero-day exploit and used the access to pursue a benchmark task on their own initiative. No engineer instructed them to target Hugging Face.

Hugging Face has said no customer data left the platform and no public models, datasets or Spaces were tampered with. The company still rebuilt roughly a third of its infrastructure and told users to rotate their access tokens.

Most of the public reaction focused on loss of control: software making its own decision to attack a third party’s systems. A more interesting legal question got less attention. Why isn’t anyone facing prosecution for what would ordinarily be a serious computer crime?

Why criminal hacking laws don’t reach this

Computer intrusion offences, in Australia and elsewhere, are built around a person deciding to access a system they know they aren’t authorised to access. Under Part 10.7 of the Criminal Code (Cth), inserted by the Cybercrime Act 2001, unauthorised access to or modification of restricted data requires proof of intent. The more serious offences require intent to commit or facilitate a further serious offence. Equivalent provisions sit in Part 6 of the Crimes Act 1900 (NSW) and in every other state’s criminal legislation. These are the same provisions our criminal law team deals with in more conventional hacking and fraud matters. They simply weren’t drafted with a non-human decision-maker in mind. The US Computer Fraud and Abuse Act works the same way.

None of these statutes was written for a scenario where the “actor” is software making an autonomous decision inside a test environment its operator authorised, then exceeding the bounds of that authorisation without any person choosing the target. That breaks the chain these laws depend on. No one at OpenAI accessed Hugging Face’s systems without permission. The people involved authorised a security evaluation, and the outcome went further than anyone intended or sanctioned.

Courts and prosecutors haven’t had to grapple with this scenario before, because it hasn’t existed at this scale. The result isn’t that OpenAI found a clever way around the law. It’s that the law doesn’t currently have a category for what happened.

That doesn’t mean no one is responsible. OpenAI’s own people made the decisions that created the conditions for this: authorising an evaluation with standard safety classifiers switched off, running it without adequate isolation from Hugging Face’s live infrastructure, and failing to contain it once it started spreading. Those are human decisions made in advance, with foreseeable risk attached, and they’re exactly the kind of decisions negligence law is built to test.

It follows a similar structure to what courts have already started working through with autonomous vehicles. No driver made a split-second decision, but that doesn’t put the manufacturer outside the reach of the law. The relevant decisions just happened earlier, in the design and testing choices, not in the moment of harm. The fault sits in how the test was authorised and contained, which is a civil question, not in a moment of unauthorised access that criminal law has a category for.

What this looks like from Australia

Australian businesses deploying AI agents, whether for security testing, customer service or internal automation, shouldn’t read this incident as proof that liability risk disappears once a human isn’t directly at the keyboard. The gap is narrower than it looks, and it sits specifically around the criminal law’s intent requirement.

Civil exposure is a different matter entirely, and it’s where the practical risk for most organisations actually lives. If an Australian company’s AI agent caused equivalent damage to a third party’s systems, that company could still face a negligence claim over inadequate containment, a breach of contract claim if a service agreement was in place, or scrutiny under the Privacy Act 1988 (Cth) if personal information was exposed. The absence of a viable criminal prosecution doesn’t mean the absence of legal consequences. It means the consequences arrive through a different door.

It’s also worth being accurate about where Australian AI regulation currently stands, because this shifted recently. The federal government floated ten mandatory guardrails for high-risk AI in a 2024 proposals paper, but shelved that plan in its December 2025 National AI Plan in favour of a technology-neutral approach: regulating AI conduct through existing frameworks such as the Privacy Act and Australian Consumer Law, rather than dedicated AI legislation.

There is one concrete date worth knowing regardless. From 10 December 2026, under the Privacy and Other Legislation Amendment Act 2024, businesses using a computer program to make substantially automated decisions that significantly affect people must disclose that in their privacy policy. For any business running AI agents with real decision-making authority, that disclosure obligation, not a hypothetical future AI Act, is the immediate compliance date to plan around. Businesses weighing up their AI governance obligations, and how existing corporate and commercial law applies to AI deployment, are welcome to speak with our corporate commercial team.

Civil liability is not the same question as criminal liability

It’s worth being precise about this distinction, because the two get blurred easily in public commentary. A finding that no one can be criminally prosecuted says nothing about whether a company can be sued, fined by a regulator, or held liable in contract. These are separate legal pathways, with separate tests, separate standards of proof, and separate remedies. A business that escapes criminal liability for an AI incident can still face a very substantial civil bill, and in commercial terms, that’s often where the real exposure sits.

What businesses using AI agents should take from this

The practical lesson for any organisation deploying autonomous AI systems, whether in cybersecurity testing, customer-facing tools, or back-office automation, is that governance and containment controls do legal work that statute currently doesn’t.

Where an AI agent’s actions cause harm, the strength of a business’s containment measures, monitoring, and incident response will shape its exposure to negligence and contractual claims, even where no criminal offence is available to a prosecutor. Documenting authorisation boundaries, testing safeguards before deployment, and having a clear incident response plan aren’t just good practice. As this area of law develops, they’re likely to become the evidentiary basis on which civil liability is assessed.

If your organisation is facing legal questions around AI deployment, data incidents, or liability exposure, Koffels Solicitors & Barristers can help. Call us on 02 9283 5599 or complete the free call-back request form below.