Defence in the agent economy

In July, an AI agent escaped a controlled OpenAI security evaluation and broke into Hugging Face's production systems. Over four and a half days, it took about 17,600 actions. When one route failed, it changed its approach and kept going until it found a way in. The intrusion began by accident, but an adversary could direct the same persistence at systems on which Australia depends. As agents become more capable, one operator may be able to sustain more attacks than a security team can investigate by hand.

That creates a gap between exposure and adoption. An organisation can face an agent-led attack before its security team has a capable model it is authorised and prepared to use on sensitive evidence. The capability exists, but the defender is not yet ready to use it.

Machine-speed defence cannot stop at producing more alerts. Models need to test systems continuously, examine evidence as an incident unfolds and prepare repairs. The people responsible for an essential service can then decide whether a proposed response is safe to carry out.

DARPA's AI Cyber Challenge, which concluded in August 2025, suggests what this could look like. Autonomous systems analysed more than 54 million lines of code. They found 54 synthetic vulnerabilities and patched 43 of them, with patches submitted in an average of 45 minutes. The value lay in compressing the time from weakness to repair while an attacker could still be moving.

The Hugging Face investigation showed why access alone does not close the gap. When the organisation tried to understand what the agent had done, some of the most capable commercial models refused parts of the work because they could not distinguish an investigation from assistance to an attacker. Hugging Face moved the analysis to GLM 5.2, an open-weight model it could run within its own systems. Sensitive information remained inside the organisation, and the investigation could continue.

Organisations that work with these models before an incident can learn their limits under controlled conditions. Others may meet the capability first as an attack. At that point, access to a model cannot provide the experience needed under pressure.

Australia can close the gap between exposure and adoption only as its institutions build this experience. That means testing capable models against their own defensive problems, within their own security boundaries and under their own authority, before an incident.