Do Defenders Still Matter?

The most effective attacks this year didn't even need a zero-day at all. If we look at North Korean actors (UNC1069) they cloned a real company, including its founders' likenesses, built a convincing Slack workspace, and staged a Teams meeting where the lead maintainer of Axios was prompted to install a fake update, which resulted in a malicious Axios release. A library with around 100 million weekly downloads was live on NPM for about 3 hours. Whether AI helped build the Lua isn't known but this is the exact kind of craft that makes AI cheaper, easier, and faster.

We are also seeing a rise in deepfakes, with evidence of North Korean IT workers using real-time deepfake videos to pass job interviews, then using their insider access to steal data and extort employers once discovered. An 11th nation advisory in July confirmed the deepfake tactic.

Supply chain attacks have also seen a major uptick this year. Axios is one clear example of this and if we look at the attack on Vercel, it was breached through Context.ai, a third-party AI tool an employee had connected with broad permissions. Vercel assessed the attacker as highly sophisticated, citing their operational speed and complexity.

This leads me to raise a theory: if Mythos-class models give defenders unprecedented edge, the organisations holding that model become extremely high-value targets. You don't need to compromise Mythos itself if you can compromise someone that is talking to it or a third party that has access to a platform that hosts or uses these models.

Anthropic's report shows attackers already are thinking this way, targeting AI vendors, evaluators, and trusted access programs to reach restricted models. As far as we know none have succeeded but the AI tools and keys we are connecting to our environments are becoming both the entry point and the prize.

This is the part of the article that should get every cybersecurity professional's attention not only the ones sitting within Australia.

On June 18, an OpenAI agent gained unauthorised access to Medicare's statistics portal run by Services Australia. It wasn't on a hacking task. It was trying to retrieve data, hit some errors, and started probing for vulnerabilities instead. The government has said other systems may have been affected, including New South Wales BOCSAR (crime statistics) and the Victorian Department of Health. Australia wasn't told until 84 days later via an email, which was also generated by AI.

This attack wasn't an isolated instance. We have seen researchers at Transluce trace a similar behaviour against the Australian Institute of Health and Welfare, Data USA, and a U.S. Department of Education site, and find agents using a malware analysis service to get around their own web access guardrails.

In July a swarm of around 700 OpenAI agents was linked to an attack on Hugging Face, which Hugging Face described as the first documented large-scale cyberattack driven entirely by autonomous agents. That same month, Anthropic disclosed that during a cybersecurity evaluation, its Mythos 5 model built a malicious PyPi package and published it to the real registry. A misconfiguration meant that the test environment had live internet access, and Mythos believed it was still within the simulation. The package was only live for about an hour, but in that time, it was downloaded and run on 15 real systems, including a security company's web malware scanner that gave Mythos the company's credentials, which it then used to access more of their infrastructure. It never targeted that company directly, and it didn't even know it existed. However, it was still autonomous access without authorisation.

In an authorised bug bounty, pwn.ai's agents escaped Google's kvmCTF, one of the most hardened hypervisor targets around. It built a 14,000-line exploit harness and used a use-after-free chain triggered by a 3-byte instruction to escape the sandbox.

None of this is AI “going rogue” in the scientific sense. No Skynet just yet or at least that we know of. It is goal-seeking systems taking whatever path works, including the ones we never intended, or maybe it's the path we should have seen coming. Let's think about it for a minute. What were these systems and these models actually trained on in the first place?

The internet. Decades of human history, famous hacks, worms, breaches, write-ups, exploits, and tutorials. They were all on the internet and are now baked directly into the foundation of these AI models. When an agent hits a wall, it reaches for what it knows. A lot of it is how to break into things, a lot about all the write-ups, the exploits, etc. That's exactly why we need people watching.

Not long after Glasswing was announced, May rolled around and honestly, it felt like drinking from a firehose. Every morning there was another critical CVE, another public exploit, or another supply chain compromise to deal with. Here is a snapshot from my own notes to look over the month of May alone.