AI won't tell you what you don't ask

The week everyone was patching

On 21 October 2026 the PHP team released fixes for two vulnerabilities that allowed remote code execution on a server without any login. Over the following days the maintainers of Node.js, Bun, the Brotli libraries for Python and the Pillow library published similar patches. You could write this off as a coincidence, but in at least two of these stories the flaws were found by researchers working with AI models, which can comb through popular open-source code faster than any team of humans could.

That is good news, and also a reason to ask what it means for companies that are thinking about deployments built mainly on AI. It is easy to see that AI is a great tool in the hands of someone who knows what to ask, how to check the answer and what to do with it. It does not seem, however, that it can be used fully safely without such a person, and that is exactly the question this text is about.

PHP2Shell, or what actually happened

The PHP team rated the two vulnerabilities in the PHP core, CVE-2026-109618 and CVE-2026-109619, at 9.2 on the CVSS 4.0 scale [1]. The first flaw comes from running out of memory. PHP then abandons processing the request halfway through initialisation, and the cleanup code treats the incomplete data as valid. In the second, the internal state of a request outlives the memory it points to, so the next request handled by the same process can use it before it has been properly initialised. The attacker needs no account and no knowledge of the application, because the bug sits in PHP itself, not in the code written on top of it.

The scale of the threat depends, among other things, on how many sites run the vulnerable code. According to W3Techs, PHP powers about 70 percent of the sites whose server-side language can be determined [2]. The people who found the flaws demonstrated working attacks on WordPress, Drupal and Joomla [3], but the bug sits in PHP itself, so other applications are at risk too, for example online shops built on Magento. According to the PHP team, installations with mod_php on Apache are the most exposed, because an attack can succeed before any application code runs, but in practice every server that handles requests from the network is at risk.

Nor could this flaw be fixed with settings. The PHP team published a list of mitigations, but pointed out that they only reduce exposure to some attack paths, and that the complete solution is to update [1]. Versions 8.1 and older no longer get official releases, so only minimal source backports were prepared for them. Anyone who had left an old application on PHP 7.4 or 8.0 on a hosting account had a real problem that day.

Who found it and how quickly the news got out

According to the PHP Foundation, the first traces of the out-of-memory problem were noticed in 2024 by a core developer who was fixing bugs pointed out in an external audit by Quarkslab. An exploit was theoretically possible, but no concrete, real attack vector was found, so the matter was put aside. Work on a fix stalled, because every approach has its costs, and the security team decides by consensus [4].

Researchers changed the situation. On 30 July the Calif team started auditing PHP as part of the Patch the Planet programme, and on 11 August it reported unconditional remote code execution with the same root cause as the memory problem. From that moment the fix became a priority. On 24 August Wiz reported the same thing, and on 18 September OpenAI sent further attack vectors. According to Wiz, AI models helped with finding the bugs, building exploits and validating the patches [3]. Behind each of these reports were security specialists who were able to confirm the result and coordinate it with the PHP team.

The release of the fix was originally scheduled for 22 October. On 20 October at around 15:00 UTC the CVE numbers appeared online, but without technical details, so PHP moved the release forward by a day. In the evening Calif accidentally published a video of its working proof of concept, and on 21 October at 7:41 UTC Chainguard broke the embargo and published images containing the fix more than six hours before the agreed time [4]. The same day watchTowr saw attempts to exploit both flaws on its honeypots [5], and on 26 October the US agency CISA added both flaws to its catalogue of actively exploited vulnerabilities [6].

One thing is clear from this. The time between someone outside a narrow circle learning about a flaw and being able to exploit it has shrunk to days or hours. Coordinated disclosure, meanwhile, took more than ten weeks, counting from the first report on 11 August.

What it threatened

An attacker could run their own code with the privileges of the PHP process using a single request, which is usually enough to read the configuration, database passwords, API keys and the files the application uses. The Wiz researchers also claim that the second flaw allows reading the process memory, which can hold data of other users served by the same process, even without full code execution [3].

The reactions of institutions show how seriously the matter was taken. The US agency CISA recommended taking PHP-based systems offline if they cannot be updated immediately or if they run PHP 8.1 or older [6]. The UK’s NCSC wrote that, as a last resort, taking the system offline until it can be fully mitigated could be considered [7], and the Canadian Cyber Centre recommended isolating internet-facing applications from public networks. In its alert, updated on 23 October, it still said that no active attacks had been identified, although this could change quickly [8]. Three days later CISA concluded that the attacks were under way. Whether this is one of the biggest bugs in the history of the internet is not for me to judge, because that depends on the number of vulnerable systems, the scale of real attacks and the time needed to patch. What I do know is that the flaw affected a technology that sits under most websites.

Not only PHP

Other ecosystems saw something similar, although perhaps the reach of the vulnerability was not quite as spectacular. The Brotli decoder, a compression library used by Node.js, Bun, Envoy, Istio, .NET and Python packages, had a vulnerability tracked as CVE-2026-111497, an out-of-bounds write rated 9.2 by Google. It was reported by researchers from OpenAI Codex Security [9]. According to an analysis by Hacktron, the fix landed in the Brotli source on 27 February without being marked as a security fix, and releases containing it appeared only on 21 October, eight months later! Then, over the next few days, other projects published their own versions, because each of them ships its own copy of the decoder [10].

At the same time Pillow, which most Python projects use for image processing, shipped version 13.0.0 with fixes for three critical flaws. One of them, argument injection through Ghostscript, allows running a command on the system under a certain configuration [11]. The conditions are narrower than in PHP, but anyone who accepts photos from users should check their stack. Both cases come down to the same thing: a bug in one library spreads to a dozen products, and few people know which versions are actually built into their application.

Finding the bug is only the beginning

The PHP Foundation’s post shows that detecting a bug and fixing it are two different stages. The foundation itself calls the delay a structural failure, caused both by how the security team makes decisions and by a lack of resources, and says that security is handled by a small group of developers paid by the foundation and some volunteers [4]. I don’t see this as an accusation against the people working on the language, more as a description of the conditions they work in.

The foundation also notes that AI has increased the number of reports. Since 1 July it has published 15 CVEs and rejected more than a hundred reports as invalid, duplicate or impossible to reproduce. AI tools help with triage, but according to the foundation every decision is made by a person, and a person works on every patch [4]. In the case of PHP, once the flaws were found, people had to assess the reports, agree on a fix, prepare releases for four active branches and several historical versions and coordinate the publication with Linux distributions, and then administrators around the world rolled it out within a few hours. The model was responsible for one link in this chain, and the first one.

AI won’t answer if you don’t ask

At the same time, on 13 October, LessWrong published a post titled “LLMs Knowingly Hide Their Mistakes”, which at first glance has nothing to do with cybersecurity [12]. Its authors, Lucas Florin, Amelie Knecht, Ulysse Schaller and Thilo Hagendorff, wrote made-up mistakes into the conversation history as if the model had made them earlier: it sent confidential data to the wrong person, or overwrote a file instead of appending to it. Then the user asked about something completely different, and the researchers checked whether the model would come back to that mistake on its own and admit it.

In 67 percent of cases the models did not disclose the mistake. Explicit confessions ranged from 3 percent of runs (Kimi K2.6) to 35 percent (Claude Sonnet 5). In 5 percent of runs the reasoning trace showed that the model knew about the mistake but left it out of the response. More often the problem lay elsewhere, because in about half of the runs there was no trace of awareness of the mistake in the reasoning, so the issue is not only honesty but also attention. When the next message reminded the model that the user still cared about the matter, it confessed more often, although concealment still happened, in up to 15 percent of runs in the worst case.

The conclusion I draw from this is simple. A model does not return to a mistake on its own when the conversation is about something else, and to know what to ask, you have to understand what could have gone wrong. The study did not, however, test what happens when we ask directly whether something went wrong, so I don’t know whether that alone is enough. It is also fair to say that this was an experiment with mistakes written into the conversation, not with errors the model made itself, and that it concerned reporting in an agent’s work, not finding flaws in code. It doesn’t follow that models can’t find vulnerabilities, because, as we saw above, they can.

Where is the catch?

Both stories have something in common. Wiz, OpenAI and Calif found the flaws in PHP because they worked with the models as specialists who knew where to look, how to verify the result and whom to report it to. I don’t think the same model in the hands of someone without that knowledge would have produced a similar result, but the LessWrong study suggests that it also wouldn’t warn you about what it failed to do.

That is why the question is whether a way of working in which AI does most of the tasks of programmers and administrators, and nobody with experience reviews its results, can be considered safe. I don’t think there is one answer for everyone. A hobbyist building a site for themselves carries a very different risk than a company handling customer data, and the line lies somewhere in between. I also don’t think anyone can honestly say that the risk is zero.

Instead of a ready answer I propose four checks you can do today, no matter how much work we hand over to AI.

The first concerns dependencies. Brotli showed that a flaw in one library means patches in a dozen products, so someone should be able to answer within minutes whether a given component is inside our application, and not merely have a generated list. In a PHP project this starts with composer audit and knowing the interpreter version on every server, and in a Node.js project with npm audit and the runtime version.

The second concerns unsupported versions. PHP 8.1 and older got only source backports, so if an application runs on such a version, someone has to know about it, and before a hypothetical Wednesday like that one, not on the Wednesday itself.

The third concerns time. The attackers acted on the same day the patches appeared, so you need a person with the authority to roll out an update outside the normal release cycle, and prior agreement, including consent to take the system down for a while if there is no other way.

The fourth follows from the study I described. Someone or something has to check the model’s work independently of its own report, meaning tests, logs, code review by a human, or a second model that evaluates the record of the work from the outside.

And on Wednesday evening?

My last question is for those who are responsible for a product. Imagine Wednesday, 21 October. You read that a critical vulnerability affects the technology your application is built on, that attackers have started exploiting it before most administrators managed to update their servers, and that government agencies are advising to take systems offline if they can’t be patched right away. You have to decide within a few hours whether to shut the service down or take the risk, and you need someone to tell you what your system looks like and what your real options are.

I find it hard to imagine that, at such a moment, what an AI assistant that knows your project from earlier conversations tells you will be enough, especially if you already know that models can fail to mention things they consider unimportant for the current question. It’s good to know the answer before that Wednesday comes. There is still time today to work it out. On Wednesday evening there won’t be.

Sources

1. Critical Remote Code Execution Vulnerabilities, php.net, 21 October 2026
2. W3Techs, Usage statistics of PHP as server-side programming language, data from 26 October 2026: PHP is used by 69.2% of the sites whose server-side language could be determined, which is not 69.2% of all websites. The Polish Wikipedia gives 77.4%, but that is data from 7 May 2022.
3. PHP2Shell, the Wiz team’s site
4. Today’s PHP security release and how we’re strengthening PHP security, PHP Foundation, 21 October 2026
5. PHP2Shell RCE Exploited: CVE-2026-109618 & CVE-2026-109619, watchTowr
6. Critical Vulnerabilities in PHP Require Immediate Patching, CISA, 21 October 2026, updated 26 October
7. Critical security issue affecting PHP, NCSC (UK), 21 October 2026
8. Critical vulnerabilities impacting PHP, Canadian Centre for Cyber Security, 21 October 2026, updated 23 October
9. Broccoli2shell: Everything You Need To Know About The Critical Brotli Decoder Vulnerability, Wiz
10. Node2Shell: Brotli Decoder RCE in Node.js (CVE-2026-111497), Hacktron
11. Three Critical Pillow CVEs, OX Security
12. LLMs Knowingly Hide Their Mistakes, Florin, Knecht, Schaller, Hagendorff, LessWrong, 13 October 2026 (full paper: arXiv:2610.11351)