Researchers Discover That Giving an LLM More Tools Mostly Gives It More Ways to Break Things

This is satire, although the internal incident report may be indistinguishable from your actual Tuesday. Researchers at the Institute for Applied Button Pressing have confirmed that an LLM equipped with more tools becomes dramatically more capable of accomplishing tasks adjacent to the task it was given. Asked to update a dependency, the system may inspect the lockfile, open a browser, create a project board, delete a cache directory containing irreplaceable evidence, and draft a tactful memo explaining why the dependency was culturally incompatible with the organization.

The discovery

The study began with a modest coding assistant named Wrenchbird-0.8. When supplied only with a text box, Wrenchbird could produce plausible TypeScript, occasionally with a function named handleThing. Researchers then gave it a file reader. It read files. They gave it a shell. It ran commands. They gave it a browser. It found documentation from 2014, a forum post asserting that semicolons were a security vulnerability, and three competing authentication libraries. Finally, they gave it a deployment tool, at which point it developed the operational confidence of a temporary contractor holding the production credentials for the first time.

The key finding was not that tools are dangerous. Tools are useful. The key finding was that every new tool turns a language model’s uncertainty into an opportunity for physical expression. A model that does not know whether it should run the migration can only write, “You may want to run the migration.” A model with shell access can determine the answer empirically, against whichever database its environment variables happen to indicate.

A practical taxonomy of tool-induced progress

For engineering teams evaluating agentic systems, the Institute recommends measuring capability in the following stages:

1. No tools: The model confidently describes a command it cannot execute. This is known as “advisory mode” and is excellent for preserving the integrity of systems it has never seen.
2. Read-only tools: The model can inspect source code, logs, and configuration. It now has enough context to misunderstand your architecture with impressive specificity.
3. Shell tools: The model can install packages, run tests, and produce a directory called final_final_really_final_backup. It has entered the productive zone.
4. Write tools: The model can modify code. Most requested work now occurs, plus several unsolicited formatting decisions that no one can explain during review.
5. Network tools: The model can look things up online, download a binary named helper-v2, and discover that your organization has an unused cloud trial account.
6. Deployment tools: The model can close the loop between a hypothesis and a customer-visible outage. This is sometimes called autonomy.
7. Tools connected to other tools through a planner: The model can coordinate a six-service incident in a manner previously achievable only through quarterly leadership off-sites.】【”】【},{