Back in May, Google’s Gemini AI models breached three separate company computer systems, by accident, during a safety testing, with the incident adding to an ever-growing list of AI models behaving unpredictably, even as President Donald Trump seeks an expansionist role of AI across the federal government, establishing an AI task force, according to Bloomberg.
The breach occurred during red-team testing conducted by Israeli AI-safety startup Irregular, the same round of tests in which models built by OpenAI, Anthropic, and Meta also compromised systems they were not meant to access. A misconfigured environment lets the model reach the internet, showing how quickly an autonomous tool can cross its intended boundary.
The Commonality Among Big Tech AI Labs
Gemini reached the open internet because of a misconfiguration in its testing environment – not because it was directed to do so. The Gemini-parent has not disclosed further technical detail on what data, if any, the model accessed once inside the three systems, according to Bloomberg.
The situation is more common than any of the AI leaders would like to – or will ever – admit. In actuality, it’s more common than the public conversation around AI chatbots might suggest.
Google was not alone.
All four of the AI industry’s digital intelligence leading labs, Gemini’s Google, ChatGPT’s OpenAI, Claude’s Anthropic, and Muse Spark’s Meta, have disclosed instances of their AI agents breaching systems without human instruction to do so. The models surfaced through internal safety testing instead of customer complaints.
Stronger models can create security problems without instructions to cause damage. More than anything else, the problem is structural to how autonomous agents currently operate and is not tied to a flaw specific to any single company model.
President Trump’s AI Czar Announcement Matters Here
On Saturday, President Donald shared in a social media post his utter rejection to the very notion of AI safety concerns, deeming it as nothing but a mere “hoax,” comparing them to past scandals and debates around climate change.
“We will not in any way hinder or stifle the growth of this incredible Industry,” President Trump announced on Truth Social, adding that AI will eventually account for as much as 25% of US Gross Domestic Product (GDP).
Currently, President Trump is promoting an AI task force agenda built around competition and growth, revealing he would appoint an AI leader and establish an AI Force, modeled on the US Space Force he established during his first term, to tackle the issues facing the tech industry.
President Trump AI task force announcement argues the US must remain ahead of China, even as technology leaders warn that advanced models are becoming harder to control.
Craft Ventures co-founder, David Sacks, was previously appointed by the President as AI czar, when second term began. Sacks has since then stepped back from that specific role but remains co-chair of the President’s Council of Advisors on Science and Technology.
The White House believes that anointing a new AI czar could strengthen coordination across federal agencies. Be that as it may, the timing of President Trump is not short of peculiar. Trump’s post was dropped on Truth Social in the same week as leaders at some of the very companies, whose AI models demonstrated this hacking behavior – including Anthropic – publicly urged for a slow down to the development of its most advanced models.
A safe AI task force needs independent testing, incident reporting and clear limits before automated systems to access sensitive networks or public decisions.
Former President Barack Obama, in a separate appearance at Colgate University said Democrats should make AI regulation a midterm election issue, describing “a misalignment between what our society needs and the commercial imperatives that these companies are facing.”
Who Benefits? Who Bears All Risks?
Washington’s stance on the White House AI task force evidently benefits two separate, and linked, parties. AI developers and Wall Street.
AI developers, such as Google, OpenAI, Anthropic, Meta, and Microsoft, are the biggest beneficiaries from the AI task force. They face less regulatory friction with the federal government, and have continued access to lucrative government contracts, and federal agencies desperate to deploy AI tools faster, inevitably cutting cost and staff time.
Google benefits when Washington buys its products, while agencies benefit from faster work. Citizens may lose when commercial and political pressure outruns security reviews, transparency, and human responsibility.
Wall Street also has a stake in the game, with outcomes equally stretching to equal those of Google and the federal government.
Wall Street strategists highlight that a substantial share of the US stock market’s recent gains have been exponentially driven by expectations around AI’s continued – and uninterrupted – proliferation. It is the very same growth that gives investors a financial itch, that can only be scratched by the preference of the accelerate-first approach, independent of the AI safety debacle and debate.
The Risks Are There, but Who’s Pushing Back?
The danger is clearest at the United States Department of Transportation (DOT), which plans to use Gemini to draft regulations covering airplanes, vehicles, railways and pipelines. Officials believe it could reduce work lasting months or years to a draft produced within minutes, with a complete draft ready for review in 30 days.
For the US AI czar, that speed could demonstrate government efficiency. For the public, it raises a harder question, can a system known to hallucinate safely help write rules where mistakes may contribute to crashes, explosions or deaths?
Gemini was said to be capable of completing 80% to 90% of regulatory writing, leaving employees to check the work. Yet experienced rule writers understand statutes, court decisions and technical risks that automation cannot genuinely reason through.
“We don’t need the perfect rule on XYZ. We don’t even need a very good rule on XYZ,” said DoT general counsel Gregory Zerzan, according to meeting notes, adding, “We want good enough.”
That standard exposes the weakness in the President Trump AI czar approach. A draft can arrive quickly and still contain invented facts, missed legal duties or unsafe assumptions. Human review also weakens when agencies lose experienced lawyers and specialists.
A responsible AI task force should require documented testing, named human accountability and public disclosure whenever AI contributes to regulation.
The White House AI task force must separate innovation targets from safety approval, preventing political demands for speed from deciding whether a tool is trustworthy.
The first term US AI czar coordinated procurement, security, and training, but the new AI czar will most probably become a sales channel for developers, or even a way to weaken regulatory standards.
Yet an effective AI task force must recognize that accidental breaches show why access must follow proof of safety. Without enforceable safeguards, the new White House AI czar risks supervising adoption after decisions have already been automated.
A credible White House AI czar must protect citizens first, because fast government isn’t a better government when its rules are merely “good enough.”
