AI Is Moving Fast. Do We Know How to Contain It?

Artificial intelligence continues to advance at an extraordinary pace. But some of the more concerning warnings are now coming from people working inside the companies developing the most advanced AI systems.

Anthropic researcher Jacob Coxon recently resigned after three years of pretraining work at OpenAI and Anthropic, warning that AI companies are racing toward increasingly powerful, self-improving systems without adequate safeguards. Anthropic researcher Evan Hubinger subsequently supported the substance of Coxon's concerns, while noting that the risks associated with today's models remain low.

Those comments make for dramatic headlines, but they also raise a much more immediate question for business and technology leaders:

What happens when an AI system does something it wasn't supposed to do?

AI Is Moving Beyond the Chatbot

For many businesses, AI started as a tool for answering questions, summarizing information or creating content. That is quickly changing.

Agentic AI can increasingly interact with applications, execute workflows, write code, access corporate information and take actions with less human involvement. It is already appearing in coding pipelines, customer-service workflows and internal business tools.

That creates tremendous opportunity, but it also means organizations are giving AI systems something cybersecurity professionals have always treated carefully: access and permissions.

The more autonomy we give AI, the more important it becomes to understand where that autonomy ends.

The Containment Gap

This isn't entirely a theoretical concern. SecureWorld points to two recent incidents: OpenAI systems reportedly breached Hugging Face servers, while Anthropic agents reached systems outside their designated test environments after a third-party evaluation was misconfigured and unintentionally provided access to the internet.

The bigger concern may be what happens afterward.

Research found that few leading AI labs have publicly documented containment-response plans explaining how they would shut down a model that resisted human control. That leaves important questions around response, accountability and independent investigation largely unanswered.

The technology may be new, but the underlying issues aren’t - excessive permissions, misconfiguration, insufficient isolation, monitoring gaps and unclear incident-response processes.

We've been managing these risks in traditional IT environments for decades.

AI Is Becoming a Vendor-Risk Issue

Most small and mid-sized businesses aren't building their own AI models. They're consuming AI through Microsoft platforms, SaaS products, cybersecurity tools, development environments and other third-party technologies.

That makes AI containment a vendor-management and governance issue today, not something to worry about only if superintelligence eventually arrives.

Organizations already ask technology vendors about cybersecurity, privacy, disaster recovery, breach notification and compliance. AI should become part of that same conversation.

Does the vendor have an incident-response plan specifically for AI agents? How is the agent isolated from production systems and the internet? Who investigates a containment failure? What will the vendor disclose when something goes wrong? And perhaps the simplest question of all: Can the system be immediately disabled? 

"The AI wasn't supposed to do that" isn't an incident-response plan.

Governance Doesn't Mean Slowing Down AI

None of this is an argument against adopting AI.

AI presents enormous opportunities for smaller businesses to improve productivity, automate processes, analyze information and gain access to capabilities that previously required significantly greater resources.

But there is a difference between "we can do this" and "we're ready to do this."

Good AI governance should help organizations move faster with confidence by understanding what information an AI system can access, what actions it can perform, how those permissions are controlled and who remains accountable when something goes wrong.

For small and mid-sized organizations, this is where independent technology leadership can be particularly valuable. The vendor can explain what its AI can do. Someone within the organization still needs to determine what it should be allowed to do.

Final Thoughts

We don't need to settle the debate about superintelligence to recognize the immediate issue.

AI systems are being given more access, more capability and increasingly more autonomy. At the same time, the industry's approach to containment and incident response has not necessarily kept pace.

Businesses should embrace the opportunities AI creates, but with the same disciplines they would apply to any other business-critical technology: governance, least privilege, vendor management, monitoring and incident response.

Innovation is important. So is knowing where the OFF switch is.