Who Is Actually in Control When an AI Agent Can Operate Your Computer?

An AI agent does not gain power because it sounds confident.

It gains power because software around the model gives it tools, files, credentials and permission to act.

That distinction is the useful way to think about “control.”

Start with the tools

A model that can only read text has very little computer authority.

Add a file-editing tool and it can change files within that tool's scope.

Add a shell and it can invoke programs.

Add browser automation, API tokens or administrator privileges and the consequences expand again.

The first security question is therefore: what tools exist?

Check which operating-system user runs the agent

Operating systems already have a permission model.

If the agent runs as an ordinary user, it inherits what that account can access.

If it runs as root or an administrator, the boundary becomes much wider.

Do not give elevated privilege simply because the agent occasionally encounters a permissions error.

That error may be the permission system doing its job.

Define filesystem scope

Which directories can the agent read?

Which can it write?

A coding agent may only need one project directory.

It usually does not need unrestricted access to every personal file, SSH key, browser profile and backup mounted under the same home directory.

Some agent tools explicitly scope access to the current working directory and ask for permission before leaving it.

That is useful because scope is visible and enforceable.

Check network access

A process with network access can contact remote services, download code or send data away from the machine.

That may be required for the task.

It should still be intentional.

For sensitive workflows, ask whether the agent needs unrestricted outbound access or only access to a small set of services.

Treat secrets as capabilities

An API token is not merely text.

It is permission to do whatever that token authorizes.

If an agent can read a credential file or environment variable, it can potentially exercise that authority through an available tool.

Give task-specific credentials rather than a collection of powerful general-purpose secrets.

Approval prompts are one layer

An approval dialog can put a human between the model's proposed action and execution.

That helps.

It is not the same as removing the permission.

If the agent has another tool or credential that reaches the same target without approval, the control can be bypassed accidentally or by poor workflow design.

Place important gates at the actual action boundary.

Logging creates accountability, not prevention

Record consequential commands, file changes, external actions and approvals where practical.

Logs help explain what happened afterward.

They do not stop a bad action by themselves.

Use logging alongside real permission boundaries.

Containers and sandboxes can narrow the boundary

A container can restrict which host files and processes are visible when it is configured carefully.

A dedicated virtual machine can create another isolation layer.

Neither helps much if the environment is handed broad host mounts, powerful credentials or direct control of the host runtime.

Isolation should be judged by what the agent cannot reach.

The model is only one part of the system

The complete authority chain looks more like:

model → tool → process identity → permissions → credentials → external system

Security controls belong throughout that chain.

Prompt instructions such as “never delete files” are useful behavioral guidance.

They are not a replacement for filesystem permissions that make deletion impossible outside the intended workspace.

For a practical container boundary, see Running Autonomous AI Tools Inside Containers to Limit What They Can Touch. Business approval gates are covered in Where a Small Business Should Require Human Approval Before an AI Agent Acts.