Computer Use

Beta

Full desktop control for workflows that no API or browser can reach.

Desktop control is the highest-risk capability in the platform. It is opt-in per workspace, gated per task, and always recorded.

Perception and action#

screenshot ─▶ visual parse ─▶ element hypotheses ─▶ action plan
     ▲                                                    │
     │                                                    ▼
     └──────────── post-action verification ◀──── click / type / key

Each action is followed by a verification observation: the agent confirms the interface actually changed as expected before continuing. Unverified actions are not chained.

Guardrails#

Guardrail	Behaviour
Application allowlist	Only listed applications may be focused or launched
Screen region limits	Optional restriction to a region of the display
Destructive action gate	Deletes, purchases, sends and installs require approval
Credential entry	Handled by the credential broker; passwords never enter model context
Session recording	Frames and actions retained for the audit window
Idle termination	Session closes when a step exceeds its wall clock budget

await pimsy.tasks.create({
  objective: class="tok-str">"Export last quarter's ledger from the desktop accounting client to CSV.",
  capabilities: [class="tok-str">"computer.gui", class="tok-str">"files.write"],
  computer: {
    applicationAllowlist: [class="tok-str">"LedgerPro"],
    approvals: [class="tok-str">"file.delete", class="tok-str">"settings.change"],
    recordSession: true
  }
});

Last updated 2026-09-09