Use the OpenAI Agents SDK’s ComputerTool when the agent needs to control a browser and inspect its screen. You provide the local browser harness: it implements the SDK’s Computer or AsyncComputer interface, including screenshot(), which returns a base64-encoded PNG. Then add ComputerTool to an agent and run it with Runner. The SDK documentation points to a Playwright-based example for the browser implementation.
How the screenshot flow works
ComputerTool connects the agent to a computer or browser runtime that your application provides. The SDK exposes the harness through the computer-use tool interface; it does not itself launch or host your local Playwright browser.
As an Amazon Associate I earn from qualifying purchases.
- Start a browser runtime in your application and open the page to capture.
- Implement the required computer actions for that runtime, including a
screenshot()method. - Construct
ComputerToolwith the implementation and add it to anAgent‘s tools. - Use
Runnerto run the agent with instructions to navigate, interact with, or inspect the page. - Return the screenshot in the format the interface requires: base64-encoded PNG data for
screenshot().
The official guide links to a runnable Playwright-based harness at the Agents SDK computer-use guide. Use that example as the reference for browser setup and interface implementation; the code below shows the integration boundary, not a complete Playwright harness.
Choose the synchronous or asynchronous interface
Match the computer interface to the execution model of your browser driver. Use Computer for a synchronous implementation and AsyncComputer for an asynchronous one. The screenshot method has the same important output contract in either case: a base64-encoded PNG of the current display.
#1 Best Overall
A screenshot alone is not enough for the full computer-use loop. Implement the interaction methods required by the selected interface, such as clicking, scrolling, typing, waiting, and keyboard input. The guide’s Playwright example demonstrates the browser-harness approach; consult it for the exact interface details for your SDK version.
Connect your harness to an agent
The following is an integration sketch for the synchronous path. Replace my_computer with an instance of your own complete Computer implementation, following the SDK example. It is deliberately not presented as a standalone runnable browser program: browser creation, navigation and the required action methods belong to that harness.
from agents import Agent, ComputerTool, Runner
# Implement the SDK Computer interface for your local browser runtime.
# Its screenshot() method must return a base64-encoded PNG string.
my_computer = ...
computer_tool = ComputerTool(computer=my_computer)
agent = Agent(
name="Website screenshot agent",
instructions="Use the browser to open the requested website and inspect its screenshot.",
tools=[computer_tool],
)
result = Runner.run_sync(
agent,
"Open https://example.com and describe what is visible on the page.",
)
print(result.final_output)For an asynchronous browser driver, implement AsyncComputer and follow the asynchronous pattern in the SDK’s computer-use example. Do not substitute an ordinary browser screenshot method without adapting it to the interface’s expected base64 PNG return value.
Check the effective model and tool format
Computer-use request behavior depends on the model actually used for the Responses API request. The current guide distinguishes the documented GA computer tool path, which sends a computer tool payload and can return batched actions[], from the older computer-use-preview path, which uses a computer_use_preview payload and a single action per call.
Check the model after accounting for any override in run configuration or prompt templates. A configured override can change which tool format applies, so do not assume the model named when constructing the agent is necessarily the effective one. Model names, defaults and preview behavior are version-sensitive; verify the current guide and the model used by your application before relying on a specific action shape.
When you need a screenshot, not browser interaction
ComputerTool is the documented choice for an agent that must operate a browser through computer-style actions. If the task is simply to capture a page and return an image or PDF, a screenshot API can avoid building and maintaining that browser harness. ScreenshotNeo is a website screenshot API and MCP server; its MCP tools let compatible AI-agent clients request screenshots directly.
Or skip the browser setup
Make one GET request to capture a page as an image. For this example, the response is saved as a WebP file; consult the ScreenshotNeo API documentation for request options and supported output formats.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpScreenshotNeo accepts cookie or consent banners before capture and removes more than 60 known consent platforms, newsletter popups and chat widgets; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads and cache hits are not billed, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info and capture_pdf tools for Claude, Cursor and other MCP clients. The Free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots. Sign up for ScreenshotNeo’s free plan.
Troubleshoot common integration problems
- The agent cannot take a screenshot: Check that the harness implements the selected
ComputerorAsyncComputerinterface and thatComputerToolreceives that implementation. - The screenshot result is rejected or unusable: Verify that
screenshot()returns base64-encoded PNG data, rather than a file path, raw image bytes, or a different image format. - The agent can capture but cannot interact: Add the other methods required by the interface and route each action to the local browser driver. A screenshot method alone does not implement clicks, scrolling, typing or waiting.
- The request uses an unexpected action format: Check the effective model on the actual request and any run-level or prompt-template override. The GA and preview paths differ in payload naming and whether actions may be batched.
- The example does not match your installed SDK: Compare against the current official computer-use guide and its linked Playwright example. The interface and model behavior can change across SDK versions.
Reliability and implementation costs
Your application owns the browser runtime and its lifecycle, so it must handle browser startup, page loading, navigation and action execution. The guide specifies the harness contract but does not establish a universal runtime setup or performance figure; actual capture time and reliability depend on your browser environment and the sites you visit. For a one-off capture workflow, compare that maintenance with a screenshot API call. For interactive tasks where the agent must act on what it sees, the computer-use loop is the more direct fit.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
