Do not build a scraper for the ChatGPT website. If you need machine-readable answers in an application, call the OpenAI API and request a JSON Schema response (Structured Outputs) on a model and endpoint that support it. Browser-page extraction is a different workflow, and OpenAI’s individual Terms of Use revision dated December 11, 2024 says users may not “Automatically or programmatically extract data or Output (defined below).” Check the agreement that actually governs your account, organization, geography and use before automating any export.
This guide shows the API approach, explains the difference between JSON mode and schema-constrained output, and covers validation, refusals, incomplete responses and common implementation failures.
First decide what “scrape ChatGPT” means
The phrase usually describes one of two jobs:
- Extracting replies from chatgpt.com: reading the consumer website’s rendered pages, browser storage or network traffic and turning those replies into records.
- Requesting structured model output: sending your prompt from your own application and receiving JSON that follows a schema.
They are not interchangeable. Website extraction depends on a changing user interface and account session. The API is the documented integration route for software that needs predictable fields.
Why website scraping is a terms and reliability problem
OpenAI’s Terms of Use, revision December 11, 2024, list among prohibited acts: “You may not … Automatically or programmatically extract data or Output (defined below).” That language applies to the individual terms; business or organizational customers may instead be covered by separate business terms or a services agreement. Those documents use their own wording and scope. Review the current agreement applicable to your account rather than assuming one contract covers every user.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →#1 Best Overall
This article therefore does not provide DOM selectors, session-cookie reuse, reverse-engineering instructions or ways to bypass bot checks and other protections. If you have a specific permitted export workflow, confirm it with the governing terms and your administrator.
Use JSON Schema Structured Outputs for application data
The API response format supports type: "json_schema" with a named schema. With strict mode enabled, the API reference says the model follows the defined schema, subject to the supported subset of JSON Schema. The same reference says, “Using json_schema is preferred for models that support it.” Support varies by model and endpoint, so verify the current API reference before choosing them.
JSON Schema versus older JSON mode
| Property | JSON Schema Structured Outputs | JSON mode |
|---|---|---|
| Format setting | type: "json_schema" plus a schema object |
type: "json_object" |
| Main guarantee | Constrains the response to the supplied, supported schema; strict mode requests exact adherence | Produces valid JSON, but does not enforce your fields or types |
| Prompt requirement | Still describe the task and desired meaning clearly | Your prompt must explicitly instruct the model to generate JSON |
| When to choose | Use when the selected model and endpoint support it and your application depends on a stable shape | Use only when schema support is unavailable or a looser JSON envelope is acceptable |
Valid JSON is not automatically complete, semantically correct or true. A response can satisfy a schema while containing a wrong date, an invented value or an incomplete interpretation. Treat schema adherence as formatting control, not fact-checking.
A minimal JavaScript request
The developer quickstart demonstrates the official JavaScript SDK and reads generated text from response.output_text. The example below follows that pattern while leaving the model in an environment variable because model and endpoint support change.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rank #2
- HTML CSS Design and Build Web Sites
- Comes with secure packaging
- It can be a gift option
- Install the SDK:
npm install openai. - Set
OPENAI_API_KEYand a currently supported model in your environment. - Save this as
structured.mjsand run it with Node.js.
import OpenAI from "openai";
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const model = process.env.OPENAI_MODEL;
if (!model) throw new Error("Set OPENAI_MODEL to a model that supports json_schema");
const response = await client.responses.create({
model,
input: [
{
role: "user",
content: "Extract the product name, price in USD, and whether it is in stock from: The Acme Lamp costs $49.99 and is in stock."
}
],
text: {
format: {
type: "json_schema",
name: "product_record",
strict: true,
schema: {
type: "object",
properties: {
product_name: { type: "string" },
price_usd: { type: "number" },
in_stock: { type: "boolean" }
},
required: ["product_name", "price_usd", "in_stock"],
additionalProperties: false
}
}
}
});
if (!response.output_text) throw new Error("No textual output returned");
const record = JSON.parse(response.output_text);
console.log(record);Expected application data is an object such as {"product_name":"Acme Lamp","price_usd":49.99,"in_stock":true}. Your own parser should still reject unexpected values and handle an API error instead of assuming every request succeeds.
Equivalent Python pattern
The exact SDK method and parameter names can change. Confirm the current Python quickstart and API reference for the model and endpoint you select.
import json
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
model = os.environ["OPENAI_MODEL"]
response = client.responses.create(
model=model,
input="Extract product_name, price_usd, and in_stock from: The Acme Lamp costs $49.99 and is in stock.",
text={
"format": {
"type": "json_schema",
"name": "product_record",
"strict": True,
"schema": {
"type": "object",
"properties": {
"product_name": {"type": "string"},
"price_usd": {"type": "number"},
"in_stock": {"type": "boolean"}
},
"required": ["product_name", "price_usd", "in_stock"],
"additionalProperties": False
}
}
}
)
if not response.output_text:
raise RuntimeError("No textual output returned")
record = json.loads(response.output_text)
print(record)Equivalent cURL request
Use the endpoint and request shape currently documented for your selected API. Keep the schema in the request body and protect the key with an environment variable.
curl https://api.openai.com/v1/responses
-H "Authorization: Bearer $OPENAI_API_KEY"
-H "Content-Type: application/json"
-d @request.json{
"model": "YOUR_SUPPORTED_MODEL",
"input": "Extract product_name, price_usd, and in_stock from: The Acme Lamp costs $49.99 and is in stock.",
"text": {
"format": {
"type": "json_schema",
"name": "product_record",
"strict": true,
"schema": {
"type": "object",
"properties": {
"product_name": {"type": "string"},
"price_usd": {"type": "number"},
"in_stock": {"type": "boolean"}
},
"required": ["product_name", "price_usd", "in_stock"],
"additionalProperties": false
}
}
}
}Do not copy a model name from an old tutorial without checking support. API fields, SDK examples and endpoint capabilities are volatile.
Recommended Free Tools
Rank #3
Design schemas that survive real inputs
Make required fields explicit
List every field the consumer needs in required and set additionalProperties to false when your integration expects a fixed contract. Use numbers for numeric calculations, booleans for flags and arrays for repeated records instead of asking the model to encode everything as prose.
Represent uncertainty deliberately
If a source may omit a value, model that case explicitly (for example, a nullable field or a status such as unknown) rather than forcing the model to invent a value. Explain the rule in the prompt and enforce it again in application code.
Separate extraction from interpretation
Give the model the text to extract and define normalization rules, such as currency handling or date format. Keep business decisions—eligibility, authorization, financial actions—in deterministic code after parsing.
Validate more than JSON syntax
- Parse the returned text and reject malformed JSON.
- Validate the object against the same schema or a runtime validator in your language.
- Apply business rules: ranges, allowed enumerations, required relationships and maximum lengths.
- Detect refusals, missing output and incomplete responses before storing a record.
- For consequential decisions, compare against source text and add human review. OpenAI’s terms say users should not rely on Output as the sole source of truth and should evaluate accuracy and appropriateness.
Streaming can improve time-to-first-token, but it adds assembly logic: buffer the complete structured result, then parse and validate it. Never expose partially assembled JSON to a downstream system as if it were final.
Rank #4
- Brand: Wiley
- Set of 2 Volumes
- A handy two-book set that uniquely combines related technologies Highly visual format and accessible language makes these books highly effective learning tools Perfect for beginning web designers and front-end developers
Troubleshooting
The API rejects the response format
Cause: the chosen model or endpoint does not support JSON Schema, or the schema uses an unsupported keyword. Fix: check the current API reference, simplify to its supported JSON Schema subset, or use JSON mode only when a looser contract is acceptable.
The result is valid JSON but fields are missing
Cause: JSON mode was used, fields were not required, or the request was truncated. Fix: use strict JSON Schema where supported, mark required fields, inspect completion status and reject failed validation.
Values are plausible but wrong
Cause: formatting does not establish factual accuracy. Fix: provide authoritative source text, constrain transformations, run deterministic checks and route high-impact cases to review.
No output text is available
Cause: refusal, incomplete generation, an API error or a response containing non-text items. Fix: log the request identifier and status, handle refusal and incomplete cases separately, and retry only according to your reliability policy.
Your scraper stops working after a website change
Cause: browser markup and interaction flows are private implementation details and can change without notice. Fix: stop automating the ChatGPT site; obtain permission for any export and move an application workflow to the API.
Cost, performance and reliability decisions
- Keep prompts and source documents as small as practical; send only the text needed for extraction.
- Use bounded input sizes, timeouts, retry limits and exponential backoff for transient failures.
- Make writes idempotent so a retry cannot duplicate a record.
- Record model, schema version, request status and validation errors for debugging.
- Pin and test SDK versions, but still monitor the current API documentation for changed capabilities.
- Evaluate representative difficult cases, not just clean examples; schema compliance alone is not an accuracy benchmark.
Or skip the browser setup
If your documentation or QA workflow also needs a screenshot of a public page, ScreenshotNeo provides a one-call website screenshot API and MCP server. It removes cookie banners, newsletter popups and chat widgets before capture; bot checks, blank pages, failed loads and cache hits are not billed; and AI agents can call its MCP tools. The free plan includes 1,000 screenshots a month with no card, and paid plans start at $5 for 3,000 shots.
See the ScreenshotNeo API documentation for options and current request details.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpCreate an account at ScreenshotNeo’s free sign-up page to use the 1,000 free monthly screenshots.
The practical decision rule
Use the API when you are building software that needs structured model responses. Use JSON Schema Structured Outputs on a supported model, then validate semantics and handle refusals or incomplete results. Do not treat a browser scraper as an equivalent integration, and check the current agreement that governs any permitted export from the ChatGPT service.
Frequently Asked Questions
Can I make ChatGPT return JSON?
Yes. In an application, request JSON Schema Structured Outputs through the OpenAI API when your chosen model and endpoint support it. JSON mode is a less restrictive fallback.
Does strict schema output guarantee truthful answers?
No. It constrains structure, not factual accuracy. Validate against source data and apply human review where the consequences justify it.
Are the individual Terms of Use the only agreement that matters?
No. Business customers and organizations may have separate business terms or a services agreement. Check the agreement applicable to your account and use.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
