Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Use curl’s built-in --parallel option (or its short form, -Z) to run multiple transfers concurrently in one invocation. Give each response its own output destination, and set --parallel-max when you need to cap the number of transfers running at once.
For example, this downloads two pages concurrently:
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
Dan Gookin's Guide to Curl Programming | $11.95 | Buy on Amazon |
| 2 |
|
Curly Girl: The Handbook | $8.19 | Buy on Amazon |
| 3 |
|
The C Programming Language | $10.01 | Buy on Amazon |
| 4 |
|
Curl by Example | $0.99 | Buy on Amazon |
| 5 |
|
A Practical Guide to Curl (Programming Series) | $24.99 | Buy on Amazon |
curl --parallel
https://example.com/one -o one.html
https://example.com/two -o two.htmlRun a fixed set of URLs with curl
Put --parallel before the URLs, then assign each response a distinct output file with -o. Curl schedules the transfers concurrently and starts further transfers as earlier ones finish. Merely listing several URLs does not request parallel operation; add --parallel when concurrency is what you want. See the curl man page for the option’s current documented behavior.
Recommended Free Tools
curl --parallel
https://api.example.test/a -o a.json
https://api.example.test/b -o b.json
https://api.example.test/c -o c.jsonReplace the example host and paths with the endpoints you are authorized to access. Naming each output makes it clear which response belongs to which URL and avoids accidentally directing multiple bodies to one file.
#1 Best Overall
Limit how many transfers run at once
Use --parallel-max N to cap the number of simultaneous transfers:
curl --parallel --parallel-max 4
https://api.example.test/a -o a.json
https://api.example.test/b -o b.json
https://api.example.test/c -o c.json
https://api.example.test/d -o d.json
https://api.example.test/e -o e.jsonThe curl project’s current online manual documents a default maximum of 50 simultaneous transfers when parallel mode is used, and a supported maximum setting of 65,535. Those are configuration limits, not sensible targets for every batch. Choose a lower value if the service has rate or concurrency policies, or if local memory, sockets, or bandwidth are constrained. The manual also describes --parallel-max as global across the command line, including option groups separated by --next.
Handle requests with different methods or data
curl options apply to the URL group they precede. Use --next to end one group and start another, resetting method and POST-data state for the next group. Add --parallel separately if the groups should run concurrently. The curl HTTP scripting guide explains how multiple URLs and --next work.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
curl --parallel
-I https://example.test/status
--next
-d 'name=curl' https://example.test/submitThis example sends a HEAD request to the first URL and a form-encoded POST to the second. It is a command structure to adapt, not a recommendation to submit arbitrary data to a real endpoint. Supply the method, headers, and body each endpoint expects, and consider where response bodies should go if you need to retain them.
Be deliberate with redirects and custom methods
If you combine --request with --location, review the redirect behavior before relying on it. The current curl man page warns that a method string specified with --request is used for all requests; depending on redirect status and curl’s method handling, that can produce unintended follow-up requests. Check the endpoint’s redirect behavior and the exact command rather than assuming the method will change as desired.
Rank #2
Download a generated list with GNU Parallel
For a list of URLs, or when each item needs its own curl process and custom shell work, GNU Parallel can read jobs from standard input. This example assigns a numbered output filename to each item:
printf '%sn' 'https://example.test/a' 'https://example.test/b' |
parallel -j 4 curl -fsS {} -o '{#}.body'Here, -j 4 limits GNU Parallel to four jobs at a time, {} stands for the current input item, and {#} is its job number. The flags -f and -sS ask curl to fail on HTTP errors and keep normal progress output quiet while still showing errors. The GNU Parallel documentation describes URL lists as a typical input and documents its job controls.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallUse this as a starting pattern, not a safe parser for every possible URL list. If URLs come from a file or another program, make sure its format and quoting are suitable for the shell and GNU Parallel. Choose output names that cannot collide. GNU Parallel can preserve output ordering, but ordered display does not make concurrent writes to the same file safe.
Choose between curl parallel mode and GNU Parallel
| Approach | Best fit | Concurrency control | Trade-off |
|---|---|---|---|
curl --parallel |
A fixed group of transfers managed in one curl process | --parallel-max; a per-host cap is available on supporting versions |
Compact command and curl-managed transfer scheduling, but less room for per-URL shell logic |
| GNU Parallel with curl | A URL list, per-item command template, or job-by-job shell workflow | GNU Parallel’s job count, such as -j 4 |
More flexible orchestration, with separate processes and additional quoting and output-handling concerns |
Decide based on whether the URL set is fixed or generated, whether requests share options and data, whether each item needs extra shell processing, how you want to name and log outputs, and which tool versions are installed. Neither approach is universally faster; the choice here is about workflow and control.
Check curl version and connection options
Confirm the executable and version available in your environment:
Rank #3
curl --versionThe curl man page records --parallel as added in 7.66.0 and --parallel-immediate in 7.68.0. The latter asks curl to prefer opening additional connections immediately instead of waiting to multiplex more streams over an existing connection. It can change connection behavior, so use it only when that is what you intend.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsThe per-host option --parallel-max-host was added in 8.16.0. The current manual says it limits concurrent connections to the same protocol, hostname, and port, with a default of 0 (unlimited). For compatibility-sensitive scripts, also check curl --help all or the installed manual; not every machine has the same curl release as the online documentation.
Performance, reliability, and cost considerations
Concurrency can reduce waiting, but is not a guarantee of faster completion
Parallel transfers overlap their work, which can help when requests are independent and time is otherwise spent waiting on separate responses. Actual completion time depends on factors such as the remote server, network path, response sizes, and local capacity. Adding more transfers can also increase contention or trigger service-side limits. There is no universal concurrency setting that is right for every host.
Respect the destination service
- Start with a modest explicit cap for a large batch, then adjust to the service’s documented policy and your operational needs.
- Do not assume that the documented default of 50 is approved by a particular API or website.
- Keep each output destination unique, and choose filenames that remain unique if a batch can overlap with another run.
- For important workflows, inspect exit status and error output and decide how your script should handle partial success; concurrent requests can have different outcomes.
Estimate what you are actually saving
Parallelism changes when transfers run, not the amount a service charges for them. If an API bills per request, sending the same requests concurrently does not by itself reduce the request count. Check the provider’s own terms for rate limits, quotas, and billing. The curl and GNU Parallel documentation cited here establishes command behavior, not service-specific limits or pricing.
Troubleshooting parallel curl commands
“Unknown option” for --parallel
Your installed curl may be older than 7.66.0, the documented introduction version. Check curl --version and whether another curl executable is earlier in your PATH. Use a version that supports the option, or use an available shell job tool such as GNU Parallel if installing or upgrading curl is not appropriate.
Rank #4
Files are missing, overwritten, or contain the wrong response
Check each URL’s -o destination. Reusing a filename makes responses compete for the same path; use a distinct name for every transfer. With a generated URL list, make sure the filename scheme is unique across all jobs and across overlapping runs.
Some requests fail while others succeed
Parallel mode schedules independent transfers; it does not make them a transaction. Inspect curl’s error output and the HTTP results for each endpoint, then retry only what is appropriate for that service. Be cautious with non-idempotent requests: blindly repeating a POST after an ambiguous failure can duplicate an action.
The destination returns errors or slows down
Reduce --parallel-max or GNU Parallel’s -j value, and check the service’s documented concurrency and rate policies. A concurrency cap controls simultaneous work; it is not a guarantee that requests will be accepted or that a rate limit will not be reached.
A mixed-method command behaves unexpectedly after a redirect
Review the combination of --request, --location, and --next. In particular, inspect curl’s documented behavior for the installed version when a custom method is carried through redirects. Test against a controlled endpoint before using such a command on a production workflow.
Free tools Windows power users keep installed
One-click scans. No signup required.
Or skip the browser setup
If the batch you need is website screenshots rather than arbitrary HTTP transfers, ScreenshotNeo is a screenshot API and MCP server for developers. Its one-request pattern returns a screenshot of a URL; this example captures one page, not a parallel batch:
Best Value
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo documentation for request parameters. Before capture, it accepts the cookie or consent banner like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each cleanup step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers identify the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients.
The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Sign up for the free plan.
Frequently asked questions
Can parallel requests finish in a different order than the URLs appear?
Yes. Transfers run concurrently, so completion order is not a dependable way to associate a response with an input URL. Use explicit, distinct output filenames or another mapping that ties each result to its request.
Does --parallel-max-host limit requests per domain name alone?
The current curl manual describes its scope as the same protocol, hostname, and port, rather than hostname alone. It is available in curl 8.16.0 and later according to that manual.
Should I use GNU Parallel for every batch?
No. For a small fixed set of straightforward transfers, curl’s own parallel mode is usually the simpler command. GNU Parallel is useful when a generated list or per-item shell workflow needs job-level orchestration.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
