Hardware FixRecommendedDevice not working? Your driver may be the problemCheck updates for common hardware issues.Fix DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix Now×
Skip to content
.NET

Making Concurrent HTTP Requests in C#

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

For a small, known batch of HTTP calls, start the requests and await them with Task.WhenAll. For an enumerable workload that must not exceed a chosen level of parallelism, use Parallel.ForEachAsync. In both cases, reuse HttpClient (or use IHttpClientFactory), pass cancellation tokens, bound work to the service’s capacity, and configure retries and timeouts for the operation’s safety.

Choose the coordination pattern first

Situation Best starting point Why
A finite set of URLs is already known Task.WhenAll Starts each operation and waits for the whole group.
A large or streaming collection needs a cap Parallel.ForEachAsync Processes items asynchronously with an explicit maximum degree of parallelism.
The service limits requests per time window A rate limiter Controls throughput, which is different from merely limiting in-flight work.

What concurrency does not mean

Concurrent tasks are not guaranteed to run on separate threads, and more simultaneous requests do not automatically produce higher throughput. The remote service, network, connection pool, CPU, and its rate policy determine a useful bound.

Run a finite batch with Task.WhenAll

This complete example reuses one client, applies a cancellation deadline, disposes responses after reading them, and treats non-success status codes as failures.

using System.Net.Http;

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));
using var client = new HttpClient
{
    Timeout = Timeout.InfiniteTimeSpan
};

var urls = new[]
{
    "https://example.com/one",
    "https://example.com/two",
    "https://example.com/three"
};

var tasks = urls.Select(url => DownloadAsync(client, url, cts.Token)).ToArray();

try
{
    string[] bodies = await Task.WhenAll(tasks);
    foreach (var body in bodies)
        Console.WriteLine(body.Length);
}
catch (OperationCanceledException) when (cts.IsCancellationRequested)
{
    Console.Error.WriteLine("The batch was cancelled or exceeded its deadline.");
}
catch (Exception ex)
{
    Console.Error.WriteLine($"At least one request failed: {ex.Message}");
}

static async Task<string> DownloadAsync(HttpClient client, string url, CancellationToken token)
{
    using HttpResponseMessage response = await client.GetAsync(
        url, HttpCompletionOption.ResponseHeadersRead, token);
    response.EnsureSuccessStatusCode();
    return await response.Content.ReadAsStringAsync(token);
}

Task.WhenAll does not itself create requests; the tasks are started when GetAsync is called. It completes only after every supplied task finishes. If one or more tasks fail, awaiting the combined task throws; inspect individual tasks when you need per-URL outcomes rather than a single batch failure.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Keep successful and failed results together

var jobs = urls.Select(async url =>
{
    try
    {
        using var response = await client.GetAsync(url, cts.Token);
        string text = await response.Content.ReadAsStringAsync(cts.Token);
        return (url, Ok: response.IsSuccessStatusCode, Status: (int)response.StatusCode, Text: text);
    }
    catch (Exception error) when (error is HttpRequestException or TaskCanceledException)
    {
        return (url, Ok: false, Status: 0, Text: error.Message);
    }
}).ToArray();

var results = await Task.WhenAll(jobs);

Use this shape when one failed URL should not hide the status of the others. Do not catch every exception indiscriminately: programming errors should normally surface.

Bound a collection with Parallel.ForEachAsync

For thousands of URLs, creating one task per item can create excessive pressure even if the requests are asynchronous. Set a deliberate maximum and use the same client for every iteration.

using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(20) };
var options = new ParallelOptions
{
    MaxDegreeOfParallelism = 8,
    CancellationToken = cancellationToken
};

await Parallel.ForEachAsync(urls, options, async (url, token) =>
{
    using HttpResponseMessage response = await client.GetAsync(url, token);
    response.EnsureSuccessStatusCode();
    string body = await response.Content.ReadAsStringAsync(token);
    Console.WriteLine($"{url}: {body.Length} bytes");
});

Choose the degree from the dependency’s documented limits and observed behavior, then adjust carefully. A value of 8 is only an example, not a universal optimum. If results must be retained, write them to a thread-safe collection or an external sink; do not assume completion order matches input order.

When a fixed batch is preferable

Task.WhenAll is simpler when the complete set is small and known. It preserves the task-array order in the returned result array, which is useful when each result maps directly to its input. It does not provide a concurrency cap by itself.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Reuse HttpClient correctly

Long-lived client

Each HttpClient has its own connection pool. Repeatedly constructing and disposing clients can leave many connections in the system and, at high rates, contribute to port exhaustion. A long-lived client avoids that churn. Configure a connection lifetime when DNS or network topology may change:

var handler = new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(5)
};
var client = new HttpClient(handler);

The often-shown 15-minute value is an illustrative documentation example, not a general recommendation. Select a lifetime that matches expected DNS changes and deployment behavior. DNS is resolved when a connection is created; HttpClient does not follow DNS record TTLs automatically.

IHttpClientFactory

In ASP.NET Core or a dependency-injection application, IHttpClientFactory creates clients while pooling handlers and centralizing configuration:

builder.Services.AddHttpClient("catalog", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
    client.Timeout = TimeSpan.FromSeconds(20);
});

// Inject IHttpClientFactory where it is needed:
HttpClient client = factory.CreateClient("catalog");

Factory-managed handlers have a cookie caveat: pooled handlers can share CookieContainer state, and recycling can discard stored cookies. If cookies represent user session state, evaluate isolation requirements before choosing this pattern.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Limit in-flight work and request rate separately

A concurrency limit controls how many requests are active at once. A rate limit controls how many requests are admitted during a period. A service may require both.

Concurrency limiter

Use Parallel.ForEachAsync for a collection, or a semaphore around each request when you need a reusable gate:

using var gate = new SemaphoreSlim(8, 8);

async Task<string> GetLimitedAsync(string url, CancellationToken token)
{
    await gate.WaitAsync(token);
    try
    {
        using var response = await client.GetAsync(url, token);
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStringAsync(token);
    }
    finally
    {
        gate.Release();
    }
}

Requests-per-window limiter

For a quota such as requests per minute, use a token-bucket, fixed-window, sliding-window, or partitioned limiter. Partitioning can give each tenant or API key its own budget. Do not substitute a maximum in-flight count for a time-window quota.

A Microsoft example uses a 1,000-per-minute database illustration and another sample uses a token limit of 8, a queue of 3, and two tokens per millisecond. Those are configuration examples, not performance guarantees. The standard resilience handler documentation also lists a default of 1,000 permits and a zero-length queue; inspect and tune such defaults for your dependency rather than adopting them blindly.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Return useful overload signals

A custom DelegatingHandler can reject work with HTTP 429 when no permit is available and include Retry-After. A queue of zero fails fast; a queue can smooth bursts but increases waiting and memory use.

Timeouts, cancellation, retries, and HTTP semantics

Pass a cancellation token through every asynchronous call. Use a total operation deadline and, where needed, a shorter per-attempt timeout. Treat cancellation separately from a server error so callers can decide whether to abandon the remaining batch.

Microsoft’s documented standard resilience pipeline includes a 30-second total timeout, a 10-second attempt timeout, three retries with exponential backoff and jitter, a circuit breaker, and a rate limiter. These are version-sensitive defaults. Retries cover transient cases such as 408, 429, server errors, and selected exceptions, but retrying increases load during an outage.

Do not blindly retry state changes

Repeating a POST can duplicate an effect. Retry only when the operation is idempotent or the API supplies an idempotency key and documented retry behavior. Honor a server-provided Retry-After, and cap total elapsed time.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Dispose what you own

Dispose each HttpResponseMessage after consuming its content. Keep a shared client alive; disposing a response returns its connection for reuse. For large downloads, stream to a file instead of buffering the entire body.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Troubleshooting concurrent C# requests

Symptom Likely cause Fix
Port exhaustion or many sockets New client and handler per request Reuse a long-lived client or use IHttpClientFactory.
429 responses Concurrency or rate exceeds the service quota Lower the bound, add the appropriate limiter, and honor Retry-After.
Requests hang indefinitely No effective deadline or cancellation Set a timeout/deadline and pass the token to every operation.
DNS changes are ignored Existing pooled connections remain open Configure PooledConnectionLifetime or handler rotation.
Cookies appear shared or disappear Pooled factory handlers and cookie containers Review cookie isolation and handler lifetime; use dedicated handling when required.
One failure hides other outcomes Awaiting only the combined exception Capture a per-item result tuple or inspect each task after completion.
Server data is duplicated Unsafe operation was retried Disable retries for that method or use an idempotency mechanism.

Or skip the browser setup

If your concurrent workload is taking website screenshots rather than API data, ScreenshotNeo provides a single HTTP endpoint and an MCP server for AI agents. It accepts consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be disabled. Bot checks, CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers identify the page verdict and billing status.

One call returns PNG, JPEG, WebP, or PDF:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

See the ScreenshotNeo API documentation for options such as full-page lazy-image loading, CSS-selector elements, device presets, retina scale, PDF paper settings, custom JavaScript and CSS, waits, blocked resources, headers, cookies, geolocation, caching, signed links, asynchronous webhooks, bulk calls, and usage reporting.

Python

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const data = Buffer.from(await res.arrayBuffer());
require('fs').writeFileSync('shot.webp', data);

The MCP server exposes take_screenshot, get_page_info, and capture_pdf to Claude, Cursor, and other MCP clients. The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Practical checklist

  • Use Task.WhenAll for a finite batch; use Parallel.ForEachAsync for bounded collection processing.
  • Reuse clients and choose DNS-aware connection lifetimes.
  • Set concurrency and rate limits from the dependency’s policy.
  • Propagate cancellation, dispose responses, and set explicit deadlines.
  • Retry only operations whose side effects are safe to repeat.
  • Record per-request status, latency, cancellation, and limiter decisions so production tuning is evidence-based.

Frequently Asked Questions

Does Task.WhenAll make HTTP requests faster?

It overlaps waiting for independent operations; it cannot overcome the server, network, or quota limits, and excessive concurrency can reduce throughput.

Can I use Parallel.ForEachAsync with an infinite stream?

It is intended for an enumerable. For continuously arriving work, feed a bounded channel or queue and apply an explicit worker and cancellation design.

Should every request have its own HttpClient?

No. Reuse a client or obtain clients from IHttpClientFactory; per-request construction can waste connections and exhaust ports.

What concurrency number should I choose?

Start below the service’s documented limit, measure latency and 429 responses, and increase gradually only when the dependency supports it.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Read next

Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.