Driver FixRecommendedSound, Wi-Fi or graphics acting up? Check drivers firstFind missing or outdated drivers fast.Check DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan Now×
Skip to content
World desk5 min

Streaming Large File Uploads in Java Without Killing Your Server

A Java upload is only memory-bounded if every layer is. Learn how request staging, stream length, multipart buffers, concurrency, and AWS transfer options affect heap and disk use.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Keep the complete upload out of the Java heap. Use bounded buffers from request intake through storage, and check every layer for hidden buffering: the servlet container, application code, storage SDK, and concurrent multipart transfers. A MultipartFile or InputStream does not by itself guarantee a memory-bounded upload.

Trace the entire upload path before choosing an API

Large-upload memory use is determined by the whole pipeline, not by the type of object passed between two methods. Inspect inbound request handling and outbound storage separately.

Inbound: request parsing and staging

Check whether your servlet container keeps multipart request data in memory, writes it to a temporary directory, or switches from memory to disk at a configured threshold. Spring Boot’s 2.1.2 reference documents configurable multipart temporary storage and a disk-flush threshold, but that older reference does not establish defaults for current Spring Boot releases. Verify the behavior and configuration for the exact Spring Boot and servlet-container versions you deploy. See the Spring Boot 2.1.2 reference.

Disk staging can reduce heap pressure, but it moves the capacity requirement to temporary storage. Account for the maximum simultaneous staged uploads, available disk space, permissions, and cleanup of abandoned files. Do not assume that a file abstraction means the request never occupied memory.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
#1 Best Overall
Sandisk 2TB Extreme Portable SSD, Up to 1050MB/s, USB-C, USB 3.2 Gen 2, IP65 Water and Dust Resistance, Updated Firmware, External Solid State Drive, SDSSDE61-2T00-G25
  • Get NVMe solid state performance with up to 1050MB/s read and 1000MB/s write speeds in a portable, high-capacity drive(1) (Based on internal testing; performance may be lower depending on host device & other factors. 1MB=1,000,000 bytes.)
  • Up to 3-meter drop protection and IP65 water and dust resistance mean this tough drive can take a beating(3) (Previously rated for 2-meter drop protection and IP55 rating. Now qualified for the higher, stated specs.)
  • Use the handy carabiner loop to secure it to your belt loop or backpack for extra peace of mind.
  • Help keep private content private with the included password protection featuring 256‐bit AES hardware encryption.(3)
  • Easily manage files and automatically free up space with the SanDisk Memory Zone app.(5). Non-Operating Temperature -20°C to 85°C

Outbound: application and storage code

Look for whole-file materialization such as byte[], copying a stream into an in-memory buffer, or handing an unknown-length stream to an SDK that must determine its content length. Also check for repeated copies and buffers retained by multipart uploads. A bounded buffer in your own loop is not enough if another layer retains the full object or several parts at once.

Use a known length for a synchronous stream when you can

AWS SDK for Java 2.x warns that synchronous S3 upload of an InputStream with unknown length can buffer the entire stream in memory to calculate content length. AWS’s documentation states: “Because the SDK buffers the entire stream in memory to calculate the content length, you can run into memory issues with large streams.” If the source can provide the exact length, supply it. The length must be accurate: AWS warns that a value smaller than the actual stream can truncate the object, while a value that is too large can fail or hang the connection. See AWS SDK for Java 2.x stream upload guidance.

Rank #2
Sandisk 1TB Portable SSD, Up to 800MB/s Read Speeds, Black (Old Model)
  • Solid state performance with up to 800MB/s read speeds in a portable drive. (Based on internal testing; performance may be lower depending on host device, interface, usage conditions and other factors. 1MB=1,000,000 bytes.)
  • Back up your content and memories on a storage solution that fits seamlessly into your mobile lifestyle.
  • Take it with you on your adventures—up to two-meter drop protection means this durable drive can take a beating. (Based on internal testing.)
  • Secure it to your belt loop or backpack for extra peace of mind thanks to the tough rubber hook.
  • From Sandisk, a brand professional photographers trust to take on assignments.

If the stream length is unknown and the object may be large, do not assume a single synchronous putObject call will remain constant-memory. Choose an upload path designed for multipart transfer or stage the content to a file so its length is known, weighing the extra disk use against memory pressure.

Choose between staging, single-request, and multipart paths

Approach What it helps with Main costs and cautions
Container multipart staging Can avoid keeping all request data in heap when the container writes parts to disk. Uses temporary disk; behavior and thresholds depend on framework and container versions.
Known-length synchronous stream Simple single-request path when the source length is known and correct. Unknown-length AWS SDK for Java 2.x synchronous streams may be buffered in memory; incorrect length risks truncation, failure, or a hung connection.
Multipart upload Transfers a large object in parts, which can be retried or sent in parallel. Requires extra API calls and a budget for part size, in-flight parts, and retained buffers.
File-backed CRT upload AWS documents direct disk streaming for large disk-based uploads, avoiding intermediate part buffering. Still depends on disk capacity and cleanup; behavior is specific to AWS CRT integration.
Sequential streaming multipart provider Supports incremental writes with a stated bounded part-buffer model in one AWS Labs provider. Requires that provider and AWS CRT; its sequential-write behavior and defaults are not universal Java or SDK behavior.

Budget multipart size and concurrency together

For Amazon S3, AWS documents a maximum of 5 GB for a single PUT and multipart uploads for objects up to 50 TB. These are S3 service limits, not Java limits. Parts can be uploaded independently, in any order, and in parallel. Multipart can improve recovery options and may improve throughput for some workloads, but it adds API calls; AWS’s Java multipart configuration guidance advises a single connection for small objects. Do not treat one object size as a universal point at which multipart becomes worthwhile. Consider object sizes, retry needs, network latency, throughput, and the memory and disk available under deployment concurrency. See S3 upload options and S3 multipart upload documentation.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Rank #3
SSK Portable SSD 500GB External Solid State Hard Drive USB C Up to 1050MB/s
  • Capacity Display Variance: 500GB external ssd often appears as around 465GB on Windows. MacOS can show full 500 GB capacity. This is binary calculation difference and doesn’t affect SSD hard drive actual physical storage
  • 1050 MB/s Speed: Instantly access to your files with blazing-fast 10Gbps external SSD read up to 1050MB/s and write up to 1000MB/s. LED Light indicates USB SSD instant activity
  • Data Security: Solid state drives S.M.A.R.T. health diagnostics​ and adaptive TRIM optimizing data block management ensures consistent write speeds and extends the longevity of the portable SSD
  • USB-C & USB-A Cable: Both cables featuring rapid USB 3.2 Gen2, this USB SSD effortlessly bridges devices, enabling seamless cross-platform file transfers and backup between computers, smartphones, tablets and iPhone
  • Always Fast: No slowdowns for large file transfers. With SLC caching (25% of current available capacity allocated as high-speed cache), this external SSD delivers steady 10Gbps for transfers within the cache capacity

Part size and simultaneous uploads multiply the buffering footprint. The AWS Java multipart configuration API exposes a multipart threshold, minimum part size, and API-call buffer size; effective part payload may need to grow to stay within the maximum number of parts. Its reference lists 8 MiB as the default minimumPartSizeInBytes. Treat that as the cited API reference’s default, not a universal setting, and verify the exact semantics and defaults for your SDK version. See the AWS Java multipart configuration reference.

Before raising concurrency, estimate how many parts can be resident per upload and how many uploads can run at once. Leave heap headroom for request parsing, application objects, SDK bookkeeping, and unrelated server traffic. A configuration that is safe for one upload may not be safe when many arrive together.

Rank #4
Sale
Seagate 2TB Portable Hard Drive | USB 3.0 (STGX2000400)
  • Easily store and access 2TB to content on the go with the Seagate Portable Drive, a USB external hard drive
  • Designed to work with Windows or Mac computers, this external hard drive makes backup a snap just drag and drop
  • To get set up, connect the portable hard drive to a computer for automatic recognition no software required
  • This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable
  • The available storage capacity may vary.

Consider AWS CRT for file-backed transfers

AWS’s S3 documentation says the CRT-based client automatically switches uploads from disk to direct disk streaming rather than intermediate part buffering for large objects; the documented Java SDK option can also enable that behavior for smaller files. This can reduce intermediate buffering for file-backed sources, but it does not remove the need to budget temporary disk. For streams that originate in memory, CRT may buffer each part, so memory can still constrain throughput. AWS identifies the Java Transfer Manager on the CRT-based client as a multipart option above a threshold. Consult the AWS large-file Java SDK guide and confirm the behavior for the client and SDK versions you use.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Use sequential streaming multipart only when its constraints fit

AWS Labs’ Java NIO.2 S3 provider documents a sequential streaming multipart mode that requires the AWS CRT client. Its documentation gives defaults of 8 MiB per part and four in-flight uploads, with approximate memory use of (maxInFlight + 1) × partSize—about 40 MiB under those stated defaults. Those figures describe this provider’s documented configuration, not a general Java upload footprint or a guarantee for every release. Check the provider’s release and configuration before relying on them. See the AWS Labs Java NIO.2 S3 provider documentation.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Best Value
Sale
Samsung T7 Portable SSD 1TB Titan Gray, USB 3.2 Gen 2, Up to 1,050MB/s
  • MADE FOR THE MAKERS: Create; Explore; Store; The T7 Portable SSD delivers fast speeds and durable features to back up any endeavor; Build your video editing empire, file your photographs or back up your blogs all in an instant
  • SHARE IDEAS IN A FLASH: Don’t waste a second waiting and spend more time doing; The T7 is embedded with PCIe NVMe technology that brings fast read and write speeds up to 1,050/1,000 MB/s¹, making it almost twice as fast as the T5
  • ALWAYS MAKE THE SAVE: Compact design with massive capacity; With capacities up to 4TB, save exactly what you need to your drive – from large working files to game data and everything in between
  • ADAPTS TO EVERY NEED: Whether using a PC or mobile phone, count on the T7 for extensive compatibility²; It’s a true team player when it comes to heavy-duty application usage or file-saving
  • HI RESOLUTION VIDEO RECORDING: Record Ultra High Resolution (4K 60fs) videos directly onto the T7 Portable SSD with your favorite camera or mobile devices; Supports iPhone 15 Pro Res 4K at 60fps video and more³

The provider describes this mode for sequential large writes. Random backward seeks trigger fallback behavior; if fallback is enabled, it retains all written data in memory so it can reconstruct the content. It is therefore a poor fit for code that may seek backward unless that fallback and its memory cost are intentional.

Quick Recap

Bestseller No. 2
Sandisk 1TB Portable SSD, Up to 800MB/s Read Speeds, Black (Old Model)
Sandisk 1TB Portable SSD, Up to 800MB/s Read Speeds, Black (Old Model)
From Sandisk, a brand professional photographers trust to take on assignments.
$188.90
SaleBestseller No. 4
Seagate 2TB Portable Hard Drive | USB 3.0 (STGX2000400)
Seagate 2TB Portable Hard Drive | USB 3.0 (STGX2000400)
This USB drive provides plug and play simplicity with the included 18 inch USB 3.0 cable; The available storage capacity may vary.
$119.99

Operational checks before enabling large uploads

  • Confirm where inbound multipart data is held and which temporary directory receives staged content.
  • Check the outbound path for byte-array conversion, unknown-length synchronous streams, duplicate copies, and SDK buffering.
  • Set explicit limits for upload size, simultaneous requests, multipart part size, and in-flight parts.
  • Include temporary disk and cleanup in capacity planning; monitor both disk pressure and heap under realistic concurrent load.
  • Verify SDK, CRT, framework, and servlet-container settings against the versions actually deployed rather than copying another version’s defaults.
  • Test failure and retry paths, including interrupted uploads and abandoned multipart work, without assuming that a successful small-file test predicts large-file memory use.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.