For a plain-text file, read it one line at a time and rotate to a new output after a chosen number of lines. This keeps memory use low and preserves line endings. If you need byte-sized chunks or must keep CSV or JSON valid, choose a boundary-aware method instead: bytes, physical lines and structured records are not interchangeable.
Choose what defines each part
Before writing code, decide what “split” means for your file. The right boundary depends on whether you need a maximum line count, a byte limit, or complete records in a format such as CSV. Splitting at the wrong boundary can produce parts that are unusable.
- By line count: Suitable for ordinary text when each physical line can be separated independently.
- By byte size: Suitable when the requirement is a precise byte limit. Use binary reads and writes; a byte boundary can cut through a text character or record.
- By structured record: Parse the format and split between records. This matters for CSV, where a quoted field may contain a line break, and for JSON, where arbitrary chunks may not be valid JSON documents.
Split a text file by line count
This example writes up to 1,000 lines per part, naming the outputs part_001.txt, part_002.txt, and so on. Change lines_per_file to the limit you need. It streams the input instead of loading the entire file into memory; Python’s tutorial recommends iterating over a file object to read lines for this reason (Python 3.11 tutorial, reading and writing files).
from pathlib import Path
source = Path("input.txt")
out_dir = Path("parts")
lines_per_file = 1000
out_dir.mkdir(parents=True, exist_ok=True)
part_number = 1
line_count = 0
output = None
try:
with source.open("r", encoding="utf-8", newline="") as src:
for line in src:
if output is None or line_count == lines_per_file:
if output is not None:
output.close()
part_path = out_dir / f"part_{part_number:03}.txt"
output = part_path.open("w", encoding="utf-8", newline="")
part_number += 1
line_count = 0
output.write(line)
line_count += 1
finally:
if output is not None:
output.close()What the code does
Path("input.txt")identifies the source andPath("parts")identifies the output directory.mkdir(parents=True, exist_ok=True)creates that directory if needed.- The source is opened with a context manager, which closes it even if an error occurs. Iterating over
srcprocesses one line at a time. - When the current part reaches the limit, the code closes it and opens the next numbered output. The
finallyblock closes the last output, including if an exception interrupts the loop. - With an empty input, the loop never starts, so no part file is created.
Line endings and overwriting
Opening both files with newline="" avoids newline translation in text mode, so each line’s terminator is written as read. A final line without a newline remains without one. If you want translated newlines instead, choose the newline behavior deliberately rather than assuming every platform will write identical bytes.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →#1 Best Overall
Write mode replaces an existing file with the same name. Use an empty output directory or add a collision check before opening each part if existing data must be preserved. Keeping outputs in a separate directory also reduces the chance that a later batch operation will treat old parts as new source files.
Split CSV without breaking records
Do not generally divide CSV by physical line: a quoted field can contain a line break, so one CSV record may span multiple lines. Use Python’s standard-library csv module to read and write parsed rows (Python CSV documentation). If each output should be usable as a standalone CSV, write the header row to every part, then split the remaining parsed records at the desired row count.
Split by byte size
If each part must stay within a byte limit, use binary mode and read at most that many bytes for each output. This enforces a byte boundary, not a text or record boundary: the cut may land inside a UTF-8 character, a line, or a CSV row. If the parts must remain independently readable, choose a boundary-aware approach and account for the format’s rules rather than treating bytes as characters or records.
Handle JSON and other structured files
First identify the representation. A single JSON document cannot usually be split at arbitrary positions and leave valid JSON in every part; newline-delimited JSON, by contrast, stores separate records on separate lines and can be split between records. More generally, parse the format and write complete units to each output. The Python tutorial documents JSON serialization and file operations, but the correct splitting rule depends on the input structure (Python 3.11 tutorial: Input and Output).
Free tools Windows power users keep installed
One-click scans. No signup required.
Check the resulting parts
After splitting, verify that the outputs match the intended boundary and can be read in the way downstream users expect. For line-based text, check the number of lines in each part and confirm that the final part may be shorter. For CSV or JSON, parse the outputs again and check record or document validity; if headers are required, confirm each part has one. These are verification steps to perform on your files, not results guaranteed by the splitting code.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
