A JSON Lines file is split into records on the LF character (U+000A), not on every character that Unicode treats as a line break. If a record contains a raw U+2028 LINE SEPARATOR or U+0085 NEXT LINE, a tool that splits on Unicode line boundaries can cut that record in two. The JSON parser then rejects one or both fragments, even though the original JSON value was valid. The bug is in the framing step, not in the JSON.
Where the two characters come from
Both characters are Unicode line-boundary characters, but they are not record delimiters in JSON Lines.
As an Amazon Associate I earn from qualifying purchases.
- U+2028 LINE SEPARATOR is described by the Unicode Standard (version 18.0.0, 2025) as an unconditional line separator. Any Unicode-aware splitter may therefore treat it as a line break.
- U+0085 NEXT LINE is a C1 control character. It is one of the default boundary characters in Unicode text segmentation (Unicode Standard Annex #29). Like U+2028, it can be recognized as a break by general-purpose text handling.
Neither the JSON Lines format nor the Unicode rules make these characters record terminators. Whether a given file is affected depends on whether something in your pipeline splits on them.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Three rules that are easy to blur
Most confusion comes from mixing three separate questions: how records are framed, whether a string is valid JSON, and what a particular library does. Keep them apart.
#1 Best Overall
Framing: what JSON Lines requires
The JSON Lines format specifies UTF-8 text with one valid JSON value per line, terminated by LF. CRLF is also supported, because JSON parsing ignores whitespace around a value, so a trailing carriage return is harmless. A final terminator is recommended but not required.
String validity: what RFC 8259 allows
RFC 8259 (IETF, 2017) requires that quotation marks, backslashes and the control characters U+0000 through U+001F be escaped inside a JSON string. U+2028 and U+0085 fall outside that range, so a JSON string may contain them unescaped and remain valid. RFC 8259 also notes that legal JSON text is not always valid JavaScript source; older JavaScript engines, before ECMAScript 2019, treated a raw U+2028 inside a string literal as a line terminator. That is a reason to parse JSON with a JSON parser, not by evaluating it as script.
Splitting: what your tools do
This is the layer that varies. Each library or command decides what it treats as a line. The JSON specifications do not dictate that behavior, and this article does not benchmark particular libraries.
Free tools Windows power users keep installed
One-click scans. No signup required.
How a valid record gets cut
Take a record whose string value contains a raw U+2028 between two words. The JSON is valid; the failure happens when the text is split before parsing.
line = '{"note": "alpha
beta"}n'
line.splitlines()
# ['{"note": "alpha', 'beta"}']
Python’s str.splitlines() documents a longer list of boundaries than LF: it also splits on carriage return, vertical tab, form feed, U+001C through U+001E, U+0085, U+2028 and U+2029. Parsing each fragment then fails:
json.loads('{"note": "alpha') # JSONDecodeError: unterminated string
json.loads('beta"}') # JSONDecodeError: expecting value
By contrast, iterating over a file opened in Python’s text mode splits only on n, r and rn (universal newlines). It does not see U+2028 or U+0085 as breaks. So the same file can parse cleanly in one reader and fail in another. Other general-purpose text APIs and editors may behave differently again.
Check whether your file is affected
-
Find out whether the characters are present. In a bash shell, run the byte-level search below. U+2028 is the UTF-8 byte sequence E2 80 A8, and U+0085 is C2 85.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsSpecial offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.LC_ALL=C grep -c $'xe2x80xa8' records.jsonl LC_ALL=C grep -c $'xc2x85' records.jsonlA count of zero means neither character appears. A non-zero count shows only that the characters exist; it does not prove that anything was split.
-
Validate the file against the LF framing rule, which is the reference. Save this script as
check_jsonl.pyand runpython3 check_jsonl.py records.jsonl.import json, sys with open(sys.argv[1], encoding="utf-8") as f: for n, line in enumerate(f, 1): try: json.loads(line) except json.JSONDecodeError as e: print(f"line {n}: {e}")Expected result: no output means every LF-delimited line is a valid JSON value. Any output names the line that failed.
-
Compare with your pipeline. If step 2 is clean but your tool reports parse errors or extra records, the file is fine and the splitter is the problem. Look for a call to
splitlines(), a generic line iterator, or a text-processing step that splits on Unicode boundaries.Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
What the standards do not settle
- The JSON Lines, RFC 8259 and Unicode documents define framing, string syntax and character properties. They do not define how any specific library, editor or command splits lines.
- No published figure establishes how often this happens in real data. The problem is a mismatch between a framing rule and a splitter, so it appears only where both coexist.
- The behavior described here applies to the examples shown. Verify your own parser and splitter before relying on a particular outcome.
Fix the producer or the consumer
Producer: escape the two characters
A JSON serializer may write U+2028 and U+0085 either raw or escaped. The escaped form (
, u0085) is valid JSON and decodes to the same string value, so escaping keeps the record conformant while removing the characters that boundary-aware tools would split on.
import json
json.dumps({"note": "alpha
beta"})
# '{"note": "alpha\u2028beta"}'
json.dumps({"note": "alpha
beta"}, ensure_ascii=False)
# '{"note": "alpha
beta"}' with the raw character
Python’s default ensure_ascii=True escapes these characters. Passing ensure_ascii=False writes them raw, which is the form that causes trouble for splitters. Other serializers have their own options; check their documentation.
The trade-off is for raw-text readers. A consumer that compares bytes or greps the file will see the escape sequences rather than the original characters.
Consumer: split on LF only
Read the file as text, split on LF, drop one trailing carriage return if present, and parse each piece. Ignore the empty string that follows a final terminator. Avoid splitlines() and generic line iterators on this path.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutetext = open("records.jsonl", encoding="utf-8").read()
for line in text.split("n"):
if line.endswith("r"):
line = line[:-1]
if not line:
continue
record = json.loads(line)
Comparing the approaches
| Approach | Conformance to JSON Lines | Compatibility with generic text tools | Risk to valid records |
|---|---|---|---|
| Split on LF (optionally strip CR), then parse each line | Follows the framing rule | Works with line tools that use LF; depends on the tool | Low, provided the file is LF or CRLF framed |
Split on Unicode boundaries (for example splitlines()) |
Does not follow the framing rule | Broad, but splits on characters inside strings | High: valid strings are cut into invalid fragments |
| Escape U+2028 and U+0085 at the producer | Remains conformant; escaped form is valid JSON | Safer for boundary-aware tools; raw-byte readers see escapes | Low for parsed consumers; changes the raw bytes |
| Escape at the producer and split on LF at the consumer | Follows the framing rule | Compatible with both boundary-aware and LF-only tools | Lowest of the options shown |
RFC 7464 is a different format
RFC 7464 (IETF, 2015) defines JSON text sequences, which are a separate format from JSON Lines. Each record begins with the ASCII Record Separator, U+001E, followed by a UTF-8 JSON text and a terminating LF. The explicit prefix gives readers a marker to resynchronize on. A JSON Lines file does not use U+001E, and a JSON Text Sequence is not a JSON Lines file. Be aware that splitlines() also splits on U+001E, so the same splitting problem applies to these streams.
The Bottom Line
Treat LF as the only record boundary. U+2028 and U+0085 are legal raw characters inside JSON strings, so a record that contains them is valid. If a splitter cuts on them, change the splitter to LF, or escape the characters when you write the file.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
