Free tools Windows power users keep installed
One-click scans. No signup required.
A character-code standard does two jobs. It decides which characters exist and gives each one a number, and it defines how that number is represented in bits. The Unicode Consortium’s technical introduction puts it this way: “Character encoding standards define not only the identity of each character and its numeric value, or code point, but also how this value is represented in bits.” Those two jobs are separate, and mixing them up is the source of most confusion about “Unicode,” “UTF-8,” and “bytes.”
What is a character code?
A character code is the number a standard assigns to a character so that software can store, compare, and exchange text. In Unicode, each encoded character gets a numeric code point and a name. The code point identifies the character; it is not, by itself, a sequence of bytes. Conventionally code points are written in hexadecimal with a “U+” prefix, such as U+0041 for the capital letter A.
As an Amazon Associate I earn from qualifying purchases.
How the standards model character encoding
Unicode’s character-encoding model (Unicode Technical Report on the character encoding model) splits the problem into four layers. Keeping them apart answers most “is it a code point or a byte?” questions.
| Layer | What it is | Example |
|---|---|---|
| Abstract character repertoire | The set of characters selected for encoding | Latin small letter e with acute |
| Coded character set | A mapping from those characters to nonnegative integers (code points) | U+00E9 |
| Character encoding form | A mapping from integers to sequences of code units | UTF-8: C3, A9 (two 8-bit units); UTF-16: 00E9 (one 16-bit unit) |
| Character encoding scheme | A reversible transformation of code-unit sequences into serialized bytes, including byte order where relevant | UTF-16BE: 00 E9; UTF-16LE: E9 00 |
A code point is a numeric value or position in a coded character set. A code unit is the minimum-width unit used for processing or interchange in an encoding form. Only at the last layer do you get bytes in a file or on a network.
#1 Best Overall
What is the relation between ISO/IEC 10646 and Unicode?
They are not rival repertoires. According to the Unicode Consortium FAQ, Unicode and the ISO working group responsible for ISO/IEC 10646 decided in 1991 to create one universal character standard, and they have worked since to keep their versions synchronized. Their character codes and encoding forms match.
The difference is scope. ISO/IEC 10646 is the character-code standard. The Unicode Standard adds implementation constraints and extensive material: character specifications, data files, algorithms, and background, intended to make text handling uniform across platforms and applications.
Code points, code units, and bytes: the difference
Take three characters and see how the same layers play out.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall| Character | Code point | UTF-8 code units (bytes) | UTF-16 code units | UTF-32 code unit |
|---|---|---|---|---|
| A | U+0041 | 41 | 0041 | 00000041 |
| € (euro sign) | U+20AC | E2 82 AC | 20AC | 000020AC |
| 😀 (grinning face) | U+1F600 | F0 9F 98 80 | D83D DE00 (a surrogate pair) | 0001F600 |
One code point can need one, two, three, or four bytes in UTF-8, and one or two 16-bit units in UTF-16. So “one character = one byte” is wrong for Unicode, and a code point is never itself a byte sequence. A code point is a number; an encoding form turns it into code units; a scheme turns code units into bytes.
What UTF-8, UTF-16, and UTF-32 mean
UTF stands for Unicode Transformation Format. The Unicode FAQ defines a UTF as “an algorithmic mapping from every Unicode code point (except surrogate code points) to a unique byte sequence,” and notes that the mappings are reversible, so text round-trips losslessly.
UTF-8
Uses 8-bit code units and is variable length. It is designed to be compatible with ASCII byte values, so plain ASCII text is already valid UTF-8, as the A row above shows.
UTF-16
Uses 16-bit code units. Code points in the Basic Multilingual Plane take one unit; others take a surrogate pair of two units. Because a unit is two bytes, serializing it requires choosing a byte order (UTF-16BE or UTF-16LE).
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
UTF-32
Uses 32-bit code units, so each code point is a single unit whose value equals the code point. It likewise needs a byte order when serialized.
Best Value
| Form | Code unit | Variable width | ASCII byte compatibility |
|---|---|---|---|
| UTF-8 | 8 bits | Yes | Yes |
| UTF-16 | 16 bits | Yes (one or two units) | No |
| UTF-32 | 32 bits | No | No |
How big is the Unicode code space?
The Unicode Standard (version 17.0 specification) describes a codespace of 1,114,112 code points, most of which are available for encoding characters. The first 65,536 form the Basic Multilingual Plane. “Available” does not mean “assigned”: not every code point has a character, and the count of assigned characters depends on the Unicode version you use. Surrogate code points are also excluded from UTF mappings, as the FAQ definition above states.
Quick Recap
Common misconceptions
- “Unicode is UTF-8.” Unicode defines the shared repertoire and code assignments and supports several encoding forms. UTF-8 is one of them.
- “A code point is a byte.” A code point is an integer in a coded character set; bytes appear only after an encoding form and scheme are applied.
- “Every code point is a character.” Most are available for encoding characters; that is not the same as all being assigned.
- “Unicode and ISO/IEC 10646 disagree.” Their code assignments and encoding forms are kept synchronized.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
