Browse Prior Art Database

UTF-8, a transformation format of Unicode and ISO 10646 (RFC2044)

IP.com Disclosure Number: IPCOM000002599D
Original Publication Date: 1996-Oct-01
Included in the Prior Art Database: 2000-Sep-13
Document File: 5 page(s) / 11K

Publishing Venue

Internet Society Requests For Comment (RFCs)

Related People

F. Yergeau: AUTHOR

Abstract

The Unicode Standard, version 1.1, and ISO/IEC 10646-1:1993 jointly define a 16 bit character set which encompasses most of the world's writing systems. 16-bit characters, however, are not compatible with many current applications and protocols, and this has led to the development of a few so-called UCS transformation formats (UTF), each with different characteristics. UTF-8, the object of this memo, has the characteristic of preserving the full US-ASCII range: US-ASCII characters are encoded in one octet having the usual US-ASCII value, and any octet with such a value can only be an US-ASCII character. This provides compatibility with file systems, parsers and other software that rely on US-ASCII values but are transparent to other values.

This text was extracted from a ASCII Text document.
This is the abbreviated version, containing approximately 27% of the total text.

Network Working Group F. Yergeau

Request for Comments: 2044 Alis Technologies

Category: Informational October 1996

UTF-8, a transformation format of Unicode and ISO 10646

Status of this Memo

This memo provides information for the Internet community. This memo

does not specify an Internet standard of any kind. Distribution of

this memo is unlimited.

Abstract

The Unicode Standard, version 1.1, and ISO/IEC 10646-1:1993 jointly

define a 16 bit character set which encompasses most of the world's

writing systems. 16-bit characters, however, are not compatible with

many current applications and protocols, and this has led to the

development of a few so-called UCS transformation formats (UTF), each

with different characteristics. UTF-8, the object of this memo, has

the characteristic of preserving the full US-ASCII range: US-ASCII

characters are encoded in one octet having the usual US-ASCII value,

and any octet with such a value can only be an US-ASCII character.

This provides compatibility with file systems, parsers and other

software that rely on US-ASCII values but are transparent to other

values.

1. Introduction

The Unicode Standard, version 1.1 [UNICODE], and ISO/IEC 10646-1:1993

[ISO-10646] jointly define a 16 bit character set, UCS-2, which

encompasses most of the world's writing systems. ISO 10646 further

defines a 31-bit character set, UCS-4, with currently no assignments

outside of the region corresponding to UCS-2 (the Basic Multilingual

Plane, BMP). The UCS-2 and UCS-4 encodings, however, are hard to use

in many current applications and protocols that assume 8 or even 7

bit characters. Even newer systems able to deal with 16 bit

characters cannot process UCS-4 data. This situation has led to the

development of so-called UCS transformation formats (UTF), each with

different characteristics.

UTF-1 has only historical interest, having been removed from ISO

10646. UTF-7 has the quality of encoding the full Unicode repertoire

using only octets with the high-order bit clear (7 bit US-ASCII

values, [US-ASCII]), and is thus deemed a mail-safe encoding

([RFC1642]). UTF-8, the object of this memo, uses all bits of an

octet, but has the quality of preserving the full US-ASCII range:

US-ASCII characters are encoded in one octet having the normal US-

ASCII value, and any octet with such a value can only stand for an

US-ASCII character, and nothing else.

UTF-16 is a scheme for transforming a subset of the UCS-4 repertoire

into a pair of UCS-2 values from a reserved range. UTF-16 impacts

UTF-8 in that UCS-2 values from the reserved range must be treated

specially in the UTF-8 transformation.

UTF-8 encodes UCS-2 or UCS-4 characters as a varying number of

octets, where the number of octets, and the value of each, depend on

the integer value assigned to the character i...