Misplaced Pages

#259740

33-474: The Universal Coded Character Set ( UCS , Unicode ) is a standard set of characters defined by the international standard ISO / IEC  10646, Information technology — Universal Coded Character Set (UCS) (plus amendments to that standard), which is the basis of many character encodings , improving as characters from previously unrepresented typing systems are added. The UCS has over 1.1 million possible code points available for use/allocation, but only

66-540: A natural language . Examples of characters include letters , numerical digits , common punctuation marks (such as "." or "-"), and whitespace . The concept also includes control characters , which do not correspond to visible symbols but rather to instructions to format or process the text. Examples of control characters include carriage return and tab as well as other instructions to printers or other devices that display or otherwise process text. Characters are typically combined into strings . Historically,

99-466: A network . Two examples of usual encodings are ASCII and the UTF-8 encoding for Unicode . While most character encodings map characters to numbers and/or bit sequences, Morse code instead represents characters using a series of electrical impulses of varying length. Historically, the term character has been widely used by industry professionals to refer to an encoded character , often as defined by

132-426: A single directionality. It can handle some combining marks by simple overstriking methods, but cannot display Hebrew (bidirectional), Devanagari (one character to many glyphs) or Arabic (both features). Most GUI applications use standard OS text drawing routines which handle such scripts, although the applications themselves still do not always handle them correctly. ISO/IEC 10646 , a general, informal citation for

165-475: A single part, which has since had a number of amendments adding characters to the standard in approximate synchrony with the Unicode standard. Related standards: Character (computing) In computing and telecommunications , a character is a unit of information that roughly corresponds to a grapheme , grapheme-like unit, or symbol , such as in an alphabet or syllabary in the written form of

198-463: A universal character set existed: Unicode , with 16 bits for every character (65,536 possible characters), and ISO/IEC 10646. The software companies refused to accept the complexity and size requirement of the ISO standard and were able to convince a number of ISO National Bodies to vote against it. ISO officials realised they could not continue to support the standard in its current state and negotiated

231-417: A varying number of 8-bit code units to define a " code point " and Unicode uses varying number of those to define a "character". Computers and communication equipment represent characters using a character encoding that assigns each character to something – an integer quantity represented by a sequence of digits , typically – that can be stored or transmitted through

264-415: Is also addressed by Unicode. For instance, Unicode allocates a code point to each of This makes it possible to code the middle character of the word 'naïve' either as a single character 'ï' or as a combination of the character 'i ' with the combining diaeresis: (U+0069 LATIN SMALL LETTER I + U+0308 COMBINING DIAERESIS); this is also rendered as 'ï ' . These are considered canonically equivalent by

297-415: Is meant by the word "character". The fact that a character was historically stored in a single byte led to the two terms ("char" and "character") being used interchangeably in most documentation. This often makes the documentation confusing or misleading when multibyte encodings such as UTF-8 are used, and has led to inefficient and incorrect implementations of string manipulation functions (such as computing

330-456: Is not enough to support ISO/IEC 10646; Unicode must be implemented. To support these rules and algorithms, Unicode adds many properties to each character in the set such as properties determining a character's default bidirectional class and properties to determine how the character combines with other characters. If the character represents a numeric value such as the European number '8', or

363-440: Is required to hold UTF-8 code units which requires a minimum size of 8 bits. A Unicode code point may require as many as 21 bits. This will not fit in a char on most systems, so more than one is used for some of them, as in the variable-length encoding UTF-8 where each code point takes 1 to 4 bytes. Furthermore, a "character" may require more than one code point (for instance with combining characters ), depending on what

SECTION 10

#1732772579260

396-636: The Blit graphical terminal for Unix, the Inferno operating system, and the Limbo programming language . Pike also developed lesser systems such as the Newsqueak concurrent programming language and the vismon program for displaying faces of email authors. Over the years, Pike has written many text editors; sam and acme are the most well known. Pike started working at Google in 2002. While there, he

429-677: The Chinese logogram for water ("水") may have a slightly different appearance in Japanese texts than it does in Chinese texts, and local typefaces may reflect this. But nonetheless in Unicode they are considered the same character, and share the same code point. The Unicode standard also differentiates between these abstract characters and coded characters or encoded characters that have been paired with numeric codes that facilitate their representation in computers. The combining character

462-600: The ISO / IEC have developed The Unicode Standard ("Unicode") and ISO/IEC 10646 in tandem. The repertoire, character names, and code points of Unicode Version 2.0 exactly match those of ISO/IEC 10646-1:1993 with its first seven published amendments. After Unicode 3.0 was published in February 2000, corresponding new and updated characters entered the UCS via ISO/IEC 10646-1:2000. In 2003, parts 1 and 2 of ISO/IEC 10646 were combined into

495-558: The "length" of a string as a count of code units rather than bytes). Modern POSIX documentation attempts to fix this, defining "character" as a sequence of one or more bytes representing a single graphic symbol or control code, and attempts to use "byte" when referring to char data. However it still contains errors such as defining an array of char as a character array (rather than a byte array ). Unicode can also be stored in strings made up of code units that are larger than char . These are called " wide characters ". The original C type

528-514: The BMP. It does this to allow for future expansion or to minimise conflicts with other encoding forms. The original edition of the UCS defined UTF-16 , an extension of UCS-2, to represent code points outside the BMP. A range of code points in the S (Special) Zone of the BMP remains unassigned to characters. UCS-2 disallows use of code values for these code points, but UTF-16 allows their use in pairs. Unicode also adopted UTF-16, but in Unicode terminology,

561-524: The ISO/IEC 10646 family of standards, is acceptable in most prose. And even though it is a separate standard, the term Unicode is used just as often, informally, when discussing the UCS. However, any normative references to the UCS as a publication should cite the year of the edition in the form ISO/IEC 10646:{year} , for example: ISO/IEC 10646:2014 . Since 1991, the Unicode Consortium and

594-458: The Unicode standard. A char in the C programming language is a data type with the size of exactly one byte , which in turn is defined to be large enough to contain any member of the "basic execution character set". The exact number of bits can be checked via CHAR_BIT macro. By far the most common size is 8 bits, and the POSIX standard requires it to be 8 bits. In newer C standards char

627-481: The advent and widespread acceptance of Unicode and bit-agnostic coded character sets , a character is increasingly being seen as a unit of information , independent of any particular visual manifestation. The ISO/IEC 10646 (Unicode) International Standard defines character , or abstract character as "a member of a set of elements used for the organization, control, or representation of data". Unicode's definition supplements this with explanatory notes that encourage

660-452: The first window system for Unix in 1981. He is the sole inventor named in the US patent for overlapping windows on a computer display. With Brian Kernighan , he is the co-author of The Practice of Programming and The Unix Programming Environment . With Ken Thompson , he is the co-creator of UTF-8 character encoding. While at Bell Labs, Pike was also involved in the creation of

693-609: The first 65,536, which is the Basic Multilingual Plane (BMP), had entered into common use before 2000. This situation began changing when the People's Republic of China (PRC) ruled in 2006 that all software sold in its jurisdiction would have to support GB 18030 . This required software intended for sale in the PRC to move beyond the BMP. The system deliberately leaves many code points not assigned to characters, even in

SECTION 20

#1732772579260

726-498: The high-half zone elements become "high surrogates" and the low-half zone elements become "low surrogates". Another encoding, UTF-32 (previously named UCS-4), uses four bytes (total 32 bits) to encode a single character of the codespace. UTF-32 thereby permits a binary representation of every code point (as of year 2024) in the APIs, and software applications. The International Organization for Standardization (ISO) set out to compose

759-600: The limitation to the UTF-16 range and under the name UTF-32 , although it has almost no use outside programs' internal data. Rob Pike and Ken Thompson , the designers of the Plan 9 operating system, devised a new, fast and well-designed mixed-width encoding that was also backward-compatible with 7-bit ASCII , which came to be called UTF-8 , and is currently the most popular UCS encoding. ISO/IEC 10646 and Unicode have an identical repertoire and numbers—the same characters with

792-400: The programming language or API . Likewise, character set has been widely used to refer to a specific repertoire of characters that have been mapped to specific bit sequences or numerical codes. The term glyph is used to describe a particular visual appearance of a character. Many computer fonts consist of glyphs that are indexed by the numerical code of the corresponding character. With

825-604: The reader to differentiate between characters, graphemes, and glyphs, among other things. Such differentiation is an instance of the wider theme of the separation of presentation and content . For example, the Hebrew letter aleph ("א") is often used by mathematicians to denote certain kinds of infinity (ℵ), but it is also used in ordinary Hebrew text. In Unicode, these two uses are considered different characters, and have two different Unicode numerical identifiers (" code points "), though they may be rendered identically. Conversely,

858-539: The same numbers exist on both standards, although Unicode releases new versions and adds new characters more often. Unicode has rules and specifications outside the scope of ISO/IEC 10646. ISO/IEC 10646 is a simple character map, an extension of previous standards like ISO/IEC 8859 . In contrast, Unicode adds rules for collation , normalisation of forms , and the bidirectional algorithm for right-to-left scripts such as Arabic and Hebrew. For interoperability between platforms, especially if bidirectional scripts are used, it

891-486: The standard could code only 679,477,248 characters, as the policy forbade byte values of C0 and C1 control codes (0x00 to 0x1F and 0x80 to 0x9F, in hexadecimal notation) in any one of the four bytes specifying a group, plane, row and cell. The Latin capital letter A, for example, had a location in group 0x20, plane 0x20, row 0x20, cell 0x41. One could code the characters of this primordial ISO/IEC 10646 standard in one of three ways: In 1990, therefore, two initiatives for

924-472: The standard from version 2.0 and onwards supports encoding of 1,112,064 code points from 17 planes by means of the UTF-16 surrogate mechanism. For that reason, ISO/IEC 10646 was limited to contain as many characters as could be encoded by UTF-16 and no more, that is, a little over a million characters instead of over 679 million. The UCS-4 encoding of ISO/IEC 10646 was incorporated into the Unicode standard with

957-465: The term character was used to denote a specific number of contiguous bits . While a character is most commonly assumed to refer to 8 bits (one byte ) today, other options like the 6-bit character code were once popular, and the 5-bit Baudot code has been used in the past as well. The term has even been applied to 4 bits with only 16 possible values. All modern systems use a varying-size sequence of these fixed-sized pieces, for instance UTF-8 uses

990-511: The unification of their standard with Unicode. Two changes took place: the lifting of the limitation upon characters (prohibition of control code values), thus opening code points for allocation; and the synchronisation of the repertoire of the Basic Multilingual Plane with that of Unicode. Meanwhile, in the passage of time, the situation changed in the Unicode standard itself: 65,536 characters came to appear insufficient, and

1023-494: The universal character set in 1989, and published the draft of ISO 10646 in 1990. Hugh McGregor Ross was one of its principal architects. This work happened independently of the development of the Unicode standard, which had been in development since 1987 by Xerox and Apple . The original ISO 10646 draft differed markedly from the current standard. It defined: for an apparent total of 2,147,483,648 characters, but actually

Universal Coded Character Set - Misplaced Pages Continue

1056-412: The vulgar fraction '¼', that numeric value is also added as a property of the character. Unicode intends these properties to support interoperable text handling with a mixture of languages. Some applications support ISO/IEC 10646 characters but do not fully support Unicode. One such application, Xterm , can properly display all ISO/IEC 10646 characters that have a one-to-one character-to-glyph mapping and

1089-508: Was called wchar_t . Due to some platforms defining wchar_t as 16 bits and others defining it as 32 bits, recent versions have added char16_t , char32_t . Even then the objects being stored might not be characters, for instance the variable-length UTF-16 is often stored in arrays of char16_t . Other languages also have a char type. Some such as C++ use at least 8 bits like C. Others such as Java use 16 bits for char in order to represent UTF-16 values. Rob Pike Pike wrote

#259740