QTextCodec(3qt)QTextCodec(3qt)NAMEQTextCodec - Conversion between text encodings
SYNOPSIS
Almost all the functions in this class are reentrant when Qt is built
with thread support. The exceptions are ~QTextCodec(), setCodecForTr(),
setCodecForCStrings(), and QTextCodec(). </p>
#include <qtextcodec.h>
Inherited by QBig5Codec, QBig5hkscsCodec, QEucJpCodec, QEucKrCodec,
QGb18030Codec, QJisCodec, QHebrewCodec, QSjisCodec, and QTsciiCodec.
Public Members
virtual ~QTextCodec ()
virtual const char * name () const = 0
virtual const char * mimeName () const
virtual int mibEnum () const = 0
virtual QTextDecoder * makeDecoder () const
virtual QTextEncoder * makeEncoder () const
virtual QString toUnicode ( const char * chars, int len ) const
virtual QCString fromUnicode ( const QString & uc, int & lenInOut )
const
QCString fromUnicode ( const QString & uc ) const
QString toUnicode ( const QByteArray & a, int len ) const
QString toUnicode ( const QByteArray & a ) const
QString toUnicode ( const QCString & a, int len ) const
QString toUnicode ( const QCString & a ) const
QString toUnicode ( const char * chars ) const
virtual bool canEncode ( QChar ch ) const
virtual bool canEncode ( const QString & s ) const
virtual int heuristicContentMatch ( const char * chars, int len ) const
= 0
virtual int heuristicNameMatch ( const char * hint ) const
Static Public Members
QTextCodec * loadCharmap ( QIODevice * iod )
QTextCodec * loadCharmapFile ( QString filename )
QTextCodec * codecForMib ( int mib )
QTextCodec * codecForName ( const char * name, int accuracy = 0 )
QTextCodec * codecForContent ( const char * chars, int len )
QTextCodec * codecForIndex ( int i )
QTextCodec * codecForLocale ()
void setCodecForLocale ( QTextCodec * c )
QTextCodec * codecForTr ()
void setCodecForTr ( QTextCodec * c )
QTextCodec * codecForCStrings ()
void setCodecForCStrings ( QTextCodec * c )
void deleteAllCodecs ()
const char * locale ()
Protected Members
QTextCodec ()
Static Protected Members
int simpleHeuristicNameMatch ( const char * name, const char * hint )
DESCRIPTION
The QTextCodec class provides conversion between text encodings.
Qt uses Unicode to store, draw and manipulate strings. In many
situations you may wish to deal with data that uses a different
encoding. For example, most Japanese documents are still stored in
Shift-JIS or ISO2022, while Russian users often have their documents in
KOI8-R or CP1251.
Qt provides a set of QTextCodec classes to help with converting non-
Unicode formats to and from Unicode. You can also create your own codec
classes (see later).
The supported encodings are:
Latin1
Big5 -- Chinese
Big5-HKSCS -- Chinese
eucJP -- Japanese
eucKR -- Korean
GB2312 -- Chinese
GBK -- Chinese
GB18030 -- Chinese
JIS7 -- Japanese
Shift-JIS -- Japanese
TSCII -- Tamil
utf8 -- Unicode, 8-bit
utf16 -- Unicode
KOI8-R -- Russian
KOI8-U -- Ukrainian
ISO8859-1 -- Western
ISO8859-2 -- Central European
ISO8859-3 -- Central European
ISO8859-4 -- Baltic
ISO8859-5 -- Cyrillic
ISO8859-6 -- Arabic
ISO8859-7 -- Greek
ISO8859-8 -- Hebrew, visually ordered
ISO8859-8-i -- Hebrew, logically ordered
ISO8859-9 -- Turkish
ISO8859-10
ISO8859-13
ISO8859-14
ISO8859-15 -- Western
IBM 850
IBM 866
CP874
CP1250 -- Central European
CP1251 -- Cyrillic
CP1252 -- Western
CP1253 -- Greek
CP1254 -- Turkish
CP1255 -- Hebrew
CP1256 -- Arabic
CP1257 -- Baltic
CP1258
Apple Roman
TIS-620 -- Thai
QTextCodecs can be used as follows to convert some locally encoded
string to Unicode. Suppose you have some string encoded in Russian
KOI8-R encoding, and want to convert it to Unicode. The simple way to
do this is:
QCString locallyEncoded = "..."; // text to convert
QTextCodec *codec = QTextCodec::codecForName("KOI8-R"); // get the codec for KOI8-R
QString unicodeString = codec->toUnicode( locallyEncoded );
After this, unicodeString holds the text converted to Unicode.
Converting a string from Unicode to the local encoding is just as easy:
QString unicodeString = "..."; // any Unicode text
QTextCodec *codec = QTextCodec::codecForName("KOI8-R"); // get the codec for KOI8-R
QCString locallyEncoded = codec->fromUnicode( unicodeString );
Some care must be taken when trying to convert the data in chunks, for
example, when receiving it over a network. In such cases it is possible
that a multi-byte character will be split over two chunks. At best this
might result in the loss of a character and at worst cause the entire
conversion to fail.
The approach to use in these situations is to create a QTextDecoder
object for the codec and use this QTextDecoder for the whole decoding
process, as shown below:
QTextCodec *codec = QTextCodec::codecForName( "Shift-JIS" );
QTextDecoder *decoder = codec->makeDecoder();
QString unicodeString;
while( receiving_data ) {
QByteArray chunk = new_data;
unicodeString += decoder->toUnicode( chunk.data(), chunk.length() );
}
The QTextDecoder object maintains state between chunks and therefore
works correctly even if a multi-byte character is split between chunks.
Creating your own Codec class
Support for new text encodings can be added to Qt by creating
QTextCodec subclasses.
Built-in codecs can be overridden by custom codecs since more recently
created QTextCodec objects take precedence over earlier ones.
You may find it more convenient to make your codec class available as a
plugin; see the plugin documentation for more details.
The abstract virtual functions describe the encoder to the system and
the coder is used as required in the different text file formats
supported by QTextStream, and under X11, for the locale-specific
character input and output.
To add support for another 8-bit encoding to Qt, make a subclass of
QTextCodec and implement at least the following methods:
const char* name() const
Return the official name for the encoding.
int mibEnum() const
Return the MIB enum for the encoding if it is listed in the IANA
character-sets encoding file.
If the encoding is multi-byte then it will have "state"; that is, the
interpretation of some bytes will be dependent on some preceding bytes.
For such encodings, you must implement:
QTextDecoder* makeDecoder() const
Return a QTextDecoder that remembers incomplete multi-byte sequence
prefixes or other required state.
If the encoding does not require state, you should implement:
QString toUnicode(const char* chars, int len) const
Converts len characters from chars to Unicode.
The base QTextCodec class has default implementations of the above two
functions, but they are mutually recursive, so you must re-implement at
least one of them, or both for improved efficiency.
For conversion from Unicode to 8-bit encodings, it is rarely necessary
to maintain state. However, two functions similar to the two above are
used for encoding:
QTextEncoder* makeEncoder() const
Return a QTextEncoder.
QCString fromUnicode(const QString& uc, int& lenInOut ) const
Converts lenInOut characters (of type QChar) from the start of the
string uc, returning a QCString result, and also returning the length
of the result in lenInOut.
Again, these are mutually recursive so only one needs to be
implemented, or both if greater efficiency is possible.
Finally, you must implement:
int heuristicContentMatch(const char* chars, int len) const
Gives a value indicating how likely it is that len characters from
chars are in the encoding.
A good model for this function is the
QWindowsLocalCodec::heuristicContentMatch function found in the Qt
sources.
A QTextCodec subclass might have improved performance if you also re-
implement:
bool canEncode( QChar ) const
Test if a Unicode character can be encoded.
bool canEncode( const QString& ) const
Test if a string of Unicode characters can be encoded.
int heuristicNameMatch(const char* hint) const
Test if a possibly non-standard name is referring to the codec.
Codecs can also be created as plugins.
See also Internationalization with Qt.
MEMBER FUNCTION DOCUMENTATIONQTextCodec::QTextCodec () [protected]
Warning: This function is not reentrant.</p>
Constructs a QTextCodec, and gives it the highest precedence. The
QTextCodec should always be constructed on the heap (i.e. with new). Qt
takes ownership and will delete it when the application terminates.
QTextCodec::~QTextCodec () [virtual]
Warning: This function is not reentrant.</p>
Destroys the QTextCodec. Note that you should not delete codecs
yourself: once created they become Qt's responsibility.
bool QTextCodec::canEncode ( QChar ch ) const [virtual]
Returns TRUE if the Unicode character ch can be fully encoded with this
codec; otherwise returns FALSE. The default implementation tests if the
result of toUnicode(fromUnicode(ch)) is the original ch. Subclasses may
be able to improve the efficiency.
bool QTextCodec::canEncode ( const QString & s ) const [virtual]
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
s contains the string being tested for encode-ability.
QTextCodec * QTextCodec::codecForCStrings () [static]
Returns the codec used by QString to convert to and from const char*
and QCStrings. If this function returns 0 (the default), QString
assumes Latin-1.
See also setCodecForCStrings().
QTextCodec * QTextCodec::codecForContent ( const char * chars, int len )
[static]
Searches all installed QTextCodec objects, returning the one which most
recognizes the given content. May return 0.
Note that this is often a poor choice, since character encodings often
use most of the available character sequences, and so only by
linguistic analysis could a true match be made.
chars contains the string to check, and len contains the number of
characters in the string to use.
See also heuristicContentMatch().
Example: qwerty/qwerty.cpp.
QTextCodec * QTextCodec::codecForIndex ( int i ) [static]
Returns the QTextCodec i positions from the most recently inserted
codec, or 0 if there is no such QTextCodec. Thus, codecForIndex(0)
returns the most recently created QTextCodec.
Example: qwerty/qwerty.cpp.
QTextCodec * QTextCodec::codecForLocale () [static]
Returns a pointer to the codec most suitable for this locale.
Example: qwerty/qwerty.cpp.
QTextCodec * QTextCodec::codecForMib ( int mib ) [static]
Returns the QTextCodec which matches the MIBenum mib.
QTextCodec * QTextCodec::codecForName ( const char * name, int accuracy = 0 )
[static]
Searches all installed QTextCodec objects and returns the one which
best matches name; the match is case-insensitive. Returns 0 if no
codec's heuristicNameMatch() reports a match better than accuracy, or
if name is a null string.
See also heuristicNameMatch().
QTextCodec * QTextCodec::codecForTr () [static]
Returns the codec used by QObject::tr() on its argument. If this
function returns 0 (the default), tr() assumes Latin-1.
See also setCodecForTr().
void QTextCodec::deleteAllCodecs () [static]
Deletes all the created codecs.
Warning: Do not call this function.
QApplication calls this function just before exiting to delete any
QTextCodec objects that may be lying around. Since various other
classes hold pointers to QTextCodec objects, it is not safe to call
this function earlier.
If you are using the utility classes (like QString) but not using
QApplication, calling this function at the very end of your application
may be helpful for chasing down memory leaks by eliminating any
QTextCodec objects.
QCString QTextCodec::fromUnicode ( const QString & uc, int & lenInOut ) const
[virtual]
QTextCodec subclasses must reimplement either this function or
makeEncoder(). It converts the first lenInOut characters of uc from
Unicode to the encoding of the subclass. If lenInOut is negative or too
large, the length of uc is used instead.
Converts lenInOut characters (not bytes) from uc, producing a QCString.
lenInOut will be set to the length of the result (in bytes).
The default implementation makes an encoder with makeEncoder() and
converts the input with that. Note that the default makeEncoder()
implementation makes an encoder that simply calls this function, hence
subclasses must reimplement one function or the other to avoid infinite
recursion.
Reimplemented in QHebrewCodec.
QCString QTextCodec::fromUnicode ( const QString & uc ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
uc is the unicode source string.
int QTextCodec::heuristicContentMatch ( const char * chars, int len ) const
[pure virtual]
QTextCodec subclasses must reimplement this function. It examines the
first len bytes of chars and returns a value indicating how likely it
is that the string is a prefix of text encoded in the encoding of the
subclass. A negative return value indicates that the text is detectably
not in the encoding (e.g. it contains characters undefined in the
encoding). A return value of 0 indicates that the text should be
decoded with this codec rather than as ASCII, but there is no
particular evidence. The value should range up to len. Thus, most
decoders will return -1, 0, or -len.
The characters are not null terminated.
See also codecForContent().
int QTextCodec::heuristicNameMatch ( const char * hint ) const [virtual]
Returns a value indicating how likely it is that this decoder is
appropriate for decoding some format that has the given name. The name
is compared with the hint.
A good match returns a positive number around the length of the string.
A bad match is negative.
The default implementation calls simpleHeuristicNameMatch() with the
name of the codec.
QTextCodec * QTextCodec::loadCharmap ( QIODevice * iod ) [static]
Reads a POSIX2 charmap definition from iod. The parser recognizes the
following lines:
<font name="sans"> <code_set_name> name</br> <escape_char>
character</br> % alias alias</br> CHARMAP</br> <token> /xhexbyte
<Uunicode> ...</br> <token> /ddecbyte <Uunicode> ...</br> <token>
/octbyte <Uunicode> ...</br> <token> /any/any... <Uunicode> ...</br>
END CHARMAP</br> </font>
The resulting QTextCodec is returned (and also added to the global list
of codecs). The name() of the result is taken from the code_set_name.
Note that a codec constructed in this way uses much more memory and is
slower than a hand-written QTextCodec subclass, since tables in code
are kept in memory shared by all Qt applications.
See also loadCharmapFile().
Example: qwerty/qwerty.cpp.
QTextCodec * QTextCodec::loadCharmapFile ( QString filename ) [static]
A convenience function for loadCharmap() that loads the charmap
definition from the file filename.
const char * QTextCodec::locale () [static]
Returns a string representing the current language and sublanguage,
e.g. "pt" for Portuguese, or "pt_br" for Portuguese/Brazil.
Example: i18n/main.cpp.
QTextDecoder * QTextCodec::makeDecoder () const [virtual]
Creates a QTextDecoder which stores enough state to decode chunks of
char* data to create chunks of Unicode data. The default implementation
creates a stateless decoder, which is only sufficient for the simplest
encodings where each byte corresponds to exactly one Unicode character.
The caller is responsible for deleting the returned object.
QTextEncoder * QTextCodec::makeEncoder () const [virtual]
Creates a QTextEncoder which stores enough state to encode chunks of
Unicode data as char* data. The default implementation creates a
stateless encoder, which is only sufficient for the simplest encodings
where each Unicode character corresponds to exactly one character.
The caller is responsible for deleting the returned object.
int QTextCodec::mibEnum () const [pure virtual]
Subclasses of QTextCodec must reimplement this function. It returns the
MIBenum (see the IANA character-sets encoding file for more
information). It is important that each QTextCodec subclass returns the
correct unique value for this function.
Reimplemented in QEucJpCodec.
const char * QTextCodec::mimeName () const [virtual]
Returns the preferred mime name of the encoding as defined in the IANA
character-sets encoding file.
Reimplemented in QEucJpCodec, QEucKrCodec, QJisCodec, QHebrewCodec, and
QSjisCodec.
const char * QTextCodec::name () const [pure virtual]
QTextCodec subclasses must reimplement this function. It returns the
name of the encoding supported by the subclass. When choosing a name
for an encoding, consider these points:
On X11, heuristicNameMatch( const char * hint ) is used to test if a
the QTextCodec can convert between Unicode and the encoding of a font
with encoding hint, such as "iso8859-1" for Latin-1 fonts," koi8-r" for
Russian KOI8 fonts. The default algorithm of heuristicNameMatch() uses
name().
Some applications may use this function to present encodings to the end
user.
Example: qwerty/qwerty.cpp.
void QTextCodec::setCodecForCStrings ( QTextCodec * c ) [static]
Warning: This function is not reentrant.</p>
Sets the codec used by QString to convert to and from const char* and
QCStrings. If c is 0 (the default), QString assumes Latin-1.
Warning: Some codecs do not preserve the characters in the ascii range
(0x00 to 0x7f). For example, the Japanese Shift-JIS encoding maps the
backslash character (0x5a) to the Yen character. This leads to
unexpected results when using the backslash character to escape
characters in strings used in e.g. regular expressions. Use
QString::fromLatin1() to preserve characters in the ascii range when
needed.
See also codecForCStrings() and setCodecForTr().
void QTextCodec::setCodecForLocale ( QTextCodec * c ) [static]
Set the codec to c; this will be returned by codecForLocale(). This
might be needed for some applications that want to use their own
mechanism for setting the locale.
See also codecForLocale().
void QTextCodec::setCodecForTr ( QTextCodec * c ) [static]
Warning: This function is not reentrant.</p>
Sets the codec used by QObject::tr() on its argument to c. If c is 0
(the default), tr() assumes Latin-1.
If the literal quoted text in the program is not in the Latin-1
encoding, this function can be used to set the appropriate encoding.
For example, software developed by Korean programmers might use eucKR
for all the text in the program, in which case the main() function
might look like this:
int main(int argc, char** argv)
{
QApplication app(argc, argv);
... install any additional codecs ...
QTextCodec::setCodecForTr( QTextCodec::codecForName("eucKR") );
...
}
Note that this is not the way to select the encoding that the user has
chosen. For example, to convert an application containing literal
English strings to Korean, all that is needed is for the English
strings to be passed through tr() and for translation files to be
loaded. For details of internationalization, see the Qt
internationalization documentation.
See also codecForTr() and setCodecForCStrings().
int QTextCodec::simpleHeuristicNameMatch ( const char * name, const char *
hint ) [static protected]
A simple utility function for heuristicNameMatch(): it does some very
minor character-skipping so that almost-exact matches score high. name
is the text we're matching and hint is used for the comparison.
QString QTextCodec::toUnicode ( const char * chars, int len ) const [virtual]
QTextCodec subclasses must reimplement this function or makeDecoder().
It converts the first len characters of chars to Unicode.
The default implementation makes a decoder with makeDecoder() and
converts the input with that. Note that the default makeDecoder()
implementation makes a decoder that simply calls this function, hence
subclasses must reimplement one function or the other to avoid infinite
recursion.
QString QTextCodec::toUnicode ( const QByteArray & a, int len ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
a contains the source characters; len contains the number of characters
in a to use.
QString QTextCodec::toUnicode ( const QByteArray & a ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
a contains the source characters.
QString QTextCodec::toUnicode ( const QCString & a, int len ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
a contains the source characters; len contains the number of characters
in a to use.
QString QTextCodec::toUnicode ( const QCString & a ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
a contains the source characters.
QString QTextCodec::toUnicode ( const char * chars ) const
This is an overloaded member function, provided for convenience. It
behaves essentially like the above function.
chars contains the source characters.
SEE ALSO
http://doc.trolltech.com/qtextcodec.html
http://www.trolltech.com/faq/tech.html
COPYRIGHT
Copyright 1992-2007 Trolltech ASA, http://www.trolltech.com. See the
license file included in the distribution for a complete license
statement.
AUTHOR
Generated automatically from the source code.
BUGS
If you find a bug in Qt, please report it as described in
http://doc.trolltech.com/bughowto.html. Good bug reports help us to
help you. Thank you.
The definitive Qt documentation is provided in HTML format; it is
located at $QTDIR/doc/html and can be read using Qt Assistant or with a
web browser. This man page is provided as a convenience for those users
who prefer man pages, although this format is not officially supported
by Trolltech.
If you find errors in this manual page, please report them to qt-
bugs@trolltech.com. Please include the name of the manual page
(qtextcodec.3qt) and the Qt version (3.3.8).
Trolltech AS 2 February 2007 QTextCodec(3qt)