[RFC] win-iconv

[Yaakov (Cygwin/X)]

Fedora's MinGW toolchain has used win-iconv[1] instead of GNU libiconv
since F16. win-iconv uses the Win32 APIs to do the heavy lifting,
making it almost 50 times smaller than GNU libiconv. It handles most of
the same codesets as libiconv and even some the latter does not
(including the glibc aliases UTF8/UTF16 for UTF-8/UTF-16 etc.). I have
attached a normalized diff between `iconv -l` from libiconv and current
win-iconv. OTOH, it doesn't provide all of the API extensions of GNU
libiconv, but neither does glibc.

We could even go one step further and incorporate win-iconv (which is
Public Domain) directly into Cygwin itself, which would provide greater
compatibility with glibc and save patches to several packages which
don't use portable ways (e.g. AM_ICONV) for searching for iconv(3), or
use $(LIBICONV) when they should use $(LTLIBICONV), etc. This would add
about 18KB to cygwin1.dll but save almost 1MB of memory and ImageBase
space for hundreds of packages.

Perhaps the tradeoffs aren't worth it in the end, but I thought it at
least warranted a discussion. (And if it's not, at least I can stop
pondering this.)


Yaakov

[1] http://code.google.com/p/win-iconv/

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)
Fedora's MinGW toolchain has used win-iconv[1] instead of GNU libiconv
since F16. win-iconv uses the Win32 APIs to do the heavy lifting,
making it almost 50 times smaller than GNU libiconv. It handles most of
the same codesets as libiconv and even some the latter does not
(including the glibc aliases UTF8/UTF16 for UTF-8/UTF-16 etc.). I have
attached a normalized diff between `iconv -l` from libiconv and current
win-iconv. OTOH, it doesn't provide all of the API extensions of GNU
libiconv, but neither does glibc.

The downside is that it doesn't support some of the codesets supported
by Cygwin, for instance big5, euccn, eucjp, euckr, georgian-ps, tis-620.

Post by Yaakov (Cygwin/X)
We could even go one step further and incorporate win-iconv (which is
Public Domain) directly into Cygwin itself, which would provide greater
compatibility with glibc and save patches to several packages which
don't use portable ways (e.g. AM_ICONV) for searching for iconv(3), or
use $(LIBICONV) when they should use $(LTLIBICONV), etc. This would add
about 18KB to cygwin1.dll but save almost 1MB of memory and ImageBase
space for hundreds of packages.

Sounds like a worthy goal to me. But it really should at least support
the codesets also supported by Cygwin.


Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Fedora's MinGW toolchain has used win-iconv[1] instead of GNU libiconv
since F16. win-iconv uses the Win32 APIs to do the heavy lifting,
making it almost 50 times smaller than GNU libiconv. It handles most of
the same codesets as libiconv and even some the latter does not
(including the glibc aliases UTF8/UTF16 for UTF-8/UTF-16 etc.). I have
attached a normalized diff between `iconv -l` from libiconv and current
win-iconv. OTOH, it doesn't provide all of the API extensions of GNU
libiconv, but neither does glibc.

The downside is that it doesn't support some of the codesets supported
by Cygwin, for instance big5, euccn, eucjp, euckr, georgian-ps, tis-620.

Big-5 and EUC-* are supported, those first four are just missing
aliases. The last two are indeed missing.

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
We could even go one step further and incorporate win-iconv (which is
Public Domain) directly into Cygwin itself, which would provide greater
compatibility with glibc and save patches to several packages which
don't use portable ways (e.g. AM_ICONV) for searching for iconv(3), or
use $(LIBICONV) when they should use $(LTLIBICONV), etc. This would add
about 18KB to cygwin1.dll but save almost 1MB of memory and ImageBase
space for hundreds of packages.

Sounds like a worthy goal to me. But it really should at least support
the codesets also supported by Cygwin.

OK, I'll have to look into this further.


Yaakov