I’m trying to understand what is happening when I type certain foreign-language characters in Cygwin’s mintty or xterm terminal, or when I copy and paste characters into a Cygwin terminal from a Windows application, or even from within the Cygwin window itself.
I have written a utility xtou that takes as its input the hexadecimal value of a UTF-8 character and outputs the corresponding UTF-8 character, and another utility utox that takes as its input a UTF-8 character and outputs the corresponding hexadecimal value.
I have tested a number of UTF-8 characters on the Cygwin terminal, and all of them are output correctly. I think that this shows that the Cygwin terminal does use UTF-8 encoding. However, if I change the Windows keyboard input language and type the same foreign-language characters in the Cygwin terminal, many of the characters are not displayed correctly. What is puzzling is that my utility still correctly identifies the character, but it is not displayed correctly when typed using the Windows system keyboard input. The same thing happens if I copy and paste characters from another Windows application into the Cygwin terminal, or even if I copy and paste characters within the Cygwin terminal.
Here are some examples:
$ xtou C3A9 // U+00E9
é
$ utox é // copied from line above, displays correctly
0XC3A9 // correct hex code is output
$ utox é // typed using Windows keyboard input, displays correctly
0XC3A9 // correct hex code is output
$ xtou d0b6 // U+0436
ж
$ utox ж // copied from line above, displays correctly
0XD0B6 // correct hex code is output
$ utox ж // typed using Windows keyboard input, displays correctly
0XD0B6 // correct hex code is output
$ xtou d096 // U+0416, upper-case equivalent of previous character
Ж
$ utox ▒^▒ // copied from line above, displays incorrectly
0XD096 // correct hex code is output
$ utox ▒^▒ // typed using Windows keyboard input, displays incorrectly
0XD096 // correct hex code is output
$ xtou e990b5 // U+9435
鐵
$ utox ▒^е // copied from line above, displays incorrectly
0XE990B5 // correct hex code is output
The version of Cygwin I am running is up to date. In case it matters, the Windows computer on which I am running Cygwin is running Windows 7. Of course W7 is no longer supported, but I believe that it had received all updates. In Cygwin, I have tried using both the xterm and mintty settings (right click on title bar -> Options -> Terminal -> Type -> xterm or mintty). I don’t think that the shell matters, but in case it does I am using the Korn shell.
I suspect that this might have something to do with the Windows clipboard and/or with the input method, but I do not understand what is happening. Can anyone explain what is going on here?
