Windows: encode and decode text as UTF-8 instead of trusting the locale

setlocale(LC_ALL, "") maps to the ANSI code page on Windows -- 936 (GBK)
on a Chinese system -- so wcrtomb() produces GBK where the protocol wants
UTF-8, and mbrtowc() reads UTF-8 as GBK. Typed Chinese therefore reached
the model as mojibake, and once the buffer held correct UTF-8 the input
line rendered as mojibake instead. Windows' C runtime has no ".UTF-8"
locale to switch to, so the conversion has to be explicit.

os_compat.[ch]
  * add os_utf8_encode() and os_utf8_decode(), locale-independent, with
    range and overlong-form checks
src/tui.c
  * input_put_wc(), the command-palette filter and the ask_user buffer use
    os_utf8_encode() instead of wcrtomb()
src/textsel.c
  * cst_utf8_decode() uses os_utf8_decode() instead of mbrtowc()
This commit is contained in:
XYZ
2026-10-06 17:56:55 +08:00
parent 3db33eddac
commit 929f7125b8
4 changed files with 121 additions and 30 deletions
+12 -6
View File
@@ -3,23 +3,29 @@
#endif
#include "cst_textsel.h"
#include "os_compat.h"
#include <stdlib.h>
#include <string.h>
#include <wctype.h>
int cst_utf8_decode(const char *s, size_t avail, wchar_t *wc, size_t *eaten) {
unsigned long cp = 0;
int n;
if (avail == 0)
return -1;
mbstate_t st;
memset(&st, 0, sizeof(st));
size_t n = mbrtowc(wc, s, avail, &st);
if (n == (size_t)-1 || n == (size_t)-2 || n == 0) {
*wc = (unsigned char)s[0];
/* os_utf8_decode(), not mbrtowc(): the latter follows setlocale(), which on
* Windows means the ANSI code page, so UTF-8 text would be decoded as GBK
* and rendered as mojibake. */
n = os_utf8_decode(s, avail, &cp);
if (n <= 0) {
/* Not a valid sequence: fall back to one raw byte, as before. */
*wc = (wchar_t)(unsigned char)s[0];
*eaten = 1;
return 0;
}
*eaten = n;
*wc = (wchar_t)cp;
*eaten = (size_t)n;
return 0;
}