REPLY: 1:154/10 65e39208
MSGID: 1:104/117 65e40f96
CHRS: US-ASCII 2
TZUTC: -0700
TID: hpt/lnx 1.9 2022-07-03
Hello Nicholas.

02 Mar 24 14:44, you wrote to me:

 VA>> Most probably it was some combination which made it looking
 VA>> almost correct. I think that screen may be the reason you saw
 VA>> pseudo-graphics

 NB> I'm no longer using screen or tmux, and this version (along with all
 NB> previous versions since you made the ncurses change) work the same and
 NB> show UTF-8 properly, except for what you describe below:

 VA>> more or less correctly. Remember those line wraps? That happens
 VA>> because GoldEd converts those symbols to UTF-8 first. All
 VA>> pseudo-graphics symbols represented as 3 bytes. So that line
 VA>> become 3 times longer in bytes. Then GoldEd tries to split
 VA>> message to lines and it uses bytes! not symbols. That's why it
 VA>> splits the line in the middle of those pseudo-graphics. Even
 VA>> worse, it may tear apart one UTF-8 symbol to two lines and it
 VA>> will be displayed incorrectly.

 NB> Yeah, I've noticed most of this.. and thank you for your explanation.
 NB> At least now I know why it is happening.

 VA>> GoldEd cannot work correctly with multibyte sequences. And even
 VA>> if it looks "correct", it's just because most English letters has
 VA>> same codes in cp437 and UTF-8.

 NB> Maybe simple ones, like german umlauts and whatnot. But cp437 doesn't
 NB> have any Cyrillic, Greek, Japanese, Chinese, etc.

 VA>> If you want to keep using UTF-8, I may only suggest to find
 VA>> version, which "works" for you and stick to it.

 NB> I already have!

 VA>> Until full UTF-8 support implemented in GoldEd (if that ever
 VA>> happen), don't expect it to work correctly, sorry.

 NB> That's ok. I had it somewhat working for awhile, the latest reverts
 NB> have changed that. I don't have an issue going back to a "lucky"
 NB> version. ;)

 VA>> It's your choice. Just be aware, that if it works - it's just
 VA>> pure luck and don't expect it to last. Until we implement UTF-8
 VA>> support. It may take years. Or never happen. It's not so easy to
 VA>> do it with backward compatibility wih all older systems like DOS
 VA>> or OS/2.

 NB> I don't mind being lucky sometimes.

I played little bit more with the code and different settings and found what
was the difference between those two versions.

First - you use "wide" ncurses - ncursesw. I use ncurses.
Second. GoldEd incorrectly initializes ncurses in reverted version (like it
was before I started to make any changes in GoldEd code). ncurses
documentation explicitly says that before call initscr(), locale has to be set
with setlocale. Otherwise it will use incorrect settings. Most probably plain
C locale. I kinda get picture very similar to yours.

Did you have an issue with non-English chars when scrolling the message? What
I see in UFT-8 terminal now - most of unicode text displayed correctly and
pseudo-graphic too. Only pseudo-graphics lines broken similar to yours. And
when I scroll text down - it corrupts.

If you may try to add one line of code to latest master and try it - that
would be helpful.

goldlib/gcui/gkbdbase.cpp
line 149, right before initscr add this:

setlocale(LC_ALL, "");

If that gives you expected result - I'll push this change to master.

Hope you still want to dig this. :)

Vitaliy

--- GoldED+/LNX 1.1.5-b20240223
 * Origin: Aurora, Colorado (1:104/117)
SEEN-BY: 15/0 18/200 50/109 90/1 104/117 105/81 106/201 128/260 129/305
SEEN-BY: 135/225 153/7715 218/700 226/30 227/114 229/110 112 113 206
SEEN-BY: 229/307 317 400 426 428 470 664 700 266/512 280/464 5555
SEEN-BY: 282/1038 291/111 292/854 301/1 320/219 322/757 342/200 396/45
SEEN-BY: 460/16 58 256 1124 5858 463/68 467/888 633/280 712/848 3634/12
SEEN-BY: 5000/111 5001/100 5005/49 5015/46 5020/828 846 1042 4441
SEEN-BY: 5030/49 5054/8 30 5061/133 5075/128 5083/444 5090/958
PATH: 104/117 5020/1042 460/58 229/426