Bad caret position on a line containing Japanese symbols

[ProgerXP]

Open Notepad 2e with default settings (/f0), set the following font settings:

• Segoe UI
• Regular
• 16 (size)

Now paste this line (switch the buffer to UTF-8, Shift+F8):

日本zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzP TOPTOPTOPTOPTOPTOPZZZZZ IIIIzzzzzI!

Enable word wrapping and adjust window size so that this line has 1 subline (see the screenshot). Put the caret at the end of the line - you will notice that it's not positioned exactly after last character. Type z - you will notice that the caret appears right in the middle of the symbol, not after it.

This only happens if the line has Japanese symbols (alike to #247).

It's probably a Scintilla bug. If so, report it to them, please.

[cshnik]

Created SciDemo repo for Scintilla testing/debugging purposes.

[cshnik]

Please consider the following:

1. It looks like there is a problem with GDI (SC_TECHNOLOGY_DEFAULT mode is used) when using Segoe UI font: wrapped line (second substring) is rendered differently unlike its representation when drawing text as single line . This make char positions cache incorrect and cause incorrect caret position when navigating on wrapped substring.
   
2. Additional links related to the issue:
   https://parnassus.co/drawing-fully-justified-text-to-a-canvas/
   Allow optional use of Scintilla's DirectWrite technology. notepad-plus-plus/notepad-plus-plus#8326
   using SC_TECHNOLOGY_DIRECTWRITE rather than SC_TECHNOLOGY_DEFAULT XhmikosR/notepad2-mod#122
3. UseDirectWrite INI option added (Default is 1). It enables SC_TECHNOLOGY_DIRECTWRITE mode to make Scintilla use DirectWrite API.
4. Scintilla bug created.

[ProgerXP]

As expected, their response says it's by design in Scintilla so I don't think it will be fixed:

GDI sometimes changes character spacing based on which language the text is in. This is shown in SegoeUI_wordwrap.png where the characters don't line up. If a Japanese character is added to the second display line then the caret appears at a more correct location.

Scintilla measures the whole text and uses that to find character positions but breaks the text up into lines for drawing separately.

However, UseDirectWrite fixed this issue entirely.