Add SGR-Pixels (mouse mode 1016)

[salt-die]

Description of the new feature

Now that sixel is implemented, SGR-Pixels mouse mode would be a nice feature to have. Also helps to implement features such as, e.g., smooth scrolling.

I've searched through issues and only found two references to this feature (below), so hopefully this isn't a duplicate:

#448 (comment) with response from @j4james :

I wasn't considering 1016, but I do have a POC of the DEC locator mode. There's no point in doing either of them until we have sixel, though, because the pixel coordinates will need to be tightly coupled to the sixel resolution.

And listed as not supported here #10531 (comment) from @DHowett

Would prefer 1016 over DEC Locator since it seems to be more widely used?
dankamongmen/notcurses#2326
wezterm/wezterm#1457
https://github.com/mahmoudimus/jexer/blob/16d27b6f4182e3c2cfecd7673934ebe09782861a/README.md?plain=1#L26C1-L26C70
Aetf/libtsm#29
xtermjs/xterm.js#4092

Proposed technical implementation details

No response

[j4james]

Also helps to implement features such as, e.g., smooth scrolling.

That's neat, but it's disappointing that they felt the need to resort to proprietary sequences. Here's a similar demo running in OpenConsole using DEC sequences from the 1980's. They could have achieved this 30+ years ago if terminal devs weren't constantly reinventing the wheel.

This is using my POC of DEC Locator mode btw. I never got around to turning it into a PR, because it required significant rewrites to the mouse handling, and I suspect there are things that got broken in the process. Implementing mode 1016 will almost certainly have the same problem.

But I'm not opposed to mode 1016. I just prefer the DEC Locator mode because it's more powerful, and has been around for 30 more years.

[schrmh]

@j4james Interesting, that's the first time I heard that it even is more powerful. I was told SGR is more useful (wezterm/wezterm#217 (comment)) but given your comment I would rather lean towards supporting the DEC way as well (and then just trying to do positive propaganda for implementing that or PRs. I think the SGR is not really used by many applications for mouse location tracking anyways since I did not stumple upon other than the mention stuff so far even when actively looking for it mid 2022).
Btw. which terminal emulator (that would need to run on GNU/Linux since I don't use Windows, sorry) would you recommend specifically for trying out stuff with DEC Locator mode? E.g. some are a bit glitchy or require a full-screened terminal. Ideally I would like to have a SIXEL-using application to respond to mouse clicks at certain locations (e.g. https://github.com/schrmh/pdfgrepSIXEL to do something with the clicked on slide)...

[j4james]

@schrmh It's been a few years since I last tested, but the only Linux terminals I'm aware of that have DEC locator support are mlterm and xterm, and it's possible you may have had to compile xterm with a specific build flag (--enable-dec-locator) to get that functionality.

Neither implementation was perfect (and I think the DEC documentation itself had errors), but if you just need basic click reporting it shouldn't be too complicated. However, I'd recommend testing with more than one terminal, just to make sure you aren't relying on behavior specific to one implementation.

[schrmh]

Yeah, sadly saitoha's drawing example did not work in mlterm at all when I tried it years ago. The full-screen thing I mentioned was with xterm regarding that example.

[j4james]

saitoha's drawing example did not work in mlterm at all when I tried it years ago.

Yeah, I've just looked at my notes on mlterm, and it seems that it doesn't (or at least didn't) handle DECRQLP requests when a mouse button is held down. And saitoha's drawing example is entirely dependent on DECRQLP polling, so I can see why that wouldn't work. But if you just need to detect click events, I think mlterm should be fine.