SIXEL: cursor positioning and shared palette in TEs

[jerch]

(Partially offtopic for VT340, where the palette is always a globally shared one...)

I wonder if we manage to get a somewhat consistent behavior for TEs implementing sixels for these aspects:

• cursor positioning
  Imho the final cursor positioning after sixels is a wild-grown nightmare among TEs currently, where most modes makes not much sense or can be achieved by additional cursor movements. Thus I'd like to discuss, if we can simplify that to 2 basic modes, that are hard to achieve otherwise:
  • text cursor at VT340 position: Make this the default mode. Its big advantage is the fact, that it can dump sixels up to the last line without doing another scroll. Furthermore it places the cursor at image offset, which has a big advantage for aligned text output.
  • cursor at last image cell: I see this as the most natural cursor behavior in terms of cursor advance, if one sees sixel input as a block of cells. This is similar to what xterm and mintty do, but they go one cell further and the nightmare starts - what, if the cursor ends at COLS+1? Should the cursor wrap? Based on which setting? DECAWM? I'd say no, imho no wrapping should happen at all, instead place the text cursor in the last image cell. From there app can move the cursor. This mode has the advantage to automatically give information about the printout size in both image dimensions. If we dont allow any automatic wrapping, the flaw of xterm's mode with the last line scrolling also vanishes.
• private vs. shared palette handling
  This one is not really VT340 related, as the palette is always shared. What I'd like to discuss here, is the question, how newer TEs could possibly handle private and shared in a uniform way.
  • private palette
    I'd like to propose, that this mode always bootstraps the initial palette from some predefined palette, an app user can expect to work with without any own color definition (we currently use VT340-color for that, I am also fine to use ANSI256 if we can agree on that).
  • shared palette
    This mode is really tricky for most newer TEs, as many underlying sixel libs dont expose indexed images, but realize colors immediately (I call this printer mode here). I still hesitate to implement a true terminal-wide shared palette handling, but would do that, if there is demand. In the meantime I'd like to propose a less disruptive shared mode instead, which still works in printer mode with immediate coloring, but carries the created palette forward to the next incoming sixel image. This would allow to lower data pressure by omitting palette creation in consecutive images using color definitions from an earlier image.

I am not really determined to the modes as proposed above, but I think we should find an agreement to these aspects, so app devs have a model expected to work in a certain way.

[j4james]

Cursor positioning: I personally don't see a need for anything other than the standard behavior, and I'd be very impressed if we could even get TEs to agree on that. I'd just be happy to get XTerm following the standard, and after that I don't care. With any luck other TEs will fall in line like they did with DECSDM.

Palette handling: For this I'd like something that by default is as close to the VT340 as possible while still being usable with modern apps, ideally with a strict compatibility mode as well. I'm still not exactly sure how I'm going to achieve that, but I can't imagine other TEs wanting to take the same approach.

That said, instead of trying to get TEs to agree on exactly matching behavior, we could possibly come up with a minimal set of guarantees that apps could rely on. For example:

1. If you set a palette value in one image, it will be carried forward to the next image, but only if you assign colors in the order #1, #2, #3 (i.e. following VT340 rules).
2. If you output an image with a new set of palette values, it won't affect previous images you've output by default (i.e. outside of any strict compatibility modes the TE might have).
3. If you change palette values half way through an image being output, it may or may not effect the pixels in that image that have already been displayed (in other words, don't to that).

That doesn't seem like a lot to ask of TEs, but still provides a reasonable base for apps to work with.

[dankamongmen]

2. If you output an image with a new set of palette values, it won't affect previous images you've output by default (i.e. outside of any strict compatibility modes the TE might have).

it would be good to lock down how the palette size relates to independent palettes. i.e. if it's 16, can i have 2 images with 16 distinct colors each? or is it simply two palette "namespaces" backed by a hard 16 colors?

[j4james]

it would be good to lock down how the palette size relates to independent palettes. i.e. if it's 16, can i have 2 images with 16 distinct colors each? or is it simply two palette "namespaces" backed by a hard 16 colors?

On modern terminals you'll have access to N distinct colors in each image, where N is typically 256 or more. On DEC terminals, and strict emulators, you'll have a hard limit of 16 colors or less shared across all images.

I'm not sure what you mean by namespaces, but modern terminals don't support color mapping in the sense that DEC terminals do. On modern terminals, sixel color #0 will usually always map to palette entry 0. On DEC terminals (and strict emulators), the first color defined will update palette entry 1 (not 0), regardless of the color number, so different images may have different color number mappings. But this is only really of relevance to palette inheritance, and as long as you assign colors in the order I recommended, it shouldn't matter.

[dankamongmen]

oh of course; i was just emphasizing that this be addressed in any protostandard y'all're developing

[jerch]

Cursor positioning: I personally don't see a need for anything other than the standard behavior, and I'd be very impressed if we could even get TEs to agree on that. I'd just be happy to get XTerm following the standard, and after that I don't care. With any luck other TEs will fall in line like they did with DECSDM.

Having less modes for cursor positioning will help app side to get things right causing less fuzz/frictions, thats true. If the xterm mode gets not removed, it needs a revamp due to its n+1 cell + scrolling issue. Imho there is some value in that mode for app side if the cursor stops at n, because it can signal the width of a sixel image with an additional CPR, which is not possible with vt340 mode alone (e.g. without inspecting the image itself). So my second suggestion is a compromise to still offer that benefit, but without the scrolling flaw.
From a technical viewpoint I dont care much, whether we have one or 5+ cursor modes (they are really easy to implement on TE side), but for the sake of usability from app side it should not be more than one or two. Currently my addon implements the xterm/mintty mode, but I will change that to vt340 as default. On a technical sidenote - the mintty offset wrapping can be "emulated" with vt340 mode + LF, the xterm mode with my second suggestion + a conditional CRLF or CUF.

If you set a palette value in one image, it will be carried forward to the next image, but only if you assign colors in the order #1, #2, #3 (i.e. following VT340 rules).

Yes I second that, it is basically what I meant with "palette sharing" - colors are carried over.

If you output an image with a new set of palette values, it won't affect previous images you've output by default (i.e. outside of any strict compatibility modes the TE might have).

I agree, again images are basically decoupled in their palette color values (beside some initial palette values from previous image).

If you change palette values half way through an image being output, it may or may not effect the pixels in that image that have already been displayed (in other words, don't to that).

Well, libsixel promotes that as "high-color" - switching the palette within an image. It only works in printer mode, thus is kinda a misuse of an implementation detail. mlterm supports that (prolly all libsixel based TEs), xterm does not (always works with indexed intermediate images, where a color slot change would re-color previous pixels of that image).
Imho this should not be promoted, as it is not covered by any spec notion and cannot be relied on. (Mind you - it theoretically allows to apply 1M RGB colors with a palette size of one by just changing that single slot color over and over.)

What I take from this - we dont even need a private palette option, that would always start from a fixed palette, e.g. ground the palette back to vt340 colors or any other predefined one? I am still unsure about this, as it gives app side some convenience about the colors to be expected. How would one restore the default palette on a VT340 before start drawing?

[hackerb9]

What I take from this - we dont even need a private palette option, that would always start from a fixed palette, e.g. ground the palette back to vt340 colors or any other predefined one? I am still unsure about this, as it gives app side some convenience about the colors to be expected. How would one restore the default palette on a VT340 before start drawing?

Restoring the palette on the VT340 before sending sixels can be done in several different ways. Here are the ones I've seen so far:

0. Do nothing. That's not as ridiculous as it sounds since one can simply use #7 as the foreground and #0 as the background and know that it will match the text colours.
1. Check if the colour table is correct using DECRQTSR (Request Terminal State Report)
2. Set the colour table to known values using DECRSTS (Reset Terminal State)
3. Change the colours using ReGIS (untested)
4. Switch between the two colormaps using DECSTGLT (Set Text / Graphics Look Up Table) [buggy?]
5. Send a sixel file that simply sets the palette and quits. .

There may be others I'm forgetting.

[j4james]

it can signal the width of a sixel image with an additional CPR, which is not possible with vt340 mode alone (e.g. without inspecting the image itself).

If you're an app that actually needs this functionality, which of these options would you honestly choose?

1. Inspecting the image itself, which is guaranteed to work.
2. Using a mode that half the terminals don't support, and those that do don't even implement in the same way, and which also requires an additional CPR query.
3. Yet another mode still to be invented, that'll likely have most of the same problems as 2, but possibly with fewer terminals implementing it.

I really can't imagine this being widely used.

Well, libsixel promotes that as "high-color" - switching the palette within an image. It only works in printer mode, thus is kinda a misuse of an implementation detail.

This is an interesting case, but it adds so much to the file size which I just don't think is justified for typical image viewing in the terminal. It might be something I'd consider as a mode one day if there was a huge demand for it, but I doubt that.

For personal use, I prefer having the ability to do palette animation on the image, even if it only works in my terminal.

How would one restore the default palette on a VT340 before start drawing?

There's always RIS, which most terminals support, but obviously not a great choice for apps to use. I've also considered resetting the palette with DECSTR, since that's what the VT240 does (and a couple of other modern terminals). But even with wider support I wouldn't recommend that to apps, because it can have some weird effects on some terminals. And anyway there's no guarantee what you'll get - different TEs have different defaults.

Again I don't imagine apps relying on something like this just to save a couple of bytes.

[hackerb9]

DECSTR was certainly what I expected would reset the palette before I actually tested it on a VT340. I wish I knew why it doesn't work. Was that a mistake? If it worked on the VT240, then perhaps Digital had a good reason.

On the other hand, DEC's ANSI Compatible Printer Protocol Programming Manual v2 (EK-PPLV2-PM) from 1994 mentions that DECSTR does reset the palette, albeit to all black for mono-printer compatibility.

Color numbers remain assigned even after leaving and reentering Sixel
Graphics mode. At power-up, Soft Terminal Reset (DECSTR), Select
Conformance Level (DECSCL), and Reset to Initial State (RIS), the
device assigns all color numbers to black.

Upon entering Sixel Graphics mode, the device selects color 0.

@jerch writes:

I'd like to propose, that this mode always bootstraps the initial palette from some predefined palette, an app user can expect to work with without any own color definition (we currently use VT340-color for that, I am also fine to use ANSI256 if we can agree on that).

Have you considered using something like the RGB colour map table from EK-PPLV2-PM? In the chapter on sixels, it describes the equation for converting from an index number to RGB values. It is designed so that one can create a sixel image that looks the same on devices with different colour capabilities, but perhaps could be repurposed or modified as a default colour map.

Here is what it says:

RGB Color Map

Table 5-5 shows the colors available for 8-color and 64-color devices. To
transform the offset found in Table 5-5 into its RGB components, apply the
following algorithm to the offset:

C = Offset
B₁ = C mod 4
G₁ = (C - B₁) mod 16
R₁ = (C - B₁ - G₁) mod 64

B = 25 * B₁
G = 25 * G₁/4
R = 25 * R₁/16

Example:

C = Color 41 — Khaki

B₁ = 41 mod 4 = 1
G₁ = 40 mod 16 = 8
R₁ = 32 mod 64 = 32

B = 25 * 1 = 25
G = 25 * 8 / 4 = 50
R = 25 * 32 / 16 = 50

RGB parameters range from 0–100.
x mod y is the remainder when x is divided by y.

Note

If you are writing an 8-color application, use the offsets for red, green,
blue, yellow, magenta, and cyan from the 64-color table. Then, if
your application is used on a 64-color device, the colors will still be as
expected.

[hackerb9]

If by ANSI256 you mean the colours XTerm uses, I think that that would be an acceptable default palette. I'd prefer it if the first sixteen colours were identical to the VT340, but ANSI256 does have some nice properties:

• Very well known
• Has the most important colours in the right palette position for a VT340:
  • #0 (background) is black
  • #7 (text foreground) is light grey
  • #15 (bold text foreground) is white
• Connection with text colours means a terminal emulator may choose to have the sixel palette change when text colours are redefined.

And a few downsides:

• Sixteen colour images for the VT340 would not work on newer terminals nor vice versa.
• Having the colours in the right position isn't actually helpful: it only matters if someone sends the palette to a VT340 and messes up the text colours, but we're talking about the default for when no palette is sent.
• Colour #7 is actually 75% grey, which is too bright for the VT340's 46% grey text.
• Similarly, #15 is 100% white which is too bright for the VT340 which uses 79% grey. (#C9C9C9)
• Changing the sixel colours when changing text colours may have benefits, but the biggest benefit of that connection on the VT340 would not be achieved: being able to create a sixel image that uses the current foreground and background colours, no matter what they are.

Another idea for a default palette

Just spitballing here, but what if the first sixteen colours were defined the same as a VT340, including the special connection with the text colours? Yes, if a user changes the default foreground and background colours, then the palette would be different, but that's actually the way it works on the VT340. One can change those colours in the Set-Up menu and it affects the default palette.

I'm imagining it being bidirectional: setting the text colour changes palette entries #7 and #0, and setting those two entries changes the current text colour. But that's probably not acceptable. It may be possible to make the connection go one way: defining colours in a sixel image affects that current image's palette but would not change the text colour. This might be a problem as it means the palette changes for those two entries would get reset to the text colours when the next sixel image is displayed.

[hackerb9]

cursor at last image cell: I see this as the most natural cursor behavior in terms of cursor advance, if one sees sixel input as a block of cells. This is similar to what xterm and mintty do, but they go one cell further and the nightmare starts - what, if the cursor ends at COLS+1? Should the cursor wrap? Based on which setting? DECAWM? I'd say no, imho no wrapping should happen at all, instead place the text cursor in the last image cell. From there app can move the cursor. This mode has the advantage to automatically give information about the printout size in both image dimensions. If we dont allow any automatic wrapping, the flaw of xterm's mode with the last line scrolling also vanishes.

A small note: the final cursor position does not give information about the size in both dimensions. I think that would have been a good idea, but on the VT340 the text cursor only changes in row, not column. I believe the manual described it as an implicit Graphics Carriage Return ($).

One benefit of this is we do not need to worry about what happens at COLS + 1.

UPDATE: Here is the entry from EK-PLv2 PM, p. 5-10:

5.6.3 State After Exiting Sixel Graphics Mode

After exiting Sixel Graphics mode, the device returns to text mode with the
following conditions:

• The text active horizontal position is the same as just before entering Sixel
  Graphics mode.
• The text active vertical position is the same as the graphics active vertical
  position. As a result, the baseline of subsequent text is contiguous with a
  point 700 centipoints below the top pixel of the last sixel line.
  In addition, the first text mode vertical motion command (LF, VT, and so
  on) causes the device to advance to the next text mode line (get back on
  grid) before executing the vertical motion command.

[jerch]

@j4james

For personal use, I prefer having the ability to do palette animation on the image, even if it only works in my terminal.

Yes thats straight up impossible with immediate coloring as in printer mode. My personal stance here - I'd favor higher output quality over palette animation (I see absolutely no need for the latter). I still dont like the high-color feature as such, because it is not covered by official docs. I kinda accidentally implemented the sixel decoder to work with that feature correctly, but it certainly undermines the original palette idea on old terminals. To me this is not more than an unreliable hack done by libsixel.

about RIS and DECSTR:
I coupled palette reset to both. For RIS is was quite clear to me, but not so for DECSTR. In the end I also went with DECSTR here, because I also purge the image storage (basically resetting whole image handling back to initial values). If there are good reasons not to do this for DECSTR, I would change that. Any clue why DEC went here different ways for VT240 and VT340 (and printers)?

@hackerb9
Thx for the list of possible palette reset commands. But dont they all need to know app side the palette in advance (prolly beside RIS)? So there is no single command in the sense of "load default palette" without any other side effects, right?

Yes with "ANSI256" I mean xterm's default 256-indexed colors. (The term ANSI256 is technically wrong, as there was no 256-color space from ANSI at all, but peeps made that up from the lower 8/16 colors called ANSI colors.)
The fact that this palette is not VT340 compatible is really nasty (and the reason why I switched back to vt340 colors). The effect gets obvious on those old sixel images from the 90s, that dont define own colors (they look quite different with ANSI256).

Just spitballing here, but what if the first sixteen colours were defined the same as a VT340, including the special connection with the text colours? Yes, if a user changes the default foreground and background colours, then the palette would be different, but that's actually the way it works on the VT340. One can change those colours in the Set-Up menu and it affects the default palette.

Thats an interesting idea. Problem I see here - the lower 16 positions in ANSI256 are exactly that on xterm, but xterm simply went with different default low colors in the first place, no clue why. This cannot be backmerged with VT340 colors without frictions, a proper backmerge would mean to also adopt VT340 ANSI color values for text (partially giving up xterm colors). Or a halfway backmerge to went with VT340 colors only in sixel default palette, thus losing the text-image color inter-linkage. I dont have an answer for that atm - what I currently do: use 16 VT340 colors as default, with no relation to any text color (no inter-linkage at all, both may mutate independently).

I really like the inter-linkage idea, as it would give the possibility to more uniform (coupled) coloring, as it would have been on a VT340, but dont know how to bridge the xterm text palette vs VT340 palette differences.

@j4james, @hackerb9 About final cursor state:
Yes I am aware of that handling (still might not understand all details about it).

In terms of the sixel protocol mechanics and how low-level it is, DEC's behavior it the only one that makes sense:

• sixel started somewhere (aka initial offset)
• right extend is not formally specified, may be ragged by individual 6-pixel line progression to the right side
• vertical "print head" position is fixed by amount of GLFs (no backmove possible), thus can be mapped back properly at the end of the image --> final ROW value
• What to choose for final COL value? Right side ragged --> no proper value can be derived, just go with initial COL value (offset)

In that sense the xterm mode is completely flawned, as the spec does not define a "right border" (not even sure how xterm deals with the right ragged problem here). Sixel is just a "print on top" protocol with very limited guarantees of a left border and the overall vertical progression.
On the other side DEC STD 070 tells us to use proper raster attributes ideally with xy extends, and that a conforming encoder may not run beyond these limits. So DEC saw a need for formalizing a rectangular image area, but prolly could only half adopt it through their older already existing lvl1 devices (therefore stating, that a decoder shall not limit outputs by xy extends in raster attributes). Idk the real issues behind that lvl1 vs lvl2 thingy, thus might jump to wrong conclusions here.

So to summarize this for myself - I think xterm mode should be removed completely, as even my compromise to keep it alive without the COLS+1 flaw has that right side ragged issue. In a strict sense it is not possible to derive a right border for sixels.

[j4james]

Just spitballing here, but what if the first sixteen colours were defined the same as a VT340, including the special connection with the text colours? Yes, if a user changes the default foreground and background colours, then the palette would be different, but that's actually the way it works on the VT340. One can change those colours in the Set-Up menu and it affects the default palette.

@hackerb9 This sounds like what I was planning for my "strict" compatibility mode. My intention was the text palette and the graphics palette would be identical, and any changes to one would affect the other. Since the text palette is controlled by the user's color scheme choice, that would apply to the graphics palette as well. If a user wanted to match the default VT340 palette exactly, they would just need to select an appropriate color scheme (I would probably argue to have a VT340 scheme included in the app by default).

For the "modern" usage, though, the text and graphics palettes have got to be separate, so there's no need for them to be the same. And since a lot of modern terminals default the first 16 colors to match the VT340 (or at least something quite similar), that's almost a de-facto standard at this point. After the first 16 there is more variation, but using the XTerm 6x6x6 + grayscale palette is a common choice (although XTerm itself just uses black for this range). And after 256, it'll typically be some shade of black/white/gray.

My personal stance here - I'd favor higher output quality over palette animation

@jerch I totally understand that. I reckon at least half of modern terminals work that way, so it's not an unreasonable choice. But my preference is for my defaults to be as close to VT340 behavior as possible, as long as that doesn't break any widely used modern apps, and there are so many fun things you can do with palette animation, so I would hate to lose that functionality.

In that sense the xterm mode is completely flawned, as the spec does not define a "right border" (not even sure how xterm deals with the right ragged problem here)

Note that the right border mode is an RLogin invention (assuming you're talking about 8452). XTerm just copied that, but not correctly, so they don't exactly match. XTerm's default is to place the cursor in the leftmost column of the screen, which obviously isn't correct either. I'm still hoping to convince XTerm to fix this at some point if I ever get around to writing a patch for it.

[jerch]

@j4james

And since a lot of modern terminals default the first 16 colors to match the VT340 (or at least something quite similar), that's almost a de-facto standard at this point. After the first 16 there is more variation, but using the XTerm 6x6x6 + grayscale palette is a common choice (although XTerm itself just uses black for this range).

If we can agree on that, I'd do it likewise - lower 16 colors from VT340 + filling up to 256 from xterm's 256-indexed colors.

But my preference is for my defaults to be as close to VT340 behavior as possible, as long as that doesn't break any widely used modern apps, and there are so many fun things you can do with palette animation, so I would hate to lose that functionality.

Yeah, and for proper VT340 emulation it is even mandatory that way. I'm still unsure what to make of that immediate vs always indexed coloring issue (printer vs. terminal mode), I def. like the higher quality possibilities by redefining the palette on the fly, but hate it for being off the record. libsixel kinda let it out of the bottle, hard to put it back there. I only see two ways for this, either discourage its usage completely, or try to formalize it somehow (as extension on XTSMGRAPHICS?), so app side can use it conditionally, if the TE indicates support.

Note that the right border mode is an RLogin invention (assuming you're talking about 8452). XTerm just copied that, but not correctly, so they don't exactly match.

Oops, thx for clarification. My association with xterm here was not meant to blame anyone, it just occurred first to me from xterm settings (did not know about RLogin).
Well, I gonna remove these cursor xterm/Rlogin/mintty modes for now, as they dont fit the basic sixel idea (RLogin) or can easily be emulated from VT340 behavior (xterm+mintty). The right border idea is nice for strictly rectangular images, but thats unreliable for sixels.

[jerch]

My take-aways from this thread:

• remove all foreign cursor modes, just stick with VT340 mode for DECSDM=off
• remove private/shared palette separation, palette entries are always carried forward, if not reset by other means in between

Thx for the discussion, helped me to get some things straight.

One more thing that still bugs me - there is no easy way to tell the TE to reset to the initial palette (yes RIS and DECSTR can do that be meh, with heavy side effects). So what would you say about this idea - whenever app side requests palette size change from XTSMGRAPHICS, it also would reset to the initial palette? This could be made as a guarantee, so even if the size does not change, at least the palette would get refreshed. I know thats like piggybacking it, but a "naked" palette resize leads to weird undefined state, so it came to me to couple it with a palette reset.

[dankamongmen]

One more thing that still bugs me - there is no easy way to tell the TE to reset to the initial palette (yes RIS and DECSTR can do that be meh, with heavy side effects). So what would you say about this idea - whenever app side requests palette size change from XTSMGRAPHICS, it also would reset to the initial palette? This could be made as a guarantee, so even if the size does not change, at least the palette would get refreshed. I know thats like piggybacking it, but a "naked" palette size change leads to weird undefined state, so it came to me to couple it with a palette reset.

you might want to tie this in with XT*COLORS https://sw.kovidgoyal.net/kitty/color-stack/