WebVTT should allow pages to modify the caption display area

[jernoble]

A common use case for captions is to modify the caption display location when custom media controls show and hide. This behavior is implemented in the default controls in Safari, Chrome, and Firefox.

However, sites with custom media controls are unable to implement the same behavior, at least not in a standards-compliant way. Sites have attempted to reach inside the UA-specific media element Shadow DOM, and use an internal-only CSS selector, but this breaks when User Agents (correctly) restrict access to internal-only Shadow DOM nodes or rename internal elements.

WebVTT should try to solve this important use case. It may be as simple as codifying a "Caption Display Area" as a block-level element, and expose it through a pseudo-element. Then pages could modify the display area by giving that selector, e.g., a padding-bottom: rule.

[jernoble]

Safari and Chrome both name this block-level element useragentpart="-webkit-media-text-track-container". Firefox names it class="caption-box". I'm personally partial to the "container" nomenclature (see my comment in issue #522).

I propose exposing this as ::cue-container, and allowing top:, left:, bottom:, and right: properties (as well as transition:, so these changes can animate).

[nigelmegitt]

Given that a common UI pattern on videos is to put controls along the top and the bottom, and big buttons in the vertical centre, a single caption area doesn't seem like a good design choice. Typically there are multiple potential regions in which subtitles could go, and a common goal is to move them as little as possible to deconflict with the controls.

Another concern we need to address if changing the container size is the reference area for font sizes - see e.g. #522 - it needs to remain clear what the container is for container-relative font sizing, and whether the expectation is that text will change size when the caption area changes size.

This general issue exists more broadly than WebVTT - arguably some solution is needed at the media element level instead of in WebVTT. For example, it might be time to revisit the design choice to prohibit <video> elements containing content, to see if we can change that in a backwards-compatible way.

[jernoble]

Well shoot, I already proposed using ::cue-container in PR #528.

[jernoble]

Given that a common UI pattern on videos is to put controls along the top and the bottom, and big buttons in the vertical centre, a single caption area doesn't seem like a good design choice.

Let's not make this more complicated than it needs to be. When sites roll-their-own-captions, they don't tend to move the captions out from under the big button in the middle of their video.

Another concern we need to address if changing the container size is the reference area for font sizes - see e.g. #522 - it needs to remain clear what the container is for container-relative font sizing, and whether the expectation is that text will change size when the caption area changes size.

I agree. The concept of the "video viewport" has problems, not the least of which is the ambiguity of "viewport" when it comes to CSS. This new "caption rendering area" object would be the natural choice of a "containment block", and a font-size: 5cqmin would be relative to the "caption rendering area". If a site wanted to push captions up without changing the font size (and knew the captions were bottom-aligned), they'd do something like:

.controls-visible video::cue-rendering-area { top: -3em; bottom: 3em; }

Which would shift the caption rendering up without changing its size.

If, on the other hand, they wanted to dodge both top and bottom controls, they might do:

.controls-visible video::cue-rendering-area { top: 3em; bottom: 3em; }

Which would change the font size to be 0.3em smaller. If they wanted a fixed font size regardless of the container size, they could just specify it; e.g. ::cue { font-size: 24px; }, or they could even do something fancy like ::cue { font-size: calc(5cqh + 0.3em); }

We'd specify this by saying something like:

Initialize the Cue Rendering Area with the following CSS Settings:

• contain: layout

[nigelmegitt]

Let's not make this more complicated than it needs to be. When sites roll-their-own-captions, they don't tend to move the captions out from under the big button in the middle of their video.

BBC is a counter-example of a large site that both defines its own controls and rolls its own captions and does indeed move them - I welcome further data points.

[nigelmegitt]

.controls-visible video::cue-rendering-area { top: -3em; bottom: 3em; }

Which would shift the caption rendering up without changing its size.

Would you also expect captions at y position 0 to appear outside the video rendering area in this case? I'd by unsurprised by an implementation that clipped the cue rendering area to the video rendering area, and ended up with a cue rendering area 3em smaller vertically, and then scaled the text as a result.

[jernoble]

BBC is a counter-example of a large site that both defines its own controls and rolls its own captions and does indeed move them - I welcome further data points.

Ok. That's not something that cannot currently be accomplished with WebVTT, but if it were, it'd be by allowing the transform: property on ::cue, which is a separate issue from this one.

Would you also expect captions at y position 0 to appear outside the video rendering area in this case?

No, it would likely clip.

I'd by unsurprised by an implementation that clipped the cue rendering area to the video rendering area, and ended up with a cue rendering area 3em smaller vertically, and then scaled the text as a result.

This would be a site choice. They could do either "move + clip", or "squish and scale", or even "squish but keep text size constant" depending on what CSS they apply.

[nigelmegitt]

it'd be by allowing the transform: property on ::cue, which is a separate issue from this one.

I'm confused - I thought that allowing pages to provide enough information to allow deconfliction of captions with custom controls is exactly this issue. If allowing transform: is a solution, isn't that one that could be used more generally?

[jernoble]

No, it would likely clip.

Actually, reading 7.2 Processing Cue Settings step 10.12-14, a cue at 0 would not clip; it would be moved to stay within the title area.

[jernoble]

I'm confused - I thought that allowing pages to provide enough information to allow deconfliction of captions with custom controls is exactly this issue. If allowing transform: is a solution, isn't that one that could be used more generally?

Insofar as allowing control over placement of individual cues through CSS necessarily allows placement of all cues through CSS, then yes. However, that is a much, much more complicated solution, both for implementors and for pages.

In the use case where a page would want to avoid the top and bottom controls, it would have to somehow target certain cues to apply a "move down 3em" rule, and other cues to apply a "move up 3em" rule. To do so would likely involve javascript, at which point the script could just modify the cue.line property instead.

Modifying the cue display area to achieve the same effect is much, much simpler, works with the existing implementation of WebVTT in all three major browsers, and solves most use cases while not precluding solving others.

[JibberJim]

Where vtt regions are supported, do players deconflict the regions? As then deconfliction can be achieved by adding cues that describe the same region as the controls without any change?

[jernoble]

Yes, cues in regions should push aside other cues. However, a lot depends on the ordering of cues within the results of obtain a set of CSS boxes step, something that is currently un-specified.

[gkatsev]

I'm wholeheartedly is support of this. While it doesn't solve all use-cases as Nigel mentioned, I think this does solve the 80% use-case and can help people from doing weird things that eventually need to be turned off.

Which, actually leads me to this previous discussion on the subject: #503. Specifically, one suggestion there was to be able to pass the video element dom regions that it should avoid when rendering cues.

That said, I'd be ok with adding this change in and separately pursue other changes that make things better, like continue the discussions from #503 and probably allow more CSS properties on ::cue, like transform:.

In #528 I suggest renaming that one to ::cue-backdrop and then we're free to use ::cue-container here.

[jernoble]

I've updated the PR in #528 to use ::cue-backdrop, so we're free to paint the bike shed ::cue-container here.