Lua: silent unintuitive behavior when passing Inlines as caption to Figure

[bpj]

(I guess this one is for @tarleb since it seems to concern the Lua API.)

The use case

I have a filter which turns divs with class fig into figures, using the first child as caption text if that child is a Para.

The problem

I misunderstood the documentatiin and thought that the Figure caption should be a list of inlines rather than a list of blocks so at first I disemboweled the Para. I noticed that when converting to PDF via LuaLaTeX whitespace in the caption disappeared. On investigation I found that this is because the Figure constructor splits the the list of inlines on Str elements and puts each word in a Plain. It so happens that this isn’t visible in rendered HTML but is visible in typeset LaTeX.

Even though the caption should be a list of blocks this behavior seems to not fit the behavior of other constructors where you can pass partial content — a single element rather than a list or even a string and the constructor Does What You Mean. I think that this should do the same or throw an error.

To reproduce run the code below with pandoc-lua.

local pdc = require("pandoc")

local para = pdc.Para('Some para text')

local capt = pdc.Inlines('Some caption text')

for _, elem in ipairs({ para, capt }) do
  print(elem)
  print("")
end

local doc = pdc.Pandoc({ para, pdc.Figure({ }, capt) })

print(doc)
print("")

local formats = {
  'native',
  'html',
  'latex'
}

for _, fmt in ipairs(formats) do
  print(fmt)
  print(pdc.write(doc, fmt))
  print("")
end

[tarleb]

This kind of behavior also shows in other parts of the API where a list of blocks is expected, e.g. with table cells and list items.

The root cause is this part in our code:

-- | Try extra-hard to return the value at the given index as a list of
-- 'Block's.
peekBlocksFuzzy :: LuaError e
                => Peeker e [Block]
peekBlocksFuzzy idx =
      ((:[]) <$> peekBlockMetamethod idx)
  <|> peekList peekBlockFuzzy idx
  <|> (pure <$!> peekBlockFuzzy idx)
  <|> (failPeek =<<
       typeMismatchMessage "Block, list of Blocks, or compatible element" idx)

This means we're first trying to read a value as a list of blocks before falling back to reading it as a singleton Block. But because a single Inline value can be interpreted as a Plain value with that element as content, that first parser succeeds, and the list of inlines becomes a list of Plain blocks.

I think switching the order in which we try retrieve the value should fix this, but I'm not 100% sure why we do it the way we are doing right now. Maybe it's just for performance, in which case we should measure the impact of changing the order. If the effect is small, then we should go for the more intuitive behavior.

[bpj]

It might be a good Idea to specify in the documentation when DWIM works and how and also when it doesn't work.

Anyway I made my filter a bit more robust by having it look for a div with class caption as first child of a div with class figure in order to potentially support multi-paragraph captions, so I'm good now as far as that is concerned