Markdown output

[davesnx]

Hi,

This PR might come out of the blue, but there's a good reason. Let me explain myself in the "Why" below.

Also, when I was half done, I discovered a stale PR (#791) targeting the same problem, and I thought it could be helpful to other people.

Why

We use odoc in Melange's documentation, to generate API references for Melange libraries, such as Belt, Js, and Stdlib. While the rest of the documentation is handled by vitepress a static site generator working with Markdown files, with a lot of interesting features.

This works reasonably well, but creates a big barrier between the documentation sites:

• Can't reference the melange.re docs articles from inside mld
• Search from melange.re docs don't find anything inside the generated HTML
• Users go from a unified experience to a different experience, melange.re is a SPA while generated HTML is an MPA
• Navigation on the header and sidebar isn't the same and can't be unified
• and a few more small details, such as favicon, tracking scripts, same syntax highlight and all features that a static site generator may give

For this reason, I wanted to generate markdown from odoc and (in the melange case) have another process that manipulates those markdown files to expand the documentation.

Another big use case for Markdown generation is Github, or any markdown renderer platform really.

How

I forked the HTML backend into a markdown2, removed what's unnecessary, and implemented Markdown construction with cmarkit.

I currently name it markdown2 since there is odoc_md and doc_of_md under the markdown folder. I suggest renaming markdown -> odoc-md (or just md) and markdown2 -> markdown in a new PR

Notes about implementation

• References aren't a thing in Markdown's headings. I removed them, but I could always render HTML and keep the references, but it would be better to have an option for this. WDYT about --allow-html where I enable this functionality of outputting HTML
• Similar cases for video/audio. Which isn't supported in pure markdown, but is supported in the CommonMark spec. We could fallback to HTML
• I left a few TODOS in the code that are mostly doubts/questions about odoc that could be good to have a 2nd pair of eyes
• Testing has been done in a few cram tests (mostly markdown-with-belt.t and markdown.t). I could probably split them or organise them differently.

Example

melange-re/melange-re.github.io#224

Future work

• Add frontmatter into markdown
• The —allow-html option I mentioned above
• Talk about ODOC_SYNTAX="re_and_ml" Support for rendering both re and ml syntaxes, within the same generation #1342

[davesnx]

I updated this PR with davesnx#1 without the cmarkit library. I worked on a separate branch to make the review easier. I added an attribution comment, but should other attribution be made on the LICENSE?

[davesnx]

i'm not very familiar with source-impl, but I wanted to add the implementation in case was helpful somewhow, would be nice to ensure this is correct

[jonludlam]

I was playing around with this and found that the includes don't make it to the output. For example, with the file foo.mli containing:

module type X = sig
  type t = int
end

module type T = sig
  include X
end

the output file Foo-module-type-T.md just contains:

# Module type `Foo.T`

[jonludlam]

I've also been playing with Claude code, so there's a commit fixing it on my fork here: jonludlam@926cca1 - do with that what you will :-)

[davesnx]

I can't reproduce the issue, even though the code might make sense. Pushed a cram update here: df7f5f1

[jonludlam]

I've dumped the contents of the two new modules in the test - do you see what I mean now?

[davesnx]

Gotcha, includes on their respective pages

[jonludlam]

Merged!

[dbuenzli]

Could this please be properly attributed. It's a bit baffling, I publish the code of my projects under one of the simplest license to abide to and people still fail to comply with it when they cut and paste or modify the code. The terms here seem pretty clear (emphasis is mine):

Copyright (c) 2020 The cmarkit programmers

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

Is it that complicated ?

Now I'm extra nice and won't complain if you add a simple spdx-license identifier rather than the full permission notice.

Basically just retain the simple header you can find in absolutely each of the sources you copy from. So that would be, at your formatting convenience:

(*---------------------------------------------------------------------------
   Copyright (c) 2020 The cmarkit programmers. All rights reserved.
   SPDX-License-Identifier: ISC
  ---------------------------------------------------------------------------*)

Or if you prefer:

(* Part of this code is:
   Copyright (c) 2020 The cmarkit programmers. All rights reserved.
   SPDX-License-Identifier: ISC *)

[jonludlam]

I'm really sorry, Daniel, you're right. I'll fix this.