Bad syntax highlighting in Kconfig file

[miggazElquez]

This file in zephyr has a wrong syntax highlighting (everything is red): https://elixir.bootlin.com/zephyr/v4.2.1/source/kernel/Kconfig.device

I suppose Elixir did not manage to detect the file type correctly ?

This is the only file I found with this issue, all other Kconfig files I checked in Zephyr were displayed correctly.
The issue seems to be only with the highlighting, the symbols are handled correctly.

[fstachura]

Hello, pygments (python library elixir uses to format code) selects systemd config file lexer for this particular file, because ".device" extension is assigned to systemd files. We will need to introduce an ability to override lexers for particular projects.

https://github.com/pygments/pygments/blob/0328cfaf1d953b3a0c7eb0ec0efd363deb2f9d51/pygments/lexers/configs.py#L120

>>> pygments.lexers.guess_lexer_for_filename("Kconfig.device", open("Kconfig.device").read())
<pygments.lexers.SystemdLexer>

[tleb]

That was quick Franek, thanks! I was doing a similar test on my own, using the CLI pygments tool:

⟩ wget -O Kconfig.device "https://elixir.bootlin.com/zephyr/v4.2.1/source/kernel/Kconfig.device?raw"
⟩ cp Kconfig.device Kconfig
⟩ pygmentize -N Kconfig
kconfig
⟩ pygmentize -N Kconfig.device
systemd
⟩ # Also confirmed without `-N`: in one case it looks highlighted and not in the other.

• Should we report back to Pygments?
• Is it really a bug as there is ambiguity in which should apply first?
• Or should we fix it at our layer by forcing it to highlight as Kconfig files that start with such substring (only on Zephyr or all projects?)?

[fstachura]

Should we report back to Pygments?

I don't think so, guessing lexer by filename probably cannot get much better than it is right now. I think it's our responsibility to override the lexer here.

Is it really a bug as there is ambiguity in which should apply first?

It is a fact that Elixir is not using the correct lexer. (But feel free to change the issue type)

Or should we fix it at our layer by forcing it to highlight as Kconfig files that start with such substring (only on Zephyr or all projects?)?

That's the simplest solution and it's what we did for SLexer -> GasLexer override here.

[miggazElquez]

I did a find with all Kconfig.something where .something is an extension recognized by Pygments, and I found more cases:

https://elixir.bootlin.com/zephyr/v4.2.1/source/kernel/Kconfig.vm
https://elixir.bootlin.com/zephyr/v4.2.1/source/subsys/mgmt/osdp/Kconfig.cp
https://elixir.bootlin.com/zephyr/v4.2.1/source/soc/espressif/esp32/Kconfig.mac
https://elixir.bootlin.com/zephyr/v4.2.1/source/soc/intel/intel_ish/intel_ish5/pm/Kconfig.pm
https://elixir.bootlin.com/zephyr/v4.2.1/source/lib/posix/options/Kconfig.timer

Some of the match did work correctly (https://elixir.bootlin.com/zephyr/v4.2.1/source/lib/posix/options/Kconfig.fs is not colored as F# for instance)

[fstachura]

Some of the match did work correctly (https://elixir.bootlin.com/zephyr/v4.2.1/source/lib/posix/options/Kconfig.fs is not colored as F# for instance)

Interesting. The mechanism of matching a lexer by text is involved. See analyse_text methods of ForthLexer (also matches *.fs) and FSharpLexer, and guess_lexer_for_filename. In case of Kconfig.fs, all three lexers return probability 0.0 (Kconfig lexer does not even have analyse_text). So it's basically an accident that the Kconfig lexer is even selected.

I guess we could improve pygments somewhat by implementing analyse_text for the Kconfig lexer, but I think it would also make sense to have this hard-coded in Elixir. We can be sure that all Kconfig.* files for Zephyr are actually Kconfig files.