Feature request: Support for DJI Mini3 DNGs lens correction

[WiLars]

Is your feature request related to a problem? Please describe.
The DJI Mini3 drone generates DNGs that have embedded lens correction profiles. These profiles are not applied on import or in the lens correction module. Correction using Lensfun is not possible, because the vignette is not only related to luminance but also has a significant color shift (/lensfun/lensfun/issues/1963). This color shift basically makes the raws unusable in Darktable.

Describe the solution you'd like
Apply the lens profiles embedded in the DNG files in the context of the lens correction module

Alternatives
Preprocessing the DNG files, so that Darktable doesn't have to apply correction profiles. I'm actually not sure how I could do that.

Additional context
I think the foundation for this was implemented in /pull/12714 . This functionality works great for my camera, but not for DJI Mini3.
@paolodepetrillo, do you have some additional insights why this does not work? As you shortly indicated in this comment you already looked into DJI drones? It looks like you have a deep understanding of OpcodeList in the DNG file format.

[Nilvus]

@sgotti: what's your view on that?

@WiLars: given your FR, darktable version related actually seems obvious but think about precising it anyway in future FR/issues. There's a template that request this and it's there on purpose. Thanks!

[WiLars]

@Nilvus thanks for the hint. I think in the template for feature requests doesn't request that info, so i wasn't aware.
Anyway, here is some additional info, as found in the bugreport template:

Platform

• darktable version : 4.2.0
• OS : Linux Kernel 5.15.85-1-MANJARO
• Linux - Distro :Manjaro
• Memory : 32 Gb
• Graphics card : Nvidia RTX 3060Ti
• Graphics driver : 525.60.11
• OpenCL installed : no
• OpenCL activated : no
• Xorg : X.Org X Server 1.21.1.6
• Desktop : KDE
  installed from repo, so i think built parameters do not really apply?

Additional context

• Can you reproduce with a RAW or Jpeg or both? only raw, in jpeg lens corrections are already applied by drone
• Are the steps above reproducible with a fresh edit (i.e. after discarding history)? yes
• If the issue is with the output image, attach an XMP file if (you'll have to change the extension to .txt)
• Is the issue still present using an empty/new config-dir (e.g. start darktable with --configdir "/tmp")? yes
• Do you use lua scripts? no

I think i can't just upload a raw file as example here, but if anyone is interested I can provide some.

[Nilvus]

@Nilvus thanks for the hint. I think in the template for feature requests doesn't request that info, so i wasn't aware. Anyway, here is some additional info, as found in the bugreport template:

You're right because for feature request, most of infos of bugreport template are not needed (included for most FR requests) darktable version. Your request is something between camera support and feature request. And for camera support, we often have some request with existing issues or fixed in a previous release.

[kmilos]

Please upload samples to raw.pixls.us, and also share here via some cloud service.

[WiLars]

here you can find one actual photo and a white one I took for my experiments with lensfun. The white one is a photo of a white monitor taken through a diffusor, so it is supposed to be uniformly white. In that one you can clearly see the color shift

I also uploaded the landscape top-down to raw.pixls.us

[kmilos]

So, there is

OpcodeList3                     : FixVignetteRadial, GainMap

and "should be applied to the raw image, just after it has been demosaiced" (which is unsupported by dt currently obviously).

Edit: Neither of these is specified as optional; FixVignetteRadial applies to all color planes per definition, while GainMap is a per-plane 65x49 map...

[spaceChRiS]

Related: #12880
As GainMap is already implemented for OpcodeList 2 (pre-demosaicing), but is here used in OpcodeList 3 (post-demosaicing), the OpcodeList would have to be distinguished in the internal metadata structure if I understand it correctly.

[kmilos]

As GainMap is already implemented for OpcodeList 2, but is here used in OpcodeList 3, the OpcodeList would have to be distinguished in the metadata if I understand it correctly.

Not only metadata (which is the easy part) - it has to be applied in the correct place in the pipeline - OpcodeList2 is applied to single-channel raw CFA data (currently implemented in the rawprepare module right after the file is opened); OpcodeList3 must be applied just after demosaic and applied to 3 color channels, and this is currently not implemented.

Not 100% sure we can even apply the FixVignetteRadial OpcodeList3 in the lens correction module (which could've been relatively easy to extend), as it might be in the incorrect place in the pipeline wrt to the DNG spec...

And then there is the crop issue @jenshannoschwalm mentioned...

[jenshannoschwalm]

I have pending work related to dng lens correction and indeed that should be as early as possible after demosaicing (or sort of Post-Processing there)

[kmilos]

It gets somewhat more complicated - this sample seems to have a (weird?) 2x digital zoom scheme: crops out middle 2000x1500, then scales pixels by 2x to get back to "expected" 4000x3000 output...

What is not clear is when the OpcodeList3 is applied - it could be to the whole active area before this crop+rescale because the FixVignetteRadial is unchanged? (The GainMap is again on a 4000x3000 size, but that doesn't tell me it's before or after.)

[paolodepetrillo]

@kmilos From that sample gallery of 4 images, some have the 2x digital zoom but 1xsingle.DNG does not. The GainMaps for all of them regardless of the 1x / 2x zoom are similar, which indicates that the OpcodeList3 should be applied to the entire image before the crop. If it was intended to be applied after the crop, the GainMap for the 2x image would look like a cropped version of the GainMap from the 1x image.

[kmilos]

Indeed, as the identical FixVignetteRadial suggests as well...

[paolodepetrillo]

@WiLars As a short term workaround until this feature is implemented in darktable, you can use my dngpreprocess utility to apply the corrections outside of darktable, resulting in a new dng file that already has the corrections applied and can be edited in darktable.

Instructions to build and use are here: https://discuss.pixls.us/t/pixel-7-pro-raw-processing/33942/32

The "-3" option must be specified to apply the OpcodeList3 processing for the DJI files:
dngpreprocess -3 DJI_9999.dng

[WiLars]

@paolodepetrillo thank you for the hint, just tried it, and with your dngpreprocess the raws look much better!

I addded some exported files (jpeg) to the folder. There is still some shade on top and bottom and some slight color shift, but I suppose this is related to the quality of the embedded profiles...