Static FFMPEG packaging.

[NevermindNilas]

Feature Request

Hey,

Is there any possibility that you could make static wheels with all of the dependencies included just like how pytorch-cuda does it?

Recent Windows issues suggest that the current approach is relatively brittle. From an end user perspective, having self contained, reproducible wheels would be significantly more reliable than requiring software developers and the consequent users of said softwares to set up shared FFMPEG, adding it to Path and hoping that everything works.

I am more than happy to help with this migration if considered.

( I didn't realise that I misclicked on documentation issues, I changed the title )

[NicolasHug]

Hi @NevermindNilas

Thanks for bringing that up. Shipping FFmpeg binaries with the wheels can be problematic in terms of licensing, because FFmpeg codecs may be GPL-licensed. TorchCodec is BSD-licensed, and I'm not sure we'd be able to retain the BSD license of the wheels if we were to ship FFmpeg binaries in there. I acknowledge that some packages do this, but whether their wheels can retain their non-GPL license (e.g. BSD or MIT) isn't clear to me. In TorchCodec, we'd rather be on the safe side and not risk that.

Shipping FFmpeg binaries in the wheel isn't a silver bullet, and may come with its own incompatibilities problems: if TorchCodec links against a shipped FFmpeg binary but the user relies on another lib that also relies on their own FFmpeg binaries (whether they're system-wide or local to a wheel), then symbol conflicts are likely to occur.

[NevermindNilas]

Hello @NicolasHug ,

Thanks for the insight, that makes sense especially around the licensing side and ABI conflicts.

One thing I am still trying to wrap my head around is, how can torchvision ( with it's FFMPEG backend ) or PYAV manage this all while being BSD licensed. Both effectively avoid shipping GPL licensed code/libraries afaik.

This is not legal advice, so please correct me if I’m mistaken, but would an alternative approach be acceptable where the pip wheel itself does not ship FFmpeg binaries, and instead:

• downloads an official shared FFmpeg build during pip install torchcodec.

• extracts only the required shared libraries.

• and treats FFmpeg as an external runtime dependency rather than a redistributed artifact?.

I think this would also be a pain to support but at a significantly improved UX.

[FFY00]

Shipping FFmpeg binaries in the wheel isn't a silver bullet, and may come with its own incompatibilities problems: if TorchCodec links against a shipped FFmpeg binary but the user relies on another lib that also relies on their own FFmpeg binaries (whether they're system-wide or local to a wheel), then symbol conflicts are likely to occur.

On any modern POSIX system, the linker tree is namespaced by default (old(?) Android being the only exception AFAIK) when loading dependency libraries, so it wouldn't really pose that much of an issue.

[FFY00]

Also, as far as the ffmpeg executable itself goes, I don't really see any issue?

[NicolasHug]

@NevermindNilas

One thing I am still trying to wrap my head around is, how can torchvision ( with it's FFMPEG backend ) or PYAV manage this all while being BSD licensed. Both effectively avoid shipping GPL licensed code/libraries afaik.

I can't speak for PyAV, but I'm familiar with the torchvision case: I think this licensing issue was the reason the FFmpeg backend was mostly only available when building from source. If TorchVision ever shipped FFmpeg libs in its wheels, then that was probably by mistake. There's also, I think, a distinction between the code-source being BSD, and the license of the wheels themselves. They can be different I think. Both TorchVision and TorchCodec aim to be BSD for both the code-base and the wheels. This "dynamically load the user-installed FFmpeg libraries at runtime" solution allows TorchCodec to be all-BSD (this solution originated in TorchAudio). TorchVision never had such a logic.

would an alternative approach

I think that'd be OK in terms of licensing. But what you're describing looks a lot like a full-fledge package manager, with dependency resolution, cross-platform considerations, etc. :D i.e. as you noticed, something that is far from trivial in terms of implementation and maintenance. For now, we choose to let the user install FFmpeg themselves. It works well for Linux which is still our primary user-base. Eventually, we hope we'll be able to address the long-tail of issues like those on Windows, and that the UX will be great across all platforms.

@FFY00 you're probably right - note though that this issue was mostly motivated by installation issues on Windows, not POSIX systems. Linux (and I think MacOS) work fine. We don't rely on the FFmpeg executable itself, only the FFmpeg libraries (libavcodec , etc.)

[NevermindNilas]

I think that'd be OK in terms of licensing. But what you're describing looks a lot like a full-fledge package manager, with dependency resolution, cross-platform considerations, etc. :D i.e. as you noticed, something that is far from trivial in terms of implementation and maintenance. For now, we choose to let the user install FFmpeg themselves. It works well for Linux which is still our primary user-base. Eventually, we hope we'll be able to address the long-tail of issues like those on Windows, and that the UX will be great across all platforms.

Thanks for the confirmation, but the implementation I had in mind was significantly simpler than a full fledged package manager 😄

To be more precise, I was thinking of something closer to:

• downloading a single, known good FFmpeg binary bundle per platform during pip install ( say ffmpeg-v8-shared.zip )
• extracting only the required shared libraries into a directory. ( get libavcodec, .. )
• loading those libraries explicitly at runtime (similar to what TorchAudio already does if my memory serves me right )

In other words, we are not solving the dependency resolution in the general sense, but isntead pinning to a known, controlled set of FFMPEG build(s) that TorchCodec is tested against and therefore treating FFMPEG as an external runtime artifact rather than a redistributed one.

I think this would also be a pain to support but at a significantly improved UX.

When I said the above, I meant that it would be a 'pain' from the FFMPEG version(s) support. From the README, TorchCodec seems to support FFMPEG v4-8.

If it is alright with you and if you see any potential in this, I would love to try implementing it as a test, for now with FFMPEG v8.

[NicolasHug]

I think that solving this correctly, i.e. making sure there is no conflict between the downloaded FFmpeg binaries and the existing user's environment, is a hard problem - that's the problem package managers like conda are there to solve.

We're actively working on improving the FFmpeg installation UX and guidelines at the moment, so I hope we'll be able to significantly reduce installation friction soon, but I'm afraid downloading pre-selected FFmpeg binaries on the fly isn't in scope at the moment.