A native C++ library for low-latency decoding of H.264 video and AAC audio streams using FFmpeg.
The decoder keeps the FFmpeg codec context alive for the entire streaming session, which allows decoding dependent frames (P/B) and significantly reduces latency compared to reinitializing the decoder for every frame.
The library is designed to be used through FFI bindings (for example from Flutter), but the native decoding layer itself is platform-agnostic.
- H.264 video decoding using FFmpeg
- AAC audio decoding
- Stateful decoder context (SPS/PPS and reference frames are preserved)
- Low latency streaming support
- Cross-platform architecture
- iOS
- Android
- FFI-friendly API
The decoder expects encoded packets and produces decoded frames.
Input
- H.264 encoded video packets
- AAC encoded audio packets
Output
- Video frames in RGBA format
- Audio samples in PCM
The library itself does not handle networking or buffering.
It only performs decoding and pixel format conversion.
h264_stream_decoder/
├── header/ # public headers
├── source/ # C++ implementation
├── android/ # Android build and integration
├── ios/ # iOS build and integration
├── docs/ # additional documentation
└── CMakeLists.txt
Additional technical details:
Platform-specific build and integration instructions are described in:
This project depends on FFmpeg 7.1.
Make sure your FFmpeg build and distribution comply with the appropriate FFmpeg license (LGPL or GPL depending on configuration).
- The decoder is stateful and must receive an IDR keyframe before decoding dependent frames.
- When joining a stream mid-session, P/B frames should be dropped until the first IDR frame arrives.
- The library performs pixel format conversion from YUV420P → RGBA before returning frames to the client.
This repository contains example code for educational and experimental purposes.
Please ensure compliance with FFmpeg licensing when distributing binaries.