Skip to content

ffmpeg with nice/ionice #1192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pwilczynskiclearcode wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

ffmpeg with nice/ionice #1192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions video/ffmpeg-nice.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash -ex
# Run ffmpeg with low CPU (nice) and IO (ionice) priority
# so it doesn't interfere with other processes too much
nice -n19 ionice -c2 -n7 ffmpeg "$@"
3 changes: 2 additions & 1 deletion video/transmux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ func MuxTStoMP4(tsInputFile, mp4OutputFile string) ([]string, error) {
// transmux the .ts file into a standalone MP4 file
ffmpegErr := bytes.Buffer{}
err := ffmpeg.Input(tsInputFile).
SetFfmpegPath("/path/to/ffmpeg-nice.sh").
Copy link
Contributor Author

@pwilczynskiclearcode pwilczynskiclearcode Mar 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://pkg.go.dev/github.com/u2takey/ffmpeg-go#Stream.SetFfmpegPath

Output(mp4OutputFile, ffmpeg.KwArgs{
"analyzeduration": "15M", // Analyze up to 15s of video to figure out the format. We saw failures to detect the video codec without this
"movflags": "faststart", // Need this for progressive playback and probing
Expand Down Expand Up @@ -81,7 +82,7 @@ func MuxTStoFMP4(fmp4ManifestOutputFile string, inputs ...string) error {

timeout, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
cmd := exec.CommandContext(timeout, "ffmpeg", args...)
cmd := exec.CommandContext(timeout, "/path/to/ffmpeg-nice.sh", args...)
Copy link
Contributor Author

@pwilczynskiclearcode pwilczynskiclearcode Mar 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pseudo-code or just a starting point for discussion. I'm not even sure if these two places are the correct ones for VOD processing

Copy link
Collaborator

@emranemran emranemran Mar 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just the MP4 generation step. There's ffmpeg usage in different places for for segmenting, thumbnailing, clipping, etc. Just grep-ing ffmpeg or CommandContext | grep ffmpeg should get you most of the instances.

Copy link
Contributor Author

@pwilczynskiclearcode pwilczynskiclearcode Mar 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're not fully sure yet but it looks like some ffmpeg for big VODs are congesting both disk writes on datapacket servers and generate big egress spikes resulting in entire server slowdown and k3s restarts.
We're testing with Rafał if moving VOD processing onto just one of the nodes in the cluster results in less or zero restarts of the rest of the nodes.


var outputBuf bytes.Buffer
var stdErr bytes.Buffer
Expand Down