FFmpeg is a general-purpose tool for processing audio, video, subtitles, and containers. Its core capabilities include demuxing, decoding, filtering, encoding, muxing, and stream copy. It solves common multimedia tasks such as format conversion, stream extraction, merging, and synchronization. Keywords: FFmpeg, transcoding, -map stream selection.
Technical specification snapshot
| Parameter | Description |
|---|---|
| Project/Tool | FFmpeg |
| Primary Language | C |
| Processing Targets | Video, audio, subtitles, containers |
| Common Protocols/Inputs | Files, pipes, network streams, device capture |
| Repository Popularity | Star count not provided in the source |
| Core Dependencies | libavutil, libavcodec, libavformat, libavdevice, libavfilter, libswscale, libswresample |
FFmpeg is a multimedia processing toolchain built around streams.
FFmpeg is often called the “Swiss Army knife” of audio and video processing. It does far more than format conversion. It can extract streams, merge media, compress content, capture screenshots, turn images into video, process subtitles, and synchronize timelines.
From an engineering perspective, the core object FFmpeg works with is not the file, but the stream. A media file often contains a video stream, one or more audio streams, subtitle streams, and metadata. Nearly every FFmpeg command revolves around these streams.
ffmpeg [global_options] {[input_file_options] -i input_url} ... {[output_file_options] output_url} ...
# Global options apply to all inputs and outputs
# -i declares an input source
# Output options apply only to the output file that immediately followsThis command skeleton defines FFmpeg’s basic execution model: declare the inputs first, then specify how to process them, and finally generate one or more outputs.
FFmpeg’s core libraries have clearly separated responsibilities.
Understanding the core libraries helps you map command-line options back to the internal processing pipeline.
| Core Library | Responsibility |
|---|---|
| libavutil | General utility library with data structures, math helpers, and foundational capabilities |
| libavcodec | Audio and video encoding/decoding |
| libavformat | Demuxing and muxing, responsible for containers and stream organization |
| libavdevice | Interaction with device frameworks for capture and playback |
| libavfilter | Frame-level filtering and processing |
| libswscale | Image scaling, pixel format conversion, and color space conversion |
| libswresample | Audio resampling, channel rematrixing, and sample format conversion |
Audio fundamentals should be clarified first.
Channels describe the spatial layout of sound, such as left, right, or 5.1 surround. Sample rate indicates how many discrete audio samples are captured per second. In general, a higher sample rate preserves more detail.
Resampling means generating audio samples again at a new sample rate. Rematrixing means reorganizing audio data into a new channel layout, such as converting stereo to mono or expanding two channels into multiple channels.
ffmpeg -i input.wav -ar 48000 -ac 2 output.wav
# -ar sets the output sample rate
# -ac sets the number of output channelsThis command shows the most common way to rebuild audio specifications: control the sample rate and channel layout.
FFmpeg’s internal pipeline progresses step by step from demuxing to muxing.
When you run -i input.mp4, FFmpeg first creates a demuxer. The demuxer separates the elementary streams inside the container, such as video, audio, and subtitles, and then passes packet-level data to the downstream processing modules.
AI Visual Insight: This diagram provides an overview of FFmpeg’s pipeline starting from the input container. It highlights the mechanism in which one input file first passes through a demuxer and is then distributed into multiple independent elementary streams. It serves as the entry point for understanding the relationship between later decoding, filtering, and muxing stages.
Next, the decoder reconstructs compressed packets into raw frames. For video, these are pixel frames. For audio, they are typically raw sample frames such as PCM.
AI Visual Insight: This diagram emphasizes that packet data output by the demuxer is expanded by the decoder into editable raw frame data. That is why all filter processing must operate on decoded frames rather than on the compressed bitstream itself.
Filters fall into two categories: simple filters and complex filters.
Simple filtering uses -filter or stream-specific filter options. It usually follows a linear path: one input stream enters a filter chain and then flows to one encoder.
AI Visual Insight: This diagram shows a linear filter chain with a single input and a single output. It is well suited to sequential processing within one stream, such as scaling, cropping, and color adjustment. The structure is simple and the dependencies are easy to follow.
Complex filtering uses -filter_complex. It can accept zero or more input streams and produce multiple output streams. It is suitable for picture-in-picture, concatenation, multi-stream mixing, split-screen layouts, and joint audio-video processing.
AI Visual Insight: This diagram illustrates the multi-input, multi-output structure of a complex filtergraph. It shows how the filtergraph can branch, merge, and assign labels internally, which is exactly what makes advanced operations like overlay, hstack, and split possible.
ffmpeg -i A.mp4 -i B.mp4 -filter_complex "[0:v][1:v]hstack[outv]" -map "[outv]" output.mp4
# Stack two input videos horizontally
# [outv] is the output label of the complex filtergraphThis command demonstrates the most important capability of a complex filtergraph: explicitly connecting multiple inputs and exporting a named stream.
Stream copy is ideal when you do not need to modify the bitstream.
If you do not need to decode, filter, or re-encode, FFmpeg can copy the original compressed stream directly into a new container. This is what -c copy does. It is fast, lossless, and extremely light on CPU usage.
AI Visual Insight: This diagram shows that the stream copy path bypasses the decoder, filter, and encoder. The compressed stream goes directly from the demuxed output into the muxer. That is why it performs so well, but it also means you give up filtering and re-encoding capabilities.
ffmpeg -i INPUT.mkv -map 0:1 -c copy OUTPUT.mp4
# Select stream 1 from input 0
# Copy directly into the output container without decoding or transcodingCommands like this are commonly used to extract audio tracks, preserve original visual quality while changing the container, or quickly merge compatible streams.
Here is a multi-input remuxing example:
ffmpeg -i INPUT0.mkv -i INPUT1.aac -map 0:0 -map 1:0 -c copy OUTPUT.mp4
# Take stream 0 from the mkv input
# Take stream 0 from the aac input
# Mux them directly into a single mp4 fileThis command shows a typical use case for selecting streams across multiple inputs and remuxing them directly.
AI Visual Insight: This diagram visualizes compressed streams from different input sources being merged back into the same output container. It shows that stream copy can do more than extraction: it can also reorganize streams across files without re-encoding.
Transcoding is only worth using when you need to change content or compatibility.
Transcoding means decode first, then encode again. It is appropriate when the container is incompatible, when you need to reduce bitrate, insert filters, change resolution, modify frame rate, or resample audio.
ffmpeg -i input.avi -b:v 64k -bufsize 64k output.mp4
# Set the target video bitrate and buffer size
# Produce a newly encoded mp4 outputThis command controls output size and bitrate budget through re-encoding.
ffmpeg -i INPUT.mkv -map 0:v -map 0:a -c:v libx264 -c:a copy OUTPUT.mp4
# Encode the video stream with libx264
# Copy the audio stream directly to avoid unnecessary transcodingThis is a very common production strategy: transcode the video while copying the audio.
AI Visual Insight: This diagram shows the video stream passing through the full decoder and encoder chain, while the audio stream can bypass encoding depending on the chosen strategy. It demonstrates that FFmpeg lets you decide, at stream granularity, whether to transcode or copy.
The -map option is the key to deterministic output.
-map explicitly specifies which stream from which input goes into which output. Without -map, FFmpeg applies automatic stream selection rules, but in complex scenarios the result is often unpredictable.
For example, -map 1:a selects all audio streams from the second input file, while -map 1:a:0 selects the first audio stream from the second input file.
ffmpeg -i A.avi -i B.mp4 -map 1:a:0 -c:a copy out.mov
# Precisely select the first audio stream from the second input
# Copy it into the output fileThis is the most direct way to avoid selecting the wrong audio track.
Timing options determine trimming and synchronization precision.
Common options include -ss for the start position, -t for processing duration, -to for the end position, -sseof for reverse seeking from the end of the file, and -itsoffset for shifting the timestamps of an entire input.
ffmpeg -ss 00:00:10 -i input.mp4 -t 5 -c copy clip.mp4
# Start at 10 seconds
# Extract a 5-second clipThis command is useful for fast, lossless clipping of key segments, although the exact cut point depends on keyframe placement.
Some global and input/output options appear constantly in real-world workflows.
-f forces a format declaration, -y overwrites output files, -n prevents overwriting, -stream_loop controls looped input, -metadata key=value writes metadata, and -timestamp sets the recorded container time.
When you work with multiple inputs that use different time bases, -isync and -itsscale also matter. The former attempts to align timestamps automatically based on a reference input. The latter is used to correct abnormal time bases, but it is more common in low-level debugging and compatibility scenarios.
FAQ
Q1: When should I use -c copy, and when is transcoding required?
A: Use -c copy when the target container supports the original codec and you do not need filters, scaling, frame-rate changes, or bitrate changes. As soon as you need to modify content or codec compatibility, transcoding becomes necessary.
Q2: Why is the cut position not accurate enough after I add -ss?
A: When used as an input option, FFmpeg often seeks to the nearest keyframe before the target timestamp. If you need higher precision, FFmpeg usually has to decode during processing, which reduces speed.
Q3: Do I always need to write -map?
A: In simple single-input cases, you can often omit it. But for multi-input files, multiple audio tracks, subtitles, or complex filtergraphs, you should explicitly specify -map; otherwise, automatic stream selection may not match your expectation.
Core summary: This article refactors raw study notes into a fact-dense FFmpeg technical guide. It systematically explains the responsibilities of FFmpeg’s core libraries, command-line syntax, the demux/decode/filter/encode/mux pipeline, the difference between stream copy and transcoding, and practical usage of key options such as -map, -ss, and -itsoffset with executable examples.