Demuxing

Demuxers read a media file and split it into chunks of data (packets). A packet contains one or more encoded frames which belongs to a single elementary stream. In the lavf API this process is represented by the avformat_open_input() function for opening a file, av_read_frame() for reading a single packet and finally avformat_close_input(), which does the cleanup. More...

Data Structures
struct	AVInputFormat

Functions
const AVInputFormat *	av_find_input_format (const char *short_name)
	Find AVInputFormat based on the short name of the input format. More...
const AVInputFormat *	av_probe_input_format (const AVProbeData *pd, int is_opened)
	Guess the file format. More...
const AVInputFormat *	av_probe_input_format2 (const AVProbeData *pd, int is_opened, int *score_max)
	Guess the file format. More...
const AVInputFormat *	av_probe_input_format3 (const AVProbeData *pd, int is_opened, int *score_ret)
	Guess the file format. More...
int	av_probe_input_buffer2 (AVIOContext *pb, const AVInputFormat **fmt, const char *url, void *logctx, unsigned int offset, unsigned int max_probe_size)
	Probe a bytestream to determine the input format. More...
int	av_probe_input_buffer (AVIOContext *pb, const AVInputFormat **fmt, const char *url, void *logctx, unsigned int offset, unsigned int max_probe_size)
	Like av_probe_input_buffer2() but returns 0 on success. More...
int	avformat_open_input (AVFormatContext **ps, const char *url, const AVInputFormat *fmt, AVDictionary **options)
	Open an input stream and read the header. More...
int	avformat_find_stream_info (AVFormatContext *ic, AVDictionary **options)
	Read packets of a media file to get stream information. More...
AVProgram *	av_find_program_from_stream (AVFormatContext *ic, AVProgram *last, int s)
	Find the programs which belong to a given stream. More...
void	av_program_add_stream_index (AVFormatContext *ac, int progid, unsigned int idx)
int	av_find_best_stream (AVFormatContext *ic, enum AVMediaType type, int wanted_stream_nb, int related_stream, const AVCodec **decoder_ret, int flags)
	Find the "best" stream in the file. More...
int	av_read_frame (AVFormatContext *s, AVPacket *pkt)
	Return the next frame of a stream. More...
int	av_seek_frame (AVFormatContext *s, int stream_index, int64_t timestamp, int flags)
	Seek to the keyframe at timestamp. More...
int	avformat_seek_file (AVFormatContext *s, int stream_index, int64_t min_ts, int64_t ts, int64_t max_ts, int flags)
	Seek to timestamp ts. More...
int	avformat_flush (AVFormatContext *s)
	Discard all internally buffered data. More...
int	av_read_play (AVFormatContext *s)
	Start playing a network-based stream (e.g. More...
int	av_read_pause (AVFormatContext *s)
	Pause a network-based stream (e.g. More...
void	avformat_close_input (AVFormatContext **s)
	Close an opened input AVFormatContext. More...

Detailed Description

Demuxers read a media file and split it into chunks of data (packets). A packet contains one or more encoded frames which belongs to a single elementary stream. In the lavf API this process is represented by the avformat_open_input() function for opening a file, av_read_frame() for reading a single packet and finally avformat_close_input(), which does the cleanup.

Opening a media file

The minimum information required to open a file is its URL, which is passed to avformat_open_input(), as in the following code:

const char    *url = "file:in.mp3";
AVFormatContext *s = NULL;
int ret = avformat_open_input(&s, url, NULL, NULL);
if (ret < 0)
    abort();

The above code attempts to allocate an AVFormatContext, open the specified file (autodetecting the format) and read the header, exporting the information stored there into s. Some formats do not have a header or do not store enough information there, so it is recommended that you call the avformat_find_stream_info() function which tries to read and decode a few frames to find missing information.

In some cases you might want to preallocate an AVFormatContext yourself with avformat_alloc_context() and do some tweaking on it before passing it to avformat_open_input(). One such case is when you want to use custom functions for reading input data instead of lavf internal I/O layer. To do that, create your own AVIOContext with avio_alloc_context(), passing your reading callbacks to it. Then set the pb field of your AVFormatContext to newly created AVIOContext.

Since the format of the opened file is in general not known until after avformat_open_input() has returned, it is not possible to set demuxer private options on a preallocated context. Instead, the options should be passed to avformat_open_input() wrapped in an AVDictionary:

AVDictionary *options = NULL;
av_dict_set(&options, "video_size", "640x480", 0);
av_dict_set(&options, "pixel_format", "rgb24", 0);
 
if (avformat_open_input(&s, url, NULL, &options) < 0)
    abort();
av_dict_free(&options);

This code passes the private options 'video_size' and 'pixel_format' to the demuxer. They would be necessary for e.g. the rawvideo demuxer, since it cannot know how to interpret raw video data otherwise. If the format turns out to be something different than raw video, those options will not be recognized by the demuxer and therefore will not be applied. Such unrecognized options are then returned in the options dictionary (recognized options are consumed). The calling program can handle such unrecognized options as it wishes, e.g.

AVDictionaryEntry *e;
if (e = av_dict_get(options, "", NULL, AV_DICT_IGNORE_SUFFIX)) {
    fprintf(stderr, "Option %s not recognized by the demuxer.\n", e->key);
    abort();
}

After you have finished reading the file, you must close it with avformat_close_input(). It will free everything associated with the file.

Reading from an opened file

Reading data from an opened AVFormatContext is done by repeatedly calling av_read_frame() on it. Each call, if successful, will return an AVPacket containing encoded data for one AVStream, identified by AVPacket.stream_index. This packet may be passed straight into the libavcodec decoding functions avcodec_send_packet() or avcodec_decode_subtitle2() if the caller wishes to decode the data.

AVPacket.pts, AVPacket.dts and AVPacket.duration timing information will be set if known. They may also be unset (i.e. AV_NOPTS_VALUE for pts/dts, 0 for duration) if the stream does not provide them. The timing information will be in AVStream.time_base units, i.e. it has to be multiplied by the timebase to convert them to seconds.

A packet returned by av_read_frame() is always reference-counted, i.e. AVPacket.buf is set and the user may keep it indefinitely. The packet must be freed with av_packet_unref() when it is no longer needed.

Seeking

Function Documentation

av_find_input_format()

const AVInputFormat* av_find_input_format

(

const char *

short_name

)

Find AVInputFormat based on the short name of the input format.

Definition at line 120 of file format.c.

Referenced by ff_alloc_input_device_context(), ff_load_image(), ff_wms_parse_sdp_a_line(), movie_common_init(), open_input_file(), opt_format(), sap_read_header(), and show_help_demuxer().

av_probe_input_format()

const AVInputFormat* av_probe_input_format	(	const AVProbeData *	pd,
		int	is_opened
	)

Guess the file format.

Parameters

pd	data to be probed
is_opened	Whether the file is already opened; determines whether demuxers with or without AVFMT_NOFILE are probed.

Definition at line 221 of file format.c.

av_probe_input_format2()

const AVInputFormat* av_probe_input_format2	(	const AVProbeData *	pd,
		int	is_opened,
		int *	score_max
	)

Guess the file format.

Parameters

pd	data to be probed
is_opened	Whether the file is already opened; determines whether demuxers with or without AVFMT_NOFILE are probed.
score_max	A probe score larger that this is required to accept a detection, the variable is set to the actual detection score afterwards. If the score is <= AVPROBE_SCORE_MAX / 4 it is recommended to retry with a larger probe buffer.

Definition at line 209 of file format.c.

Referenced by av_probe_input_buffer2(), av_probe_input_format(), init_input(), and read_gab2_sub().

av_probe_input_format3()

const AVInputFormat* av_probe_input_format3	(	const AVProbeData *	pd,
		int	is_opened,
		int *	score_ret
	)

Guess the file format.

Parameters

is_opened	Whether the file is already opened; determines whether demuxers with or without AVFMT_NOFILE are probed.
score_ret	The score of the best detection.

Definition at line 130 of file format.c.

Referenced by av_probe_input_format2(), ff_img_read_packet(), and set_codec_from_probe_data().

av_probe_input_buffer2()

int av_probe_input_buffer2	(	AVIOContext *	pb,
		const AVInputFormat **	fmt,
		const char *	url,
		void *	logctx,
		unsigned int	offset,
		unsigned int	max_probe_size
	)

Probe a bytestream to determine the input format.

Each time a probe returns with a score that is too low, the probe buffer size is increased and another attempt is made. When the maximum probe size is reached, the input format with the highest score is returned.

Parameters

pb	the bytestream to probe
fmt	the input format is put here
url	the url of the stream
logctx	the log context
offset	the offset within the bytestream to probe from
max_probe_size	the maximum probe buffer size (zero for default)

Returns

the score in case of success, a negative value corresponding to an the maximal score is AVPROBE_SCORE_MAX AVERROR code otherwise

Definition at line 227 of file format.c.

Referenced by av_probe_input_buffer(), and init_input().

av_probe_input_buffer()

int av_probe_input_buffer	(	AVIOContext *	pb,
		const AVInputFormat **	fmt,
		const char *	url,
		void *	logctx,
		unsigned int	offset,
		unsigned int	max_probe_size
	)

Like av_probe_input_buffer2() but returns 0 on success.

Definition at line 317 of file format.c.

Referenced by hls_read_header(), and reopen_demux_for_component().

avformat_open_input()

int avformat_open_input	(	AVFormatContext **	ps,
		const char *	url,
		const AVInputFormat *	fmt,
		AVDictionary **	options
	)

Open an input stream and read the header.

The codecs are not opened. The stream must be closed with avformat_close_input().

Parameters

ps	Pointer to user-supplied AVFormatContext (allocated by avformat_alloc_context). May be a pointer to NULL, in which case an AVFormatContext is allocated by this function and written into ps. Note that a user-supplied AVFormatContext will be freed on failure.
url	URL of the stream to open.
fmt	If non-NULL, this parameter forces a specific input format. Otherwise the format is autodetected.
options	A dictionary filled with AVFormatContext and demuxer-private options. On return this parameter will be destroyed and replaced with a dict containing options that were not found. May be NULL.

Returns

0 on success, a negative AVERROR on failure.

Note

If you want to use custom IO, preallocate the format context and set its pb field.

Examples

avio_reading.c, demuxing_decoding.c, filtering_audio.c, filtering_video.c, hw_decode.c, metadata.c, qsvdec.c, remuxing.c, transcode_aac.c, transcoding.c, and vaapi_transcode.c.

Definition at line 220 of file demux.c.

Referenced by ds_open(), ff_load_image(), ff_wms_parse_sdp_a_line(), handle_file(), hls_read_header(), LLVMFuzzerTestOneInput(), main(), movie_common_init(), open_file(), open_input_file(), open_track_resource_context(), rdt_init(), read_gab2_sub(), read_thread(), reopen_demux_for_component(), sap_read_header(), seek_test(), video_decode(), and video_decode_example().

avformat_find_stream_info()

int avformat_find_stream_info	(	AVFormatContext *	ic,
		AVDictionary **	options
	)

Read packets of a media file to get stream information.

This is useful for file formats with no headers such as MPEG. This function also computes the real framerate in case of MPEG-2 repeat frame mode. The logical file position is not changed by this function; examined packets may be buffered for later processing.

Parameters

ic	media file handle
options	If non-NULL, an ic.nb_streams long array of pointers to dictionaries, where i-th member contains options for codec corresponding to i-th stream. On return each dictionary will be filled with options that were not found.

Returns

>=0 if OK, AVERROR_xxx on error

Note

this function isn't guaranteed to open all the codecs, so options being non-empty at return is a perfectly normal behavior.

Todo:

Let the user decide somehow what information is needed so that we do not waste time getting stuff the user does not need.

Examples

avio_reading.c, demuxing_decoding.c, filtering_audio.c, filtering_video.c, hw_decode.c, metadata.c, remuxing.c, transcode_aac.c, transcoding.c, and vaapi_transcode.c.

Definition at line 2415 of file demux.c.

Referenced by ff_load_image(), handle_file(), hls_read_header(), LLVMFuzzerTestOneInput(), main(), movie_common_init(), open_file(), open_input_file(), read_thread(), reopen_demux_for_component(), seek_test(), video_decode(), and video_decode_example().

av_find_program_from_stream()

AVProgram* av_find_program_from_stream	(	AVFormatContext *	ic,
		AVProgram *	last,
		int	s
	)

Find the programs which belong to a given stream.

Parameters

ic	media file handle
last	the last found program, the search will start after this program, or from the beginning if it is NULL
s	stream index

Returns

the next program which belongs to s, NULL if no program is found or the last program is not among the programs of ic.

Definition at line 296 of file avformat.c.

Referenced by av_find_best_stream(), mpegts_push_data(), scte_data_cb(), stream_cycle_channel(), update_stream_timings(), and update_wrap_reference().

av_program_add_stream_index()

void av_program_add_stream_index	(	AVFormatContext *	ac,
		int	progid,
		unsigned int	idx
	)

av_find_best_stream()

int av_find_best_stream	(	AVFormatContext *	ic,
		enum AVMediaType	type,
		int	wanted_stream_nb,
		int	related_stream,
		const AVCodec **	decoder_ret,
		int	flags
	)

Find the "best" stream in the file.

The best stream is determined according to various heuristics as the most likely to be what the user expects. If the decoder parameter is non-NULL, av_find_best_stream will find the default decoder for the stream's codec; streams for which no decoder can be found are ignored.

Parameters

ic	media file handle
type	stream type: video, audio, subtitles, etc.
wanted_stream_nb	user-requested stream number, or -1 for automatic selection
related_stream	try to find a stream related (eg. in the same program) to this one, or -1 if none
decoder_ret	if non-NULL, returns the decoder for the selected stream
flags	flags; none are currently defined

Returns

the non-negative stream number in case of success, AVERROR_STREAM_NOT_FOUND if no stream with the requested type could be found, AVERROR_DECODER_NOT_FOUND if streams were found but no decoder

Note

If av_find_best_stream returns successfully and decoder_ret is not NULL, then *decoder_ret is guaranteed to be set to a valid AVCodec.

Definition at line 347 of file avformat.c.

Referenced by find_stream(), main(), open_codec_context(), open_input_file(), read_thread(), seek_test(), video_decode(), and video_decode_example().

av_read_frame()

int av_read_frame	(	AVFormatContext *	s,
		AVPacket *	pkt
	)

Return the next frame of a stream.

This function returns what is stored in the file, and does not validate that what is there are valid frames for the decoder. It will split what is stored in the file into frames and return one for each call. It will not omit invalid data between valid frames so as to give the decoder the maximum information possible for decoding.

On success, the returned packet is reference-counted (pkt->buf is set) and valid indefinitely. The packet must be freed with av_packet_unref() when it is no longer needed. For video, the packet contains exactly one frame. For audio, it contains an integer number of frames if each frame has a known fixed size (e.g. PCM or ADPCM data). If the audio frames have a variable size (e.g. MPEG audio), then it contains one frame.

pkt->pts, pkt->dts and pkt->duration are always set to correct values in AVStream.time_base units (and guessed if the format cannot provide them). pkt->pts can be AV_NOPTS_VALUE if the video format has B-frames, so it is better to rely on pkt->dts if you do not decompress the payload.

Returns

0 if OK, < 0 on error or end of file. On error, pkt will be blank (as if it came from av_packet_alloc()).

Note

pkt will be initialized, so it may be uninitialized, but it must not contain data that needs to be freed.

Examples

demuxing_decoding.c, filtering_audio.c, filtering_video.c, hw_decode.c, qsvdec.c, remuxing.c, transcode_aac.c, transcoding.c, and vaapi_transcode.c.

Definition at line 1438 of file demux.c.

Referenced by asf_read_pts(), compute_crc_of_packets(), concat_read_packet(), dash_read_packet(), decode_audio_frame(), ds_run(), ff_load_image(), get_input_packet(), hls_read_packet(), imf_read_packet(), LLVMFuzzerTestOneInput(), main(), movie_decode_packet(), mpc_read_seek(), mpegts_get_dts(), read_interval_packets(), read_thread(), sap_fetch_packet(), seek_frame_generic(), video_decode(), and video_decode_example().

av_seek_frame()

int av_seek_frame	(	AVFormatContext *	s,
		int	stream_index,
		int64_t	timestamp,
		int	flags
	)

Seek to the keyframe at timestamp.

'timestamp' in 'stream_index'.

Parameters

s	media file handle
stream_index	If stream_index is (-1), a default stream is selected, and timestamp is automatically converted from AV_TIME_BASE units to the stream specific time_base.
timestamp	Timestamp in AVStream.time_base units or, if no stream is specified, in AV_TIME_BASE units.
flags	flags which select direction and seeking mode

Returns

>= 0 on success

Definition at line 634 of file seek.c.

Referenced by avformat_seek_file(), compute_crc_of_packets(), dash_seek(), movie_common_init(), process_command(), and rewind_file().

avformat_seek_file()

int avformat_seek_file	(	AVFormatContext *	s,
		int	stream_index,
		int64_t	min_ts,
		int64_t	ts,
		int64_t	max_ts,
		int	flags
	)

Seek to timestamp ts.

Seeking will be done so that the point from which all active streams can be presented successfully will be closest to ts and within min/max_ts. Active streams are all streams that have AVStream.discard < AVDISCARD_ALL.

If flags contain AVSEEK_FLAG_BYTE, then all timestamps are in bytes and are the file position (this may not be supported by all demuxers). If flags contain AVSEEK_FLAG_FRAME, then all timestamps are in frames in the stream with stream_index (this may not be supported by all demuxers). Otherwise all timestamps are in units of the stream selected by stream_index or if stream_index is -1, in AV_TIME_BASE units. If flags contain AVSEEK_FLAG_ANY, then non-keyframes are treated as keyframes (this may not be supported by all demuxers). If flags contain AVSEEK_FLAG_BACKWARD, it is ignored.

Parameters

s	media file handle
stream_index	index of the stream which is used as time base reference
min_ts	smallest acceptable timestamp
ts	target timestamp
max_ts	largest acceptable timestamp
flags	flags

Returns

>=0 on success, error code otherwise

Note

This is part of the new seek API which is still under construction.

Definition at line 657 of file seek.c.

Referenced by av_seek_frame(), main(), open_file(), open_input_file(), open_track_resource_context(), read_interval_packets(), read_thread(), seek_subtitle(), seek_to_start(), and try_seek().

avformat_flush()

int avformat_flush

(

AVFormatContext *

s

)

Discard all internally buffered data.

This can be useful when dealing with discontinuities in the byte stream. Generally works only with formats that can resync. This includes headerless formats like MPEG-TS/TS but should also work with NUT, Ogg and in a limited way AVI for example.

The set of streams, the detected duration, stream parameters and codecs do not change when calling this function. If you want a complete reset, it's better to open a new AVFormatContext.

This does not flush the AVIOContext (s->pb). If necessary, call avio_flush(s->pb) before calling this function.

Parameters

s	media file handle

Returns

>=0 on success, error code otherwise

Definition at line 750 of file seek.c.

av_read_play()

int av_read_play

(

AVFormatContext *

s

)

Start playing a network-based stream (e.g.

RTSP stream) at the current position.

Definition at line 202 of file demux_utils.c.

Referenced by read_thread().

av_read_pause()

int av_read_pause

(

AVFormatContext *

s

)

Pause a network-based stream (e.g.

RTSP stream).

Use av_read_play() to resume it.

Definition at line 211 of file demux_utils.c.

Referenced by read_thread().

avformat_close_input()

void avformat_close_input

(

AVFormatContext **

s

)

Close an opened input AVFormatContext.

Free it and all its contents and set *s to NULL.

Examples

avio_reading.c, demuxing_decoding.c, filtering_audio.c, filtering_video.c, hw_decode.c, metadata.c, qsvdec.c, remuxing.c, transcode_aac.c, transcoding.c, and vaapi_transcode.c.

Definition at line 368 of file demux.c.

Referenced by avi_read_close(), close_demux_for_component(), close_input_file(), concat_read_close(), concat_seek(), ds_free(), ff_load_image(), ff_rtsp_close_streams(), ff_wms_parse_sdp_a_line(), ffmpeg_cleanup(), free_playlist_list(), free_representation(), get_resource_context_for_timestamp(), handle_file(), imf_seek(), imf_virtual_track_playback_context_deinit(), LLVMFuzzerTestOneInput(), main(), movie_uninit(), open_file(), open_input_file(), open_track_resource_context(), rdt_close_context(), read_thread(), sap_read_close(), seek_test(), stream_close(), video_decode(), and video_decode_example().