Android APIs for media playback: MediaPlayer and MediaCodec.

  • MediaPlayer
mediaPlayer.setDataSource(path);    //fd or url
mediaPlayer.setDisplay(SurfaceHolder sh);  //SurfaceView or VideoView  
mediaPlayer.prepare(); // 
MediaPlayer.start(); //
  • MediaCodec

/* init  
use MediaExtractor to get mime data  
create decoder by mime type 
configure decoder by video format and surface view  
*/ 
MediaExtractor mExtractor;
MediaCodec mDecoder;
mExtractor = new MediaExtractor();
mExtractor.setDataSource(filePath);
MediaFormat format = mExtractor.getTrackFormat(track_index);  
String mime = format.getString(MediaFormat.KEY_MIME);
if mime.startsWith("video/")  
    mExtractor.selectTrack(track_index);
    mDecoder = MediaCodec.createDecoderByType(mime);
    mDecoder.configure(format, surface, null, 0 /* Decoder */);  
    mDecoder.start();


/*run
start decode video, fill / empty buffer  
*/

MediaCodecExample

Simple Video Decode/Encode Flow

  • FFmpeg


Demultiplexer
Mux? Demux? Remux? Huh?

  • GStreamer

  • The Chromium

Reference:
A Weekend with GStreamer
The Chromium Projects
Handy FFMPEG commands for all video processing needs

History of Android Media Framework

  • OpenCore (Android 1.0)

  • OpenCore and Stagefright (Android 2.1, Eclair)


stagefright框架（一）Video Playback的流程

  • Stagefright and AwesomePlayer (Android 4.0, Ice Cream Sandwich)

  • Stagefright and NuPlayer (Android 5.1, Lollipop)


Android MediaPlayerService Architecture

What are StagefrightPlayer, AwesomePlayer, NuPlayer, OpenCore, OpenMAX, and Exoplayer?

OpenCore is a media framework witch is replaced by Stagefright in Android 2.x. Stagefright relies only on OpenMAX interface for all the codecs.

StagefrightPlayer is an implement of AwesomePlayer, it call AwesomePlayer’s API. StagefrightPlayer Constructor

NuPlayer and AwesomePlayer both are a player. At first NuPlayer is designed for stream playback, but it replaces AwesomePlayer in Android 5.1 (Lollipop).

ExoPlayer is an open source, application level media player built on top of Android’s low level media APIs, support Android 4.1 above(API level 16).

ExoPlayer Developer Guide
Introduction to Android ExoPlayer
Interface ExoPlayer

Dive into Android 6 Media Framework

Current Media Framework

Current Media Framework

  • Application Framework
    At the application framework level is the app’s code, which utilizes the android.media APIs to interact with the multimedia hardware.
  • Binder IPC
    The Binder IPC proxies facilitate communication over process boundaries. They are located in the frameworks/av/media/libmedia directory and begin with the letter “I”.
  • Native Multimedia Framework
    At the native level, Android provides a multimedia framework that utilizes the Stagefright engine for audio and video recording and playback. Stagefright comes with a default list of supported software codecs and you can implement your own hardware codec by using the OpenMax integration layer standard. For more implementation details, see the various MediaPlayer and Stagefright components located in frameworks/av/media.
  • OpenMAX Integration Layer (IL)
    The OpenMAX IL provides a standardized way for Stagefright to recognize and use custom hardware-based multimedia codecs called components. You must provide an OpenMAX plugin in the form of a shared library named libstagefrighthw.so. This plugin links your custom codec components to Stagefright. Your custom codecs must be implemented according to the OpenMAX IL component standard.

AwesomePlayer is deprecated, but source code still in the path frameworks/av/media/libmediaplayerservice/StagefrightPlayer.cpp

Media Playback Architecture

MediaPlayerFactory: For different format media files, there are different scoring rule for choosing player. Each factory must has its own score function implement.

Android Multimedia Framework
Inside of Stagefright
Android’s Stagefright Media Player Architecture

Media Playback Use MediaPlayer

android多媒體框架之流媒體具體流程篇1—-base on jellybean

Media Playback Use MediaExtractor

Resources

Stagefright vs Gstreamer
Android多媒体开发
Handy FFMPEG commands for all video processing needs
MediaExtractor介绍
Implement a custom codec
NuPlayer for HTTP live streaming