Android APIs for media playback: MediaPlayer and MediaCodec.
mediaPlayer.setDataSource(path); //fd or url mediaPlayer.setDisplay(SurfaceHolder sh); //SurfaceView or VideoView mediaPlayer.prepare(); // MediaPlayer.start(); //
/* init use MediaExtractor to get mime data create decoder by mime type configure decoder by video format and surface view */ MediaExtractor mExtractor; MediaCodec mDecoder; mExtractor = new MediaExtractor(); mExtractor.setDataSource(filePath); MediaFormat format = mExtractor.getTrackFormat(track_index); String mime = format.getString(MediaFormat.KEY_MIME); if mime.startsWith("video/") mExtractor.selectTrack(track_index); mDecoder = MediaCodec.createDecoderByType(mime); mDecoder.configure(format, surface, null, 0 /* Decoder */); mDecoder.start(); /*run start decode video, fill / empty buffer */
Simple Video Decode/Encode Flow
- The Chromium
History of Android Media Framework
- OpenCore (Android 1.0)
- OpenCore and Stagefright (Android 2.1, Eclair)
Stagefright and AwesomePlayer (Android 4.0, Ice Cream Sandwich)
Stagefright and NuPlayer (Android 5.1, Lollipop)
What are StagefrightPlayer, AwesomePlayer, NuPlayer, OpenCore, OpenMAX, and Exoplayer?
OpenCore is a media framework witch is replaced by Stagefright in Android 2.x. Stagefright relies only on OpenMAX interface for all the codecs.
StagefrightPlayer is an implement of AwesomePlayer, it call AwesomePlayer’s API. StagefrightPlayer Constructor
NuPlayer and AwesomePlayer both are a player. At first NuPlayer is designed for stream playback, but it replaces AwesomePlayer in Android 5.1 (Lollipop).
ExoPlayer is an open source, application level media player built on top of Android’s low level media APIs, support Android 4.1 above(API level 16).
ExoPlayer Developer Guide
Introduction to Android ExoPlayer
Dive into Android 6 Media Framework
Current Media Framework
- Application Framework
At the application framework level is the app’s code, which utilizes the android.media APIs to interact with the multimedia hardware.
- Binder IPC
The Binder IPC proxies facilitate communication over process boundaries. They are located in the frameworks/av/media/libmedia directory and begin with the letter “I”.
- Native Multimedia Framework
At the native level, Android provides a multimedia framework that utilizes the Stagefright engine for audio and video recording and playback. Stagefright comes with a default list of supported software codecs and you can implement your own hardware codec by using the OpenMax integration layer standard. For more implementation details, see the various MediaPlayer and Stagefright components located in frameworks/av/media.
- OpenMAX Integration Layer (IL)
The OpenMAX IL provides a standardized way for Stagefright to recognize and use custom hardware-based multimedia codecs called components. You must provide an OpenMAX plugin in the form of a shared library named libstagefrighthw.so. This plugin links your custom codec components to Stagefright. Your custom codecs must be implemented according to the OpenMAX IL component standard.
AwesomePlayer is deprecated, but source code still in the path frameworks/av/media/libmediaplayerservice/StagefrightPlayer.cpp
Media Playback Architecture
MediaPlayerFactory: For different format media files, there are different scoring rule for choosing player. Each factory must has its own score function implement.
Media Playback Use MediaPlayer
Media Playback Use MediaExtractor