Some Android Projects might require high or low level media processing. This article rounds up a few useful libraries, like FFmpeg, MP4Parser, Intel Media for Mobile, etc. All the libraries have both pros and cons. You need to carefully consider, according to your initial target API-s, devices, and specification, which one you’ll use . I won’t go into too much detail, the article’s main purpose is to help you decide which lib fits best for a given issue. The selected library or libraries can have great effect on the size of your final application, and will also affect the code complexity and amounts of future maintenance.
1. FFmpeg (FFmpeg on GitHub)
As far as I know, this library is used most often for video processing. There are tons of articles, forum threads and SO posts about this and that’s why most developers use it. It can take care of almost any kind of video processing tasks and it can be compiled on almost any platform. And here comes the painful part: you need to compile it for your project. It’s a modular library, it can be extended or shrank, according to your needs. The usage of the library depends on the output of the compilation. On one hand, you can make a standard unix executable shell program. It can then be loaded from the device memory and executed with the Java process. In this case you have to pass all the needed command line arguments for the process. On the other hand, you can compile it as a Shared Library. Then it can be used in a wrapper class, using the Java JNI and the Android NDK. In this case you don’t have to generate command line arguments. But to handle all the input and output data, you will have to write the code for the wrapper class in C or C++.
Pros of FFmpeg:
-Has good codec support (depends on compiler)
-It’s capable of almost all of the video processing tasks
-Has tons of useful articles and sample codes online
Cons of FFmpeg:
-If you need most of the functions from the library, the output of the compilation can be as large as 20~30Mb for each platform: huge overhead in the APK
This library is good for almost any kind of video processing task (concatenate, cut, loop, mux, demux) and it can even apply effects such as fading, distortion, zoom, Picture-in-picture, etc. You have to keep a good balance between included features and compiled binary size or else you will end up with large APKs.
Fun with FFmpeg:
This is a very compact library with decent encoding and decoding capabilities, as it was developed and is maintained by Intel. It’s sufficient out of the box, but the samples only provide primitive use cases. Plus, you might have to write complex scripts for the OpenGL platform if you want to use the lib for complex video processing. Its API is similar to the built in Android MediaPlayer and the MediaCodec API, however, the Intel INDE can be used from API level 16.
Pros of Intel INDE:
-has good codec support
-smaller than FFmpeg
-it’s capable of most of the primitive video processing tasks (encode, decode, overlay, concatenate)
Cons of Intel INDE:
-For complex tasks you have to develop shader code
-Complex sintax for complex tasks
-It’s hard to find examples on forums
-The Github samples are really good, but few
-It doesn’t transcode H264 MP4
-Only works on x86 and ARM
Concatenating, cutting and transcoding media. It can also be used for capturing and streaming videos and you can apply effects using OpenGL.
It’s a small library for managing basic video processing tasks on similar media files. It’s not capable of transcoding files, though it’s fast, because it doesn’t encode. Basically, it will only separate the data streams in the video container, then you can do your modifications on these streams.
Pros of MP4Parser:
-can be used for concatenation, but the input files must be in same format (Resolution, bitrate…)
-can be used for slicing the input.
-Audio and video merge can be achieved, but you need audio format compatible with the video container.
-Good API, easy to use and can produce relatively clean code.
-Has some info on SO, and there are a few samples
-The codebase is small
Cons of MP4Parser:
-Doesn’t know the MP3 audio format
-If you want to use differently formatted input data, you need something else to transcode them.
Cutting and concatenating media, recorded with the camera of the device with fixed recording configurations for compatibility. The MP4Parser can be paired with Ringdroid for user friendly audio editing.
Pros of Gstreamer:
-It’s even bigger than FFmpeg
-Uses plugins > modular
-Compatible with lots of codecs (if included)
-It’s capable of almost any video processing task!
Cons of Gstreamer:
-It has the most complicated API (out of the mentioned libraries)
-Complex tasks may need shader code
-The library is large, will produce a large APK
-Have to compile with the NDK
Same as FFmpeg, but more complex. In my opinion this library is far too complex for everyday applications, most of the issues that usually appear during development can be covered with other libraries mentioned in this article.
This is the built in Android API for processing video. However, it has some serious limitations if you want to target a wide scale of APIs. The most useful part is the MediaMuxer class, but you need at least an API level 18 to use this class. (Currently using a minimum API level 16 is recommended.) Therefore, you will lose a decent percentage of the currently working devices if you decide to use this API. Its usage is similar to the MediaPlayer API, but modifications are possible using a Surface in your app and then can you modify its content through the openGL ES API.
Manipulating media files which are compatible with the device by default. You can apply effects using OpenGL. If you pair it with the MP4Parser library you will get a powerful tool for media processing, but you’ll have to ensure the media compatibility using this library in case you want to manage the output with the MP4Parser.
Unlike the libraries previously mentioned, this one only processes audio input, video files can also be used, but only the audio track will be imported. It is compatible with the most common audio formats like AAC, MP3, WAV, OGG. The library also includes some CustomViews for the editor user interface.
Pros of Ringdroid:
-It’s stable and well maintained
-Knows common formats and able to encode and decode
-Capable of editing the input
-Apache 2.0 License
Cons of Ringdroid:
-The import and export part usually reduces the audio quality
-Audio processing only!
Using its built in CustomViews, you can create userfriendly UI for audio editing and it can be paired with other media processing libraries.
All in all:
As you perhaps expect, I am not going to tell you which library is best. There is no fix-all kind of solution: you should always take your specification into account, combined with this article I hope I could ease your choice.
Latest posts by Tamás Agócs (see all)
- ConstraintLayout 1.0 - June 20, 2017
- RxMp4Parser for Android App Developers - September 12, 2016
- Rx Wrapper How-To for Critical Resource Tasks - August 10, 2016
- Android Developers! Here are 5 of the Best Video Transcoding Libraries +1 for Only Audio! - July 13, 2016
- ConstraintLayout on Android – How Ready Is It? - May 31, 2016
- How to Render More Than a Lot of Markers on Google Maps for Android - May 5, 2016