ExoPlayer Hacks: Part 1 – Stats For Nerds

Google’s ExoPlayer is best in class media player for Android and it supports many features which are not currently supported by stock MediaPlayer.

Best part about Exo is it’s ¬†OpenSource-ness ūüôā

In this series of blog posts I’m gonna walk through a couple of useful hacks/customisations on some of Exo components:

  • Stats For Nerds – Part 1
  • Improved Buffering – Part 2

Find the demo on github which covers both features.

Stats for Nerds:

charts

With Adaptive streaming techniques(HLS, Smooth, DASH), playback is heavily relied on connection speed, meaning resolution changes to adapt to fluctuations in bandwidth which means data consumption varies with time.

As a geek I’ve wondered what’s my n/w consumption, conn speed & sometimes buffer length when I stream a video online. Youtube on Chrome answered these concerns with ‘Stats For Nerd’ mode which depicts all those stats using simple dynamic charts and I found it intriguing. So I decided to do something similar on ExoPlayer and turned out it’s easier than I thought.

Which Stats?

  • Connection Speed:
  • Network Activity
  • Buffer Health
  • Dropped Frames
  • Video & Screen Resolutions

Code Please

  • Connection speed & n/w activity can be obtained by passing BandwidthMeter.EventListener¬†while creating¬†DefaultBandwidthMeter object.
bandwidthMeter = new DefaultBandwidthMeter(mUiUpdateHandler, new BandwidthMeter.EventListener() {
    @Override
    public void onBandwidthSample(int elapsedMs, long bytes, long bitrate) {
        bitrateEstimate = bitrate;
        bytesDownloaded = bytes;
    }
});
  • ExoPlayer exposes all available Tracks and their corresponding Formats and setting a Video debug listener will give update you whenever input video format changes. Format is a container of all meta data related to a Video Rendition (width, height, bitrate etc.)
  • Dropped frames can be obtained by Video debug listener too.
player.setVideoDebugListener(new VideoRendererEventListener() {
  
    ....
    @Override
    public void onVideoInputFormatChanged(Format format) {
        currentVideoFormat = format;
    }

    @Override
    public void onDroppedFrames(int count, long elapsedMs) {
        droppedFrames += count;
    }
    ......
}

 Buffer health & LoadControl

LoadControl is an Exo Component interface to control buffering of Media and there is a DefaultLoadControl in SDK which takes of when to start buffering and for how long.

We can create our version of DefaultLoadControl with an event listener & a handler to notify player on a buffer data. Check CustomLoadControl class for more info.

player = ExoPlayerFactory.newSimpleInstance(renderersFactory, trackSelector, new CustomLoadControl(new CustomLoadControl.EventListener() {
    @Override
    public void onBufferedDurationSample(long bufferedDurationUs) {
        bufferedDurationMs = bufferedDurationUs;
    }
}, mUiUpdateHandler));
player.addListener(this);

OK. How to draw those awesome charts?

MPAndroidChart is a second to none charting library in Android and it covers all our charting requirements with no exceptions.

Create a Handler to repetitively call to itself every 500ms to update speed & buffer data and N/W activity has to be updated atleast with 1100ms so that updates wont overlap.

private void startPlayerStats() {
    mUiUpdateHandler.removeMessages(MSG_UPDATE_STATS);
    mUiUpdateHandler.removeMessages(MSG_UPDATE_STATS_NW_ONLY);
    depictPlayerStats();
    depictPlayerNWStats();
}

protected void depictPlayerStats() {
    if (!canShowStats())
        return;
    String buffer = DemoUtil.getFormattedDouble((bufferedDurationMs / Math.pow(10, 6)), 1);
    String brEstimate = DemoUtil.getFormattedDouble((bitrateEstimate / Math.pow(10, 3)), 1);
    updateStatChart(player_stats_health_chart, Float.parseFloat(buffer), ColorTemplate.getHoloBlue(), "Buffer Health: " + buffer + " s");
    updateStatChart(player_stats_speed_chart, Float.parseFloat(brEstimate), Color.LTGRAY, "Conn Speed: " + DemoUtil.humanReadableByteCount(
            bitrateEstimate, true, true) + "ps");
    player_stats_size.setText("Screen Dimensions: " + simpleExoPlayerView.getWidth() + " x " + simpleExoPlayerView.getHeight());
    player_stats_res.setText("Video Resolution: " + (null != currentVideoFormat ? (currentVideoFormat.width + " x " + currentVideoFormat.height) : "NA"));
    player_stats_dropframes.setText("Dropped Frames: " + droppedFrames);
    mUiUpdateHandler.sendEmptyMessageDelayed(MSG_UPDATE_STATS, 500);
}

protected void depictPlayerNWStats() {
    if (!canShowStats())
        return;
    updateStatChart(player_stats_nw_chart, (float) (bytesDownloaded / Math.pow(10, 3)), Color.CYAN, "Network Activity: " + DemoUtil.humanReadableByteCount(
            bytesDownloaded, true));
    bytesDownloaded = 0;
    mUiUpdateHandler.sendEmptyMessageDelayed(MSG_UPDATE_STATS_NW_ONLY, 1100);
}

 

Follow my next blog in series for Part 2 – Improved Buffering.

SHAREthem – Android library to simulate How SHAREit works

ic_launcherSHAREthem library makes File Sharing & Transfer a lot easier!!

Peer-to-Peer transfers are the best way to go when it comes to file sharing.

SHAREit App¬†takes full advantage of it.¬†I’ve seen many questions on StackOverFlow asking about how SHAREit works. (Just search shareit on SO and you’ll see a pageful of questions asking about the same thing). Well, ¬†it’s easy to assume that they use Android¬†Wi-Fi P2P libraries because android developer site says those libraries are used for P2P.

But, if you observe SHAREit app creates a Hotspot on receiver side which defeats the whole purpose of using Android P2P libraries and more over SHAREit works all the way from Android 2.2(at least it¬†used to) where as Android P2P libraries works only on and above Android 4.0. May be they are using combination of both to take leverage of platform apis. But I’m gonna talk about Hotspot way since¬†SHAREthem library is¬†built around it.

That’s enough, tell me how it works¬†ūüôĄ

google-play-badge
Also app is posted on Playstore (Still in Beta). Click on the badge to see it action.

SHAREthem library facilitates file sharing and transfers using below components. Library also supports app to web sharing if receiver has no app installed :

  • Hotspot Controller
    HC uses Java Reflection since there are NO APIs available on Android for enabling/disabling Hotspots. Functionalities include:

    • Controller creates an OPEN¬†Wifi hotspot configuration with an SSID which can intercepted by Receivers to recognize SHAREthem senders including port and sender names.
    • Restores user Hotspot-Configuration¬†when Share mode is disabled
    • Provides a list of connected WiFi clients.
  • SHAREthem¬†Server

A tiny HTTP server extended from NanoHttpd, serves the sender data to receivers using IP address as hostname and works on port assigned by user or system by default.

  • SHAREthem Service

Android service which manages lifecycle of SHAREthem-server and also handles foreground notification with stop action.

  • UI (Activities)
    Android activities to handle share/receive actions

    • Receiver – provides UI to list the files available to download.¬†Posts a download request to Android Download Manager to start¬†file downloads.
    • Sender¬†– displays IP, Port & connected clients info along with file transfer status for each connected client(Receiver).

How to use it? Talk some code!!

I’ve made a DEMO App using SHAREthem library to demonstrate Sender & Receiver Modes. Demo Activity uses a nice library called android-filepicker for selecting files on Device.

Github link is included at the end of this article. Feel free to checkout.

 

Sender Mode:

To start Sender mode, you need to pass an array of strings holding references of files you want to share along with port(optional) and sender name using an Intent to start SHAREthemActivity

Intent intent = new Intent(getApplicationContext(), ShareActivity.class);
intent.putExtra(ShareService.EXTRA_FILE_PATHS, files);
intent.putExtra(ShareService.EXTRA_PORT, 52287);
intent.putExtra(ShareService.EXTRA_SENDER_NAME, "Sri");
startActivity(intent);

ShareService.EXTRA_FILE_PATHS: holds location references to local files on device.
In Sample app of Demo project on github i’ve used a nice looking File Explorer library to select files on device memory to share.
ShareService.EXTRA_PORT (optional): incase if you want to start SHAREserver on a specific port. Passing 0 or skipping this lets system to assign its own. One of the downside of letting system to do this is SSID may be not the same for a subsequent Sharing session. (PORT number is used in algorithm to generate SSID)
ShareService.EXTRA_SENDER_NAME (optional): used by Receiver to display connection information.

ShareActivity invokes SHAREService which in turn starts an instance of SHAREServer with device IP as wildcard address and port if unspecified assigned by system. On successful server setup, Hotspot Controller creates an OPEN WIfi Configuration with an SSID using combination of Android ID, Port number and Sender Name.

Activity scans¬†for connected clients constantly using HotspotControl’s methods and adds them list as shown in below screenshots:

Receiver Mode:

Starting receiver mode is pretty simple as no intent extras needed. Receiver Activity starts scanning for senders automatically, you can turn off Receiver mode anytime though.

startActivity(new Intent(getApplicationContext(), ReceiverActivity.class));

Receiver scans for SHAREthem wifi based on naming convention used and fetches files information from SHAREthem server after successful connection. Here is the glance of Server method of server:

@Override
public Response serve(IHTTPSession session) {
    Response res = null;
    try {
        String url = session.getUri();
        Log.d(TAG, "request uri: " + url);
        if (TextUtils.isEmpty(url) || url.equals("/") || url.contains("/open"))
            res = createHtmlResponse();
        else if (url.equals("/status"))
            res = new NanoHTTPD.Response(Response.Status.OK, NanoHTTPD.MIME_PLAINTEXT, "Available");
        else if (url.equals("/apk"))
            res = createApkResponse(session.getHeaders().get("http-client-ip"));
        else if (url.equals("/logo") || url.equals("/favicon.ico"))
            res = createLogoResponse();
        else if (url.equals("/files"))
            res = createFilePathsResponse();
        else if (url.contains("/file/")) {
            int index = Integer.parseInt(url.replace("/file/", ""));
            if (index != -1)
                res = createFileResponse(m_filesTobeHosted[index], session.getHeaders().get("http-client-ip"));
        }
    } catch (Exception ioe) {
        ioe.printStackTrace();
        res = createErrorResponse(Response.Status.FORBIDDEN, ioe.getMessage());
    } finally {
        if (null == res)
            res = createForbiddenResponse();
    }
    res.addHeader("Accept-Ranges", "bytes");
    res.addHeader("Access-Control-Allow-Origin", "*");
    res.addHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
    return res;
}

Receiver activity embeds FilesListingFragment which is responsible for listing of Sender files and enqueue download request to Android Download Manager

Does it work on WEB browsers too?

Yes indeed. SHAREthem server serves an HTML file response which fires download requests upon selecting links. So all you need to do is key in

http://[ip]:[port]

on a web browser after connecting to hotspot. screen-shot-2017-01-09-at-11-42-31-pm

Nice. Any catches?

Yes, few

  • increasing targetSdkVersion version might impact behaviour of this library
    if targetSdkVersion >= 23
  • ShareActivity has to check for System Write permissions to proceed
  • Get Wifi Scan results method needs GPS to be ON and COARSE location permission.
  • library checks the targetSdkVersion to take care of above scenarios if targetSdkVersion > 20
  • If an application’s targetSdkVersion is LOLLIPOP or newer, network communication may not use Wi-Fi even if Wi-Fi is connected.
    this might impact when Receiver connectivity to SHAREthem hotspot, library checks for this scenario and prompts user to disable data
    For more info: https://developer.android.com/reference/android/net/wifi/WifiManager.html#enableNetwork(int, boolean)

Known issues??

  • On some devices with API level 21 and above, “`connectToOpenHotspot()“` method on WifiUtils disables other Wifi networks in-order to connect to SHAREthem WIfi Hotspot. Ideally increasing priority should be enough for making WifiConfiguration but recent Android version are making it hard for Wifi n.ws with no internet connectivity.
  • Enabling SHARE mode when System Wifi Hotspot is already active (enabled by user via Settings) might not work.

Troubleshooting

  • Receiver Connected but not able to display Sender info:¬†As mentioned above, on API level 21 & above, n/w communication may not use Wifi with no internet access if MOBILE DATA has internet access. Library can prompt to disable MOBILE DATA if this scenario is met, but not yet implemented.
    So turn-off mobile to make Receiver work.
  • ReceiverActivity cannot connect to SHAREthem Wifi:¬†Receiver tries to connect to SHAREthem Wifi in every way possible, but throws an error dialog if it fails. Dialog says to manually select SHAREthem Wifi on WIfi Settings.
    As a last resort, you can always manually connect to Sender Wifi.

Are you gonna give me Source Code??

Sure thing. Just remember that it is licensed under¬† Apache License Version 2.0 ūüôā

Github link: https://github.com/Sriharia/share-them

Please leave your feedback in comments. Thanks

Android app to browse/serve media across DLNA devices

After spending couple of weekends on creating demo app, finally I am able to publish this post on DLNA/Upnp media management.

First of all – What is DLNA?

DLNA is a standard to facilitate access to multimedia content between devices connected within the same computer network. For more information about DLNA, see http://en.wikipedia.org/wiki/Digital_Living_Network_Alliance.

This blog post shows how to create an Android app which can –

  • Browse¬†contents of any DLNA media server and renders their content ¬†locally on Android device.
  • Controls playback on other Media Renderers on same network.
  • Also I have added a functionality to server local device media files to a selected Renderer device thorough hosting a¬†NanoHttpD server in application. (Yes, not a standard DMS device, but works to serve files to other devices)

I have used CyberLinkForJava library which controls DLNA protocols automatically and supports to create your devices and control points quickly. For more info on how this library works, here is a quick programming guide for CyberLink Java.

Let’s run through the details of the demo app – Source & demo APK links are available at the end of blog post ūüôā

Backbone of this app is DLNA service which starts a HandlerThread to handle lifecycle of a ControlPoint which notifies when a device is added/removed along with search response callbacks. This Service gets started in Launcher activity and runs indefinitely until we stop it.

Launcher activity¬†hosts two listviews –¬†PlayFrom & PlayTo.¬†I have appended local device¬†entry to both lists explicitly (not through search callbacks from service). You can browse through ‘PlayFrom‘ devices to pick media content to play on selected Renderer(default is local device).

device-2015-03-07-093227      device-2015-03-07-094149

BrowseDMSManager class takes care of enquiring content directories using service name: ‘urn:schemas-upnp-org:service:ContentDirectory:1

If you select local device to browse, I’ve used FilePickerLibrary¬†to pick files locally and start a HttpServer to serve those files to selected Renderer device. Read my earlier blog¬†post on how to serve local files using HttpServer, though I’ve used NanoHttpD here to handler all possible scenarios.

RemoteDlnaController class is responsible for managing communications b/w client and server devices. It has utilities to send PLAY, PAUSE, STOP, VOLUME controls etc.. actions to Renderer device. device-2015-03-07-094701  Screen Shot 2015-03-07 at 9.45.11 am

I’ve added zoom functionality to Image Pageview and some Jazziness, thanks to TouchView, JazzyViewPager & Universal ImageLoader.

Posted demo source to my GitHub. APK is available here.

let me know what you think of it ūüôā