Create an Android Cardboard 360 Video Viewer

Augmented and virtual reality, while still relatively new, have quickly become popular for apps, including for gaming and education. Previously, I showed you how to install Cardboard using the Android SDK and how to create a panoramic image viewer. This post will show you how to use 360-degree video in your apps.

• 
  Android SDK
  
  
  Get Started With Android VR and Google Cardboard: Panoramic Images
  
  
  Paul Trebilcox-Ruiz

Setup

Before you start building your video viewer app, you will need to clone the Cardboard Android SDK onto your computer via Git. You can find instructions for this in the previous article of this series.

For our sample, create a new Android project with a minimum SDK of API 19 (KitKat) and use the Empty Activity template.

Once your base project is created, you will need to copy the common, commonwidget and videowidget folders from the Cardboard SDK to the root of your project. Once those directories are moved over, you will need to include them as modules in your project by editing your settings.gradle file to look like the following snippet. 

include ':app', ":common", "commonwidget", "videowidget"

Finally, include these and other required libraries in your project by adding the following lines to your app module's build.gradle file under the dependencies node.

dependencies {
    compile 'com.android.support:appcompat-v7:25.0.0'

    compile project(':common')
    compile project(':commonwidget')
    compile project(':videowidget')

    compile 'com.google.android.exoplayer:exoplayer:r1.5.10'
    compile 'com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-7'
}

You'll notice that we added the Protocol Buffers library from Google, which helps manage runtime resources on the device, and ExoPlayer, which is a Google-created video player library that the VrVideoView component is built on. Both of these libraries are required by the Cardboard SDK in order to function, and you may have noticed that the ExoPlayer version used is from the first release, not the second, so may cause conflicts if you use ExoPlayer v2 in your own projects.

Next, we'll want to update our activity_main.xml file for our sample project to include a VrVideoView, a SeekBar, and a Button that we will work with later.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="https://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">

    <com.google.vr.sdk.widgets.video.VrVideoView
        android:id="@+id/video_view"
        android:layout_width="match_parent"
        android:layout_height="250dp"/>

    <SeekBar
        android:id="@+id/seek_bar"
        android:layout_height="32dp"
        android:layout_width="match_parent"
        style="?android:attr/progressBarStyleHorizontal"/>

    <Button
        android:id="@+id/btn_volume"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Volume Toggle"/>

</LinearLayout>

Once you're done setting up the Cardboard libraries and have created the layout that we will use, it's time to jump into the Java code.

Working With Cardboard and VR Video

Before we can start writing all of the code that controls playback state, position, and loading in your 360 video file, we will need to determine the source of our video. For this tutorial we will simply create an assets folder under the main directory, and place a 360 video there. While there are a few sources for 360 videos online, I have included a short public domain video from Sea World returning a sea turtle to the ocean in the accompanying GitHub project for this tutorial. 

Initializing and Structure

Now that you have a video file to play, open up your MainActivity class.

You will first need to declare and initialize the View items that are defined by the layout file, as well as two boolean values to keep track of muted and play/pause state. In addition, we will place an OnClickListener on our volume Button object.

public class MainActivity extends AppCompatActivity {

    private VrVideoView mVrVideoView;
    private SeekBar mSeekBar;
    private Button mVolumeButton;
    
    private boolean mIsPaused;
    private boolean mIsMuted;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        initViews();

    }

    private void initViews() {
        mVrVideoView = (VrVideoView) findViewById(R.id.video_view);
        mSeekBar = (SeekBar) findViewById(R.id.seek_bar);
        mVolumeButton = (Button) findViewById(R.id.btn_volume);

        mVolumeButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                onVolumeToggleClicked();
            }
        });
    }

    public void playPause() {

    }

    public void onVolumeToggleClicked() {

    }
}

Next, create a new inner class that extends VrVideoEventListener. This class will have five methods that we can implement for our simple video viewer.

private class ActivityEventListener extends VrVideoEventListener {
    @Override
    public void onLoadSuccess() {
        super.onLoadSuccess();
    }

    @Override
    public void onLoadError(String errorMessage) {
        super.onLoadError(errorMessage);
    }

    @Override
    public void onClick() {
        super.onClick();
    }

    @Override
    public void onNewFrame() {
        super.onNewFrame();
    }

    @Override
    public void onCompletion() {
        super.onCompletion();
    }
}

You will also need to implement SeekBar.OnSeekBarChangeListener in your class and create the method stubs for that interface.

public class MainActivity extends AppCompatActivity implements SeekBar.OnSeekBarChangeListener {

...

    public void onPlayPausePressed() {

    }

    public void onVolumeToggleClicked() {

    }

    @Override
    public void onProgressChanged(SeekBar seekBar, int i, boolean b) {

    }
}

Once you have created this new inner class and SeekBar implementation, associate them with your VrVideoView and SeekBar, respectively, at the end of the initViews() method that was defined above.

mVrVideoView.setEventListener(new ActivityEventListener());
mSeekBar.setOnSeekBarChangeListener(this);

There's one more piece of setup that needs to be taken care of. You will need to handle the Android activity lifecycle by supporting the onPause(), onResume() and onDestroy() methods to pause or resume rendering in the VrVideoView, or to completely shut it down. You will also need to keep track of the pause state in these methods.

@Override
protected void onPause() {
    super.onPause();
    mVrVideoView.pauseRendering();
    mIsPaused = true;
}

@Override
protected void onResume() {
    super.onResume();
    mVrVideoView.resumeRendering();
    mIsPaused = false;
}

@Override
protected void onDestroy() {
    mVrVideoView.shutdown();
    super.onDestroy();
}

Now that the initial setup of our tutorial class is completed, we can move on to the more interesting topic: loading a video, controlling playback, and customizing the user experience.

Starting and Controlling VrVideoView

Because loading a 360 video can take anywhere from a fraction of a second to several seconds, you will want to handle loading your video through a background task. Let's start by creating a new AsyncTask that will create a new VrVideoView.Options object, set the input type to match our video's formatting (in the case of this tutorial, TYPE_MONO), and then load our video from the assets directory.

class VideoLoaderTask extends AsyncTask<Void, Void, Boolean> {

    @Override
    protected Boolean doInBackground(Void... voids) {
        try {
            VrVideoView.Options options = new VrVideoView.Options();
            options.inputType = VrVideoView.Options.TYPE_MONO;
            mVrVideoView.loadVideoFromAsset("seaturtle.mp4", options);
        } catch( IOException e ) {
            //Handle exception
        }

        return true;
    }
}

Next, go into your onCreate() method and create a new instance of this task, and then call execute() to start it. While there's some more that should be done to properly maintain this task, we'll just use it locally in this method for simplicity and not worry about AsyncTask lifecycle considerations.

VideoLoaderTask mBackgroundVideoLoaderTask = new VideoLoaderTask();
mBackgroundVideoLoaderTask.execute();

At this point, you should be able to run your application and watch the 360 video play in the Cardboard video view. Now that that's working, let's add in some utility for our user. Return to the ActivityEventListener object that you created earlier in this tutorial, as we will want to flesh out some of the methods. When the video has successfully loaded, we need to set the max value for our SeekBar, as well as keep track of the play/pause state of our video.

@Override
public void onLoadSuccess() {
    super.onLoadSuccess();

    mSeekBar.setMax((int) mVrVideoView.getDuration());
    mIsPaused = false;
}

As the video plays, we will update that SeekBar through onNewFrame(), and reset the video to the initial position in onCompletion(). Lastly, in onClick(), we will trigger our play/pause toggle method.

@Override
public void onClick() {
    playPause();
}

@Override
public void onNewFrame() {
    super.onNewFrame();

    mSeekBar.setProgress((int) mVrVideoView.getCurrentPosition());
}

@Override
public void onCompletion() {
    //Restart the video, allowing it to loop
    mVrVideoView.seekTo(0);
}

While updating our SeekBar based on playback is important, we will also want to allow the user to change where they are in the video by interacting with the SeekBar. We can do this using the SeekBar.OnSeekBarChangeListener interface that we implemented earlier.

@Override
public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
    if( fromUser ) {
        mVrVideoView.seekTo(progress);
    }
}

To round off our VrVideoView controls, we will need to implement the play/pause and volume toggle methods.

public void playPause() {
    if( mIsPaused ) {
        mVrVideoView.playVideo();
    } else {
        mVrVideoView.pauseVideo();
    }

    mIsPaused = !mIsPaused;
}

public void onVolumeToggleClicked() {
    mIsMuted = !mIsMuted;
    mVrVideoView.setVolume(mIsMuted ? 0.0f : 1.0f);

}

At this point, you should have a fully working and interactive 360 video player working in your app. 

However, if you rotate your device, you may notice the unwanted behavior of the video completely restarting. We can fix this by working with Android's onSaveInstanceState() and onRestoreInstanceState() to save and reset the state of our VrVideoView.

private static final String STATE_PROGRESS = "state_progress";
private static final String STATE_DURATION = "state_duration";

@Override
protected void onSaveInstanceState(Bundle outState) {
    outState.putLong(STATE_PROGRESS, mVrVideoView.getCurrentPosition());
    outState.putLong(STATE_DURATION, mVrVideoView.getDuration());

    super.onSaveInstanceState(outState);
}

@Override
protected void onRestoreInstanceState(Bundle savedInstanceState) {
    super.onRestoreInstanceState(savedInstanceState);

    long progress = savedInstanceState.getLong(STATE_PROGRESS);

    mVrVideoView.seekTo(progress);
    mSeekBar.setMax((int) savedInstanceState.getLong(STATE_DURATION));
    mSeekBar.setProgress((int) progress);
}

Now, when the device is rotated, your video should return to its original position and your user can continue with their uninterrupted experience.

Conclusion

While there are a few small details that need to be handled to use the Cardboard SDK's VrVideoView, the difficult parts, such as actual playback and optimization, are handled for you by a component that's easy to implement. 

You should now be able to add 360 videos to your media apps, providing your users with an interesting feature that enriches their experience. In the next tutorial of this series, we will focus on Google's new VR experience called Daydream, and how to use the paired controller with your apps.

In the meantime, check out some of our other tutorials on Android virtual reality and augmented reality!

• 
  Android SDK
  
  
  Get Started With Android VR and Google Cardboard: Panoramic Images
  
  
  Paul Trebilcox-Ruiz


• 
  Mobile Development
  
  
  Pokémon GO Style Augmented Reality With Vuforia
  
  
  Tin Megali


• 
  Mobile Development
  
  
  Create a Pokémon GO Style Augmented Reality Game With Vuforia
  
  
  Tin Megali


• 
  Android
  
  
  Explore VR With Google Cardboard
  
  
  Sue Smith