Lecture 3
In this lecture we will cover more advanced topics of Android development. We will continue with our framework from the previous lecture and show how to add a separate thread for running long tasks. We will also introduce Android NDK and some basics of the Camera2 API.
In this part we will look at how to run long running tasks asynchronously. First, let’s write a function that will take a long time execute and see what happens if we run it on the main application thread.
private void longRunningTask( long taskDurationInMs )
{
long startTime = System.currentTimeMillis();
mMainText.append("Started long running task at" + startTime +"\n");
long currentTime = startTime;
do
{
try {
Thread.sleep( taskDurationInMs );
} catch (InterruptedException e) {
}
currentTime = System.currentTimeMillis();
} while ( currentTime < startTime + taskDurationInMs );
mMainText.append("Ended long running task at" + currentTime +"\n");
}Modify the OnClickListener of Button 2, make it call this long running task with
a duration of 6000 ms, and launch the app. You should notice that when you
press Button 2 the UI becomes sluggish and doesn’t respond well.
Ideally we would like to run the long running tasks on a separate thread so that
the UI keeps responsive. Let’s create a separate thread to run our long task.
Add a variable of type Thread mWorkerThread; and then modify the OnClickListener
to create and start the thread:
mButton2.setOnClickListener( new OnClickListener() {
@Override
public void onClick(View v) {
mMainText.append("Button 2 was pressed!\n");
Log.i(TAG, "Button 2 was pressed");
logMessage("Button 2 was pressed");
mWorkerThread = new Thread ( new Runnable() {
@Override
public void run() {
longRunningTask(6000);
};
});
mWorkerThread.start();
}
});Run the code. What happened?
The application crashed. The reason for the crash can be found by looking at
the logcat output. Android is complaining that we are trying to update a View
from a thread that is different from the one that created it. The culprit here
is the line that calls mMainText.append inside long running task.
To solve this problem, we need to create a Handler that will receive
messages from worker threads and execute actions on the main thread.
Let’s add a Handler class variable:
private Handler mHandler;
private final static int MSG_ASYNC_TASK_STARTED = 0;
private final static int MSG_ASYNC_TASK_COMPLETED = 1;Now, let’s create a Handler.Callback that we will pass on creation to
the Handler. Note that this function will modify the UI components.
private Handler.Callback mHandlerCallback = new Handler.Callback() {
@Override
public boolean handleMessage(Message msg) {
long currentTime = System.currentTimeMillis();
switch( msg.what )
{
case MSG_ASYNC_TASK_STARTED:
mMainText.append("Async task started at " + currentTime + "\n");
return true;
case MSG_ASYNC_TASK_COMPLETED:
mMainText.append("Async task ended at " + currentTime + "\n");
return true;
default:
// The message was not handled, return false
return false;
}
}
};We now create the Handler instance during onCreate:
@Override
public void onCreate(Bundle settings) {
setContentView(R.layout.main_layout);
mLogWriter = openLogFile();
mHandler = new Handler(mHandlerCallback);
...and change the long running task to post messages to the Handler:
private void longRunningTask( long taskDurationInMs )
{
long startTime = System.currentTimeMillis();
mHandler.sendEmptyMessage(MSG_ASYNC_TASK_STARTED);
long currentTime = startTime;
do
{
try {
Thread.sleep( taskDurationInMs );
} catch (InterruptedException e) {
}
currentTime = System.currentTimeMillis();
} while ( currentTime < startTime + taskDurationInMs );
mHandler.sendEmptyMessage(MSG_ASYNC_TASK_COMPLETED);
}We are ready to launch. Run the app and see how responsive it is after
pressing 'Button 2'. The final task for this part is to add a progress dialog
to show a message on the screen. Add a new ProgressDialog variable and
create an instance of a ProgressDialog object.
...
private Handler mHandler;
private ProgressDialog mProgress;
@Override
public void onCreate(Bundle settings) {
setContentView(R.layout.main_layout);
mLogWriter = openLogFile();
mHandler = new Handler(mHandlerCallback);
mProgress = new ProgressDialog(this);
...Then modify the handler callback to show and dismiss the progress dialog:
private Handler.Callback mHandlerCallback = new Handler.Callback() {
@Override
public boolean handleMessage(Message msg) {
long currentTime = System.currentTimeMillis();
switch( msg.what )
{
case MSG_ASYNC_TASK_STARTED:
mMainText.append("Async task started at " + currentTime + "\n");
mProgress.setTitle("Running async task");
mProgress.setMessage("Wait...");
mProgress.show();
return true;
case MSG_ASYNC_TASK_COMPLETED:
mMainText.append("Async task ended at " + currentTime + "\n");
mProgress.dismiss();
return true;
default:
// The message was not handled, return false
return false;
}
}
};Run the app and see the dialog showing up when pressing Button 2
A Java class can have native methods. Native methods are compiled to binary code into a shared library that is loaded into the Java Virtual Machine. These are the steps to add native code to your Android application:
-
Add a 'C' nature to your Eclipse project.
-
Add your native methods.
-
Create your native headers.
-
Add your native code implementation.
-
Add an Android.mk
-
Add an Application.mk
-
Load the shared library with the implementation.
Converting your Eclipse project to C/C++ project allows you to build both Java and native code.
Then, to convert your project, first open the C++ perspective in Eclipse:
Once you have switched to the C/C perspective, goto File -> New -> Convert to C/C project and select the project that you are converting. In the dialog that follows, select 'Makefile project' and 'Android GCC':
Now, whenever you click on the 'Build' button, your native code will be compiled. This behavior is different from the Java perspective, in which the code is continuously being compiled while editing.
We haven’t written a makefile yet, so building will do nothing for the time being.
Native methods have the 'native' keyword. Add the following code to the HelloAndroidActivity class:
private native int square(int n);To write our C code we will use JNI (the Java Native Interface). The JNI provides means for accessing Java objects from C/C++ code. First we need to bind the java code to the C code. One way to do this binding is through the use of special name mangling of the C functions. Because it is difficult to remember how the mangling is done, we can use the javah.
> cd TopDirectoryOfYourProject (where AndroidManifest.xml is)
> javah -d jni -classpath .\bin\classes;<your Android sdk directory>platforms\android-21\android.jar edu.stanford.cs231m.helloandroid.HelloAndroidActivityAfter you run the javah command you will end up with a C header file that contains the C declarations for the native functions:
/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class edu_stanford_cs231m_helloandroid_HelloAndroidActivity */
#ifndef _Included_edu_stanford_cs231m_helloandroid_HelloAndroidActivity
#define _Included_edu_stanford_cs231m_helloandroid_HelloAndroidActivity
#ifdef __cplusplus
extern "C" {
#endif
#undef edu_stanford_cs231m_helloandroid_HelloAndroidActivity_MSG_ASYNC_TASK_STARTED
#define edu_stanford_cs231m_helloandroid_HelloAndroidActivity_MSG_ASYNC_TASK_STARTED 0L
#undef edu_stanford_cs231m_helloandroid_HelloAndroidActivity_MSG_ASYNC_TASK_COMPLETED
#define edu_stanford_cs231m_helloandroid_HelloAndroidActivity_MSG_ASYNC_TASK_COMPLETED 1L
/*
* Class: edu_stanford_cs231m_helloandroid_HelloAndroidActivity
* Method: square
* Signature: (I)I
*/
JNIEXPORT jint JNICALL Java_edu_stanford_cs231m_helloandroid_HelloAndroidActivity_square
(JNIEnv *, jobject, jint);
#ifdef __cplusplus
}
#endif
#endifNow we are going to create a HelloAndroid.cpp file under the jni folder with the following code:
#include <jni.h>
#ifdef __cplusplus
extern "C" {
#endif
/*
* Class: edu_stanford_cs231m_helloandroid_HelloAndroidActivity
* Method: square
* Signature: (I)I
*/
JNIEXPORT jint JNICALL Java_edu_stanford_cs231m_helloandroid_HelloAndroidActivity_square
(JNIEnv *jni, jobject thiz, jint n)
{
return n * n;
}
#ifdef __cplusplus
}
#endifThe Android.mk is the makefile for the NDK.
LOCAL_PATH := $(call my-dir)
LOCAL_MODULE := HelloAndroid
LOCAL_SRC_FILES := HelloAndroid.cpp
include $(BUILD_SHARED_LIBRARY)The Application.mk defines which NDK API we are targeting, what type of STL library we will be using, and most importantly, which ABI we are targeting. In the example below, we target ARM processors.
APP_PLATFORM := android-19
APP_ABI := armeabi-v7a
APP_STL := gnustl_staticWe can now build our native code. Click on the Build project button and see the build output on the Eclipse console.
Before we can instantiate a class with a native method, we need to load the shared library that we have just built into the Java VM. We do this in the class initializer method:
static {
System.loadLibrary("HelloAndroid");
}When we compile a native library, Eclipse is invoking under the hood the 'ndk-build' command. You could even try yourself to call 'ndk-build' from the command-line. 'ndk-build' parses and executes the build instructions in the 'Android.mk' and 'Application.mk' files. By default, the generated binary is optimized and is stripped of debugging information. If you want to debug your native code, you will need to add the 'NDK_DEBUG=1' build option. To do this, right-click on the project, choose 'Properties→C/C++ Build', and add the 'NDK_DEBUG=1' option as shown below:
Note that the native code debugger is different from the Java debugger. It is a different instance. When launching an application for native debugging, you will need to choose 'Debug as → Android Native Application'.
Unfortunately, the native debugger attaches to the application process after the application has launched. It is possible then that the code you want to debug has already executed by the time the debugger is attached. When this happens, we can use a trick: add an infinite loop in your native code that you will break from the debugger. Here is a way to do it; first, modify the 'Android.mk':
ifeq (1, $(NDK_DEBUG))
LOCAL_CFLAGS += -DWAIT_FOR_DEBUGGER
endifThen define the 'waitForDebugger' function and call it from your code at the point where you would like your program to wait for the debugger to attach:
#if defined(WAIT_FOR_DEBUGGER)
void waitForDebugger()
{
static volatile int debug = 1;
while( debug )
{}
}
#else
void waitForDebugger() {}
#endif;When your debugger attaches to the running application, break into the debugger and change the value of the variable 'debug' from 1 to 0.
To operate the camera, follow these steps:
-
Modify your AndroidManifest.xml requesting permissions for camera access
-
Get a reference to the
CameraManager. -
Find the camera you want to operate on.
-
Open the camera.
-
Decide which outputs you need from the camera.
-
Create a
CameraCaptureSession -
Create and submit capture requests.
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />private void enumerateCameras() {
mCameraManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
try {
String cameraIdList[] = mCameraManager.getCameraIdList();
mMainTxt.append("Number of cameras " + cameraIdList.length + "\n");
for ( int i = 0; i < cameraIdList.length; ++i )
{
// Get the camera characteristics
CameraCharacteristics properties;
properties = mCameraManager.getCameraCharacteristics(cameraIdList[i]);
// Which way is this camera facing?
int lensFacing = properties.get( CameraCharacteristics.LENS_FACING );
// Print properties
mMainTxt.append("Camera id: " + cameraIdList[i] + "\n");
mMainTxt.append(" -Orientation: ");
if ( lensFacing == CameraCharacteristics.LENS_FACING_FRONT ) {
mMainTxt.append("FRONT_FACING\n");
}
else if ( lensFacing == CameraCharacteristics.LENS_FACING_BACK )
{
mMainTxt.append("BACK_FACING\n");
}
}
} catch( CameraAccessException e ) {
}
}First, let’s create a function to find the cameraId of the back-facing camera:
private String findBackFacingCamera()
{
try {
String cameraIdList[] = mCameraManager.getCameraIdList();
for ( int i = 0; i < cameraIdList.length; ++i )
{
// Get the camera characteristics
CameraCharacteristics properties;
properties = mCameraManager.getCameraCharacteristics(cameraIdList[i]);
if ( properties.get( CameraCharacteristics.LENS_FACING ) ==
CameraCharacteristics.LENS_FACING_BACK ) {
return cameraIdList[i];
}
}
} catch( CameraAccessException e ) {
}
return null;
}Now, we can open the camera by calling the openCamera function on our
CameraManager instance. Note that this function does not immediately
return a CameraDevice instance. Instead, we pass a CameraDevice.StateCallback
that gets called when the camera has been successfully opened, or when the
operation has miserably failed. After the camera has been opened, we will
create the CameraCaptureSession.
private void openBackFacingCamera()
{
String cameraId = findBackFacingCamera();
try
{
mCameraManager.openCamera(
cameraId,
new StateCallback() {
@Override
public void onDisconnected(CameraDevice arg0) {
Log.e(TAG, "Camera was disconnected\n");
}
@Override
public void onError(CameraDevice arg0, int arg1) {
Log.e(TAG, "Error opening camera");
}
@Override
public void onOpened(CameraDevice arg0) {
Log.i(TAG, "Opened camera!");
mCameraDevice = arg0;
createCaptureSession();
}
}
, null);
}
catch ( CameraAccessException e)
{
}
}A CameraCaptureSessions configures a set of outputs. These outputs will be available for the camera to fill. Outputs can be:
-
A SurfaceView object that will show an image on screen.
-
A SurfaceTexture for rendering in an OpenGL context.
-
An ImageReader object that will hold a YUV or RAW image for CPU processing.
-
An ImageReader object that will hold a JPEG image.
-
A MediaCodec surface that can be sent to the video encoder for video recording.
For this simple example we will use a SurfaceView object. For each of the above
objects, only certain sizes and formats are permitted. The sizes and formats
available are obtained from a StreamConfigurationMap object.
StreamConfigurationMap map = mCameraManager.getCameraCharacteristics(cameraIdList[i]).
get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
android.util.Size sizes[] = map.getOutputSizes(SurfaceHolder.class);
for ( int j = 0; j < sizes.length; j++ ) {
Log.i(TAG, "- " + sizes[j].getWidth() + " x " + sizes[j].getHeight());
}Since we want to show the camera output on a SurfaceView, we need a new
layout that contains one. Here is one:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="horizontal" >
<SurfaceView
android:id="@+id/cameraView"
android:layout_width="1296px"
android:layout_height="972px"
android:layout_weight = "0"
/>
<LinearLayout
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:layout_weight = "1"
android:orientation="vertical" >
<Button
android:id="@+id/button1"
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:text="Button 1" />
<Button
android:id="@+id/button2"
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:text="Button 2" />
</LinearLayout>
</LinearLayout>Now, we can only create the CameraCaptureSession after the SurfaceView
has been created. Let’s declare all our class instance variables first:
public class CameraActivity extends Activity {
final static String TAG = "CameraActivity";
TextView mMainTxt;
SurfaceView mCameraView;
CameraManager mCameraManager;
CameraDevice mCameraDevice;
CameraCaptureSession mCaptureSession;The onCreate method has to set a callback to get notified that the
SurfaceView has been created. Since a SurfaceView requires some system
resources, creation is done in an asynchronous way, and notifications
provided through callbacks. After the SurfaceView has been created,
we fix its size to one of the sizes provided by getOutputsSize. Then
we call our function to open the camera:
@Override
public void onCreate(Bundle settings) {
setContentView(R.layout.main_layout);
mCameraView = (SurfaceView) findViewById(R.id.cameraView);
mCameraView.getHolder().addCallback( new SurfaceHolder.Callback() {
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
@Override
public void surfaceCreated(SurfaceHolder holder) {
holder.setFixedSize(1296, 972);
openBackFacingCamera();
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width,
int height) {
// TODO Auto-generated method stub
}
});
mCameraManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
super.onCreate(settings);
}We are now ready to write the function that will create the
CameraCaptureSession. The function createCaptureSession has a similar
spirit as openCamera: we need to pass a callback object to get notified
when the CameraCaptureSession has been completely configured. This is
a heavy-weight operation that requires allocating a number of system resources.
We create our session to capture to our SurfaceView instance. After
the session is created, we can start the camera.
private void createCaptureSession() {
// Create the list of outputs.
ArrayList<Surface> outputs = new ArrayList<Surface>();
outputs.add( mCameraView.getHolder().getSurface() );
try {
mCameraDevice.createCaptureSession(
outputs,
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigureFailed(CameraCaptureSession session) {
Log.e(TAG, "Failed to configure capture session");
}
@Override
public void onConfigured(CameraCaptureSession session) {
Log.i(TAG, "CaptureSession configured successfuly");
mCaptureSession = session;
startCamera();
}
},
null);
} catch( CameraAccessException e) {
Log.e(TAG, "Camera Access Exception in call to createCaptureSession");
}
}To start the camera we submit a request. A request contains all the configuration
parameters for the camera and the outputs that will be enabled by this request.
The camera subsystem will do its best to match the parameters of the request.
Because the number of the parameters in a request is quite large, we build a
request from a pre-existing template. The PREVIEW template is optimzied
for low-quality high-speed captures, and that is the one we will use in
this example:
private void startCamera() {
try {
CaptureRequest.Builder builder;
// Create the request builder.
builder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
builder.addTarget( mCameraView.getHolder().getSurface() );
mCaptureSession.setRepeatingRequest( builder.build(), null, null);
} catch( CameraAccessException e) {
Log.e(TAG, "Camera Access Exception in call to createCaptureRequest");
}
}That’s it! Now we can run the app and you should see the camera view on the screen.
When your Activity is about to go to the background, don’t forget to close
the camera! A camera is an exclusive access device, if you don’t close it
no other Activity will be able to open the camera. Add an onPause method
to your Activity:
@Override
protected void onPause() {
if (mCameraDevice != null) {
mCameraDevice.close();
}
super.onPause();
}