Android Rendering Options

I've given some talks about Android graphics recently, and the same questions keep coming up:

• What is Renderscript appropriate for?
• Is it a replacement for OpenGL rendering?
• How do I access Renderscript from NDK code?
• When do I choose between the different rendering options?
• Can I have a free tablet?

It seemed to me that a brief article might help. In particular, I wanted to write something that lays out the various rendering options that Android provides and describes how we envision developers using them (and how we use them ourselves in the applications that we ship) so that you might better understand how and when to use them in your applications. Of course you're not bound to what we think; if you come up with new ways to use what the platform provides, all the better. But we can at least provide a little guidance to help you navigate the myriad of options. Hopefully this article will help answer the questions above. Except for the last question about the free tablet - I can't help you there.

First, I should explain what I mean by rendering, for everyone reading this that's not a graphics geek. "Rendering" is a synonym for drawing. Rendering is also defined as the process of turning an animal carcass into fat and other component pieces... but that's not what I'm talking about here. This article is about is all of the ways, in the Android platform, of drawing things onto the display. For example, you might have a drawing application, or a photo album, or a game, or a reading program, or anything else that needs to draw its content to the screen. If you're just using the SDK components (ImageView, Button, TextView, and so on), then we handle the rendering of these components internally. But if you have custom components that draw their own graphical elements, or if you're writing a fullscreen game that draws everything itself, then you need to know how to draw those items onto the device screen so that the user can see and interact with them.

There are two options for writing Android applications, each with their own ways of rendering graphics. I'll cover each of these in the following sections.

• SDK
• NDK

The Android SDK

The SDK is the typical, default approach for most GUI applications. You write your application using the Java programming language and the libraries that the SDK provides. Most of the graphics on the screen are drawn by the framework for the standard View objects such as Button, TextView, ListView, and so on. If you create custom Views or have customized rendering that you want to do for your application, then you need to know about the rendering options available to SDK applications. There are basically three options:

• The Canvas API
• Renderscript
• OpenGL wrappers

The Canvas API

What I mean by "Canvas API" is the standard rendering for a typical SDK application that uses View objects (standard and custom) and performs and rendering by calling the various methods in the Canvas class. The rendering process of the Android view hierarchy consists of calls to each View's onDraw() method. This method takes a single parameter, Canvas, which is the object used by the view to draw its content. For example, a Button might tell its background Drawable to draw itself into the Canvas and then draw its label with a call to Canvas.drawText().

If you have a custom View, in which you implement the onDraw() method and draw your own graphic elements, then you would typically call methods in the Canvas object passed in to the onDraw() method. For example, you might draw lines with Canvas.drawLine() and circles with Canvas.drawCircle().

This approach to rendering is typical for most GUI applications, and is fine for most purposes when you just need standard GUI components and graphics and for which the performance of your application is not an issue. With the GPU acceleration that we added in Android 3.0, the performance for the Canvas API is even better than it used to be. As discussed in the article Android 3.0 Hardware Acceleration, all rendering done by the SDK now happens via OpenGL calls to the GPU. This level of hardware acceleration for standard as well as custom views and graphics can provide excellent performance and allows for parallelism by offloading rendering work from the CPU onto the GPU.

But sometimes you may want to go beyond what the Canvas API provides, either in terms of the graphics functionality you need or the performance that your particular situation requires. For example, the Canvas API does not provide complete 3D rendering capabilities. In these situations, you might want to look beyond the Canvas API to the other options available in the SDK.

Renderscript

The Renderscript API was introduced in Android 3.0. Renderscript was actually in use in an earlier form long before this release; the Live Wallpapers that shipped with earlier releases were written using Renderscript. But Android 3.0 introduced the first public form of the API and the runtime library. Describing what Renderscript is and how to use it is beyond the scope of this article. Instead, I'll just refer you to Jason Sams's introductory articles, Introducing Renderscript and Renderscript Part 2. Very briefly, Renderscript is a language, API, and runtime library that you can use to achieve high performance rendering and computation for Android applications.

An application using Renderscript is an SDK application, using any of the SDK APIs and facilities provided at that level, with addition pieces of the application written using the Renderscript language. These scripts are initialized from Dalvik code. The scripts automatically provide glue code that wraps fields at the Renderscript level, so that SDK code can set values in Renderscript code by calling simple methods. Renderscript has a reference to a drawing surface that it renders to in a thread that is separate from the UI toolkit thread. It handles its own rendering and the timing of when that rendering happens.

One of the things that makes Renderscript compelling is that it can determine, at runtime, the best way to achieve high performance for a particular operation. For example, it might send graphics operations down to the GPU, or it might parcel out independent computations to separate cores in a multi-core device. All of this is transparent to the application; it simply starts the script, sets the values, and lets it go.

An important use case for Renderscript arises for SDK applications that need functionality or performance that cannot be had through the SDK directly. For example, your application may need to perform 3D graphics operations or heavy computation that takes too much time at the SDK level. You might consider taking these portions of your application and farming them out to Renderscript to do the work there instead.

An important consequence of the Renderscript approach using SDK and runtime-compiled code is that Renderscript applications are fully portable between devices. That is, you only need to compile your application once, not per-architecture (as you must do with the NDK). This single apk is then runnable on any device that supports the SDK level that you compiled for (3.0 and above, since 3.0 is the first Android release to support Renderscript).

Examples of Renderscript in Android 3.0 include the Live Wallpapers, the video wall view in the YouTube application, and the Books application (including that beautiful page-turn effect). Other examples can be found in the SDK samples; you should check these out to understand how to write Renderscript as well as how to initialize and interact with the scripts from SDK applications.

OpenGL Wrapper Functions

Android also provides wrappers for OpenGL APIs at the SDK level. That is, you can write an application using the SDK, with full access to usual SDK APIs and functionality, and still use OpenGL ES 1.x and OpenGL ES 2.0 APIs, by calling the wrapper functions in GLES10 or GLES20 classes. These wrappers call the underlying OpenGL APIs at the native level for those versions of OpenGL ES.

For casually exploring or using OpenGL, this may be a reasonable option. But while this approach works, it's not a recommended practice as a general approach for complex scenes that require high-performance graphics. For one thing, it is difficult to achieve high performance levels equivalent to native access to OpenGL due to the overhead of calling down from the SDK to the native level for every OpenGL call.

The Music application that shipped with Android 3.0 used this approach. The application was an SDK application which needed some simple 3D graphics operations, such as the carousel view of albums. The main reason that it uses the SDK/OpenGL approach is because it is an unbundled application (not dependent upon a particular release of the SDK) and needed to work on releases back to 2.2. It therefore used APIs that were available on those previous releases.

NDK

The NDK exists to provide an easy porting layer for existing applications written in native code, or which use native libraries. Porting might be more easily and quickly accomplished by using the NDK than by converting existing code to the language and APIs used by the SDK.

The NDK does not provide the rich GUI toolkit of the Android platform at the native level, so developers do not have access to the View objects, or the events, or the rest of the infrastructure that is provided in the SDK APIs. But there is a graphics environment at the NDK level that is sufficient for some specific purposes. In particular, game developers that simply want a fullscreen game experience can find what they need with OpenGL. This API provides low-level graphics functionality that lets applications display 2D and 3D graphics using the GPU for maximum graphics performance.

One important restriction of the NDK to keep in mind is that it compiles applications to specific CPU architectures. This means that if you only build your application for one specific chip, then the application will not work on Android devices that do not have that chip architecture. This is particularly important in the broad and growing Android ecosystem where new devices are coming out all the time. You probably want your application to work as well on new chips as it did on the ones you used to develop the application. So while the NDK OpenGL solution provides a reasonable route to fast graphics performance, it does so at the cost of the portability that other solutions offer.

One of the questions that came up about Renderscript when I talked to developers was how to access it from the NDK. This is not currently possible. Renderscript is specifically created to be a companion to SDK applications. We envision users of Renderscript as SDK applications that use Renderscript for their high-performance graphics or computation needs. These applications might be primarily SDK applications, with GUIs and interaction and most other facilities provided by the SDK APIs and small but important pieces of functionality provided by Renderscript, such as graphics effects that would otherwise not be possible or not perform as well. Or these applications might use Renderscript for most of what they do, just using the SDK to initialize the scripts and letting Renderscript take over from there.

And So...

I've tried to explain and categorize the different rendering options that we provide in the Android platform. What may seem confusing at first, because of the potential overlap, is actually simple once you understand how these libraries differ from each other in their purpose, their use, and their advantages. Here's one last brief take at a breakdown of when you might consider the various options:

• SDK Canvas API: If you're writing a normal GUI application, using the SDK and the Canvas API is probably the most sensible option. If you need custom 2D graphics, there is plenty of power and reasonable performance in the Canvas API.
• SDK Renderscript: If you want a fully portable, SDK application with access to the power, speed, and functionality of the Renderscript library and underlying GPU and multi-core CPU architectures, then it's worth exploring this new feature of Android.
• SDK OpenGL Wrappers: If you are writing an SDK application and want to sprinkle in some 3D effects with OpenGL, you could see whether the OpenGL wrapper functions suit your needs.
• NDK OpenGL: If you are porting your application from a different platform or existing native code and are looking for the fastest way to get it up and running on Android, then using the NDK with OpenGL is worth considering.

Whatever your rendering need, Android has something for you. So keep those cool graphics applications coming! (Sorry I still can't help you with the free tablet, though. There were tablets given away at Google I/O this year, so you could get one of those if you can travel back in time. I think there's an app for that. It probably uses Renderscript.)