Devoxx 2011 Recordings: Now Free on Parleys.com

All of the recordings of the technical sessions I gave with Romain Guy at Devoxx 2011 are now free on the Parleys.com site. This means that you'll have a chance to watch them all before we go to Antwerp next month and present several more sessions. Which means we'll need another five hours of new material. Dangit.

In the meantime:

Android Awesomeness (Part 1 and Part 2)
This vaguely-titled talk was in two parts. In the first half, Romain and I did a very quick introduction to the Android 4.0 release. The second half was more interactive, as we showed how we use the tools that ship with the SDK to debug performance, memory, and UI issues.
You can also download the slides for this talk here: Part 1 and Part 2.

Graphics Goodies
This session was an updated version of the Android Accelerated Rendering talk we did at Google I/O 2011.
Here are the slides.

Sticky GUIs
This presentation was a collection of techniques and principles for creating GUI applications that will make your users stick around: graphics, performance, animations, GUIs; they're all important.
Here are the slides.

There are many other good (and free!) technical talks from Devoxx 2011, including other Android talks on tools and application development and plenty of other geeky topics as well. Check them out on the Parleys Devoxx  '11 site. (Did I mention they're free?)

And hopefully see some of you in Antwerp in a month. After we've written a ton of new material.

Thanks for the Memories

(This tip and anecdote is specifically about Android, but the same technique applies to every platform I've ever worked on, which at this point is quite a few. In fact, it's a technique that Romain and I have stressed in our book and in most graphics/GUI talks that we've given over the years, such as this one from Devoxx in 2008) .
I was debugging an application recently (names withheld to protect the completely and utterly guilty) and discovered that the source of a serious performance bottleneck was simply the size of the bitmaps involved.

The application's job is to display lots of pictures, so using bitmaps is a given. And the size of the bitmaps being loaded and displayed is signficant, so there were going to be issues around memory and performance anyway. But it was the way in which the application was treating the source and destination sizes that was at the root of the problem.

In particular, the application was loading each image into a bitmap of size X x Y. Meanwhile, they wanted to draw that bitmap into a destination rectangle half that size, .5X x .5Y. This is easy to do; you just call the appropriate Canvas.drawBitmap() method with the relevant source/destination rectangles, or specify a scale on the Canvas object, and we'll take care of the details.

Simple.

But.

Wrong.

That is: Scaling on the fly is easy, and it works. But asking us to do your work for you on every frame in which you draw that bitmap might cost you significantly in performance and memory when there's a very easy way for you to do this once and simplify all future operations with that bitmap.

Here's the right thing to do: pre-scale the bitmap to exactly the size you need. Then when you need to copy it into the destination, you call Canvas.drawBitmap(left, top, Paint) (the version that doesn't take a dest rectangle) and then all that Android needs to do is copy the bitmap. Much simpler. And what's more: it requires potentially far less memory than downscaling to a smaller size.

This is probably (hopefully? please?) obvious when you're running in a software-rendering situation (e.g., all releases prior to 3.0, or any app targeting pre-4.0 releases and not specifying hardwareAccelerated="true" in the manifest); having the framework scale the image every time it's drawn means going through a much slower path than a simple 1-to-1 copy entails.

But what about on GPUs, with our wonderful new hardware-accelerated world of Android apps as of version 3.0+? Aren't GPUs supposed to be faster at stuff like this? What are we paying them for, anyway?

Yes.

But.

Here's the problem: the actual scaling operation is quite cheap on a GPU, even negligible. But that's not all that you should be concerned about as a mobile developer. Mobile developers should always worry about memory. You should profile your application. You should think about memory consumption at night when you can't sleep. You should bring it up on first dates*, and fester on it while on vacation.

If you're displaying several images per frame, you want to be very aware of how much memory those bitmaps are soaking up. This is true for the bitmaps in CPU memory, but also true for bitmaps that we upload to textures. Just because it's cheap for a GPU to scale a large texture into into a small space on the GPU doesn't mean it's fast to upload it into texture memory, or cheap for the GPU to have that large image sitting around in memory. Memory is a constrained resource and should be treated as such. I'm sure your date will tell you as much (possibly as they leave the date in search of more interesting prospects).

If you're going to scale from a 2k x 2k image into a 32x32 icon, wouldn't it make more sense to pre-scale it once, chuck the original one, and thenceforth deal with only the smaller version instead?

Of course, if you really need full-size images, then go ahead and do what you need to do. And if you're animating an image's size (such as zooming in on it), then pre-scaling to each intermediate size probably doesn't make much sense. But if you know that you're going to be using a smaller version for a while, then you should probably pre-scale to that size rather than drag around the memory and performance baggage associated with the original version, no matter what the hardware acceleration situation on the target device is like.

* The advantage of discussing memory consumption on first dates is not only that it will help you keep it in mind at all times, but also that this will inevitably lead to more first dates on which you can continue discussing it. Or it will at least result in less second dates.

Google I|O 2012 Presentations

The videos and slides are now posted for the presentations that I did with Romain Guy at Google I/O a couple of weeks ago.

What's New in Android: A developer's overview of some of the new features available in Android 4.1 Jelly Bean, including a deep dive into the enhanced Notifications capabilities (this part by Daniel Sandler, one of the engineers responsible).

For Butter or Worse: A very geeky look into how hardware accelerated rendering works on Android, what we did to make things smoother and faster in the latest release, and what Android developers can do to make their applications faster and smoother as well.

These videos were posted within a day of the talks, but the slides took a bit longer (possibly explained by the rush up to IO followed by everyone collapsing in fatuge when it was over).

There are tons of other great talks from the conference that are posted as well, both in the Google Developers channel on YouTube and (with slides) on the IO site itself.

Enjoy...

Curved Motion in Android

The animation support added in Android 3.0 and enhanced since then is useful, allowing a flexible system of property animation to animate literally anything you want. Like a store going out of business: if it ain't nailed down, it's up for grabs.

But that doesn't mean we're done yet; there are many things that we'd like to do to keep improving the animation system and making Android animations easier to write and more powerful. One of those things is curved motion.

Right now, if you animate something between a start and end point, it will move in a straight line between those values. For example, a translation animation, where the object moves from one (x, y) location to another, will move in a straight line beween those points. You can (and should) use non-linear timing for these animations, and the animation framework uses ease-in/ease-out timing by default. But the actual spacial movement is linear; there is no way for you to specify a curved path for an object to take between the end values of an animation.

Even if you use a multi-point keyframe animation, where you specify several intermediate points for the animation to pass through along the way, you are still going to move linearly between each of those points.

One goal with animations is to make them feel as natural as possible. And just as motion in the real world (you know, the one we have to use as we move between the machines of our lives) is not linear, animations in our GUIs should not be limited to linear. For example, f a view moves from corner of the screen to the opposite corner, wouldn't it be nice to have the option to give it a little curve in and out at both ends, instead of being locked into a straight-line movement?

We'd like to add this kind of capability to the animation system in a future release; it needs to be easy to create such animations, we just have to provide the APIs and functionality to let you do it.

As I was looking into the problem, I created some prototype code using the existing animation APIs and realized that there's nothing stopping you from having this capability already. The flexibility of the Animator APIs allow you to do exactly the kinds of operations you need to get curved motion. You just need a little more code to do it.

I thought it would help to post some of my prototype code to show you how. In particular, I thought this sample was interesting because it shows:

• How to move things along curves and complex paths
• How to use the Animator APIs to do more than what the base library supports. In particular, how to use TypeEvaluator to animate custom types.

Some notes and caveats before we begin:

• This is a prototype only, and does not necessarily represent the way it would appear in any final API. It's just a sample program, and a pretty simple one at that.
• Simply computing the location based on the fraction time elapsed in a curve interval is probably not the motion you want. It will give the mathematically correct motion along that path, but the time spent traveling along any particular length of that curve is dependent on the structure of the curve. Basically, you'll end up with slower and faster portions of the curve. This problem is admirably described on this blog. A more complete solution flattens the curve and ensures uniform speed. But again, it's just a simple demo, so I'll leave correct path-time-distance navigation as an exercise for the reader (and for the writer, since this would be a part of any future implementation in the SDK).
• The timing of the animation along a multiple-point path such as the one in the demo app is not as flexible as I'd like it to be. Basically, you end up with something that gives equal time in the overall animation to each individual interval. In addition, any "move" operations in the middle of the path cause the animation to wait at that location for that interval's equal slice of the duration. It should be possible, in a more complete implementation, to define the time associated with any particular interval.
• This description assumes a foreknowledge of Bézier curves; if you have no idea what I'm talking about, you might want to go look them up (such as on the blog linked above or various other places on the web, such as Wikipedia). Or you can just read along, refer to the mathematically imprecise sketch to the right, and hopefully not get too lost.
• The code as written requires Android 4.0 to build. Actually, it's mostly compatible back to 3.0, but the PathEvaluator class uses a generic specification for TypeEvaluator that was introduced in 4.0 (not necessary, just convenient when I wrote the code).

On with the code.

The activity code is in PathAnimationActivity.onCreate(). First, we set up the path itself:

AnimatorPath path = new AnimatorPath();
    path.moveTo(0, 0);
    path.lineTo(0, 300);
    path.curveTo(100, 0, 300, 900, 400, 500);

Here, we are constructing an AnimatorPath (which is part of the demo project that we'll see below) and supplying it with operations that will become points in the path, along with the operations to navigate the intervals up to those points. The first operation defines where the path starts, (0, 0). Then we move in a straight line to (0, 300). Finally, we move along a curve (a cubic Bézier curve, to be precise) to the point (400, 500), using control points (100, 0) and (300, 900) along the way.

Next, we set up an ObjectAnimator to animate this path:

    final ObjectAnimator anim = ObjectAnimator.ofObject(this, "buttonLoc", 
            new PathEvaluator(), path.getPoints().toArray());

This animator uses a new PathEvaluator object (introduced below). It also queries the AnimatorPath object to get an array of PathPoint (covered below) objects; these will become the points in the animation that define the intervals that we animate between. The animator will send the animated values to the this object, which is the activity instance itself. We implement the setter below to receive those values and pass them along to the actual Button object that we want to move on the screen:

    public void setButtonLoc(PathPoint newLoc) {
        mButton.setTranslationX(newLoc.mX);
        mButton.setTranslationY(newLoc.mY);
    }

The AnimatorPath class referred to above stores information about the overall path. Its API consists of everything seen above:

public void moveTo(float x, float y);
    public void lineTo(float x, float y);
    public void curveTo(float c0X, float c0Y, float c1X, float c1Y, float x, float y);
    public Collection getPoints();

Internally, AnimatorPath uses PathPoint to store the information at each point along the path. The PathPoint class is a simple data structure that just holds an x/y location, optional control point information (for curves), and the operation that tells the path containing that path point how to nagivate the interval leading up to that point. There are three factory methods that AnimatorPath uses to construct PathPoints as its API is called:

    public static PathPoint moveTo(float x, float y);
    public static PathPoint lineTo(float x, float y);
    public static PathPoint curveTo(float c0X, float c0Y, float c1X, float c1Y, float x, float y);

All of the logic of actually animating between points along the path (besides that in the Android Animator engine itself) is in the class PathEvaluator. This class implements the single method in the TypeEvaluator interface, evaluate():

    public PathPoint evaluate(float t, PathPoint startValue, PathPoint endValue) {...}

The return value of evaluator() depends on the operation described by the endValue PathPoint object.

For curves, we calculate the x/y values given the anchor points (the location at startValue and endValue) and control points (both control points are stored in the endValue structure).

    if (endValue.mOperation == PathPoint.CURVE) {
        float oneMinusT = 1 - t;
        x = oneMinusT * oneMinusT * oneMinusT * startValue.mX +
                3 * oneMinusT * oneMinusT * t * endValue.mControl0X +
                3 * oneMinusT * t * t * endValue.mControl1X +
                t * t * t * endValue.mX;
        y = oneMinusT * oneMinusT * oneMinusT * startValue.mY +
                3 * oneMinusT * oneMinusT * t * endValue.mControl0Y +
                3 * oneMinusT * t * t * endValue.mControl1Y +
                t * t * t * endValue.mY;
    }

For lines, we perform a simple linear interpolation between the start and end points:

    else if (endValue.mOperation == PathPoint.LINE) {
        x = startValue.mX + t * (endValue.mX - startValue.mX);
        y = startValue.mY + t * (endValue.mY - startValue.mY);
    }

For moves, we simply jump to the location defined by the end value:

    else {
        x = endValue.mX;
        y = endValue.mY;
    }

Finally, we create a new PathPoint with this calculated xy information, which will be passed to the setButtonLoc() method seen above:

    return PathPoint.moveTo(x, y);

... and that's it. There's really not much to it, which was the whole point of posting this code. If you want to add curved motion to your animations, you can wait for the framework to do it for you... or you can take what I've done here and modify it to suit your needs. Yay, TypeEvaluator!

I've posted the code on code.google.com; check it out for yourself. Or you can download the Eclipse project (including the pre-built apk) here.