Video: Resizing AIR Windows with Flex 4 Effects

Resize Adobe AIR Windows through Custom Flex 4 Effects Interpolation, the next and verbosely entitled episode in the gripping and suspenseful series CodeDependent, is now available from Adobe TV.

In this show, we see how to apply what we learned about arbitrary type interpolation in a previous episode to the specific use case of resizing an AIR window. The ability to animate properties of arbitrary type is one of the key new features in the Flex 4 effects system, and it comes in handy for this use case where animating the x, y, width, and height properties of the native window aren't good enough. Instead, we need to interpolation a Rectangle object so that we can atomically set the bounds of the native window with each animation update.

Here's the video:

Instead of embedding the application like I normally do, I'm just going to provide this link to it. It's a link to an AIR application which you would need to install and run locally, because that's the way that AIR works. You may not care enough to do this, since it's just a live demonstration of what you see in the video. But it's here for completeness.

And here is the source code.

Finally, here's where you can find the CodeDependent videos on iTunes.

Enjoy.

Off the [Bubble]Mark

James Ward has been looking into the BubbleMark application recently, and we were talking about some of the strange results we saw in different situations. For example, I was able to take the existing application, which gets something like 65 frames per second in IE7 on my Vista laptop, recast it as an AIR application, and get about double that frame rate, or 130 fps. James posted some variations of the application on his blog; check them out to see what I mean.

(Some background, for those that didn't avail themselves of the awesome opportunity of clicking on the recent link to the site: BubbleMark is an application that moves a number of 'balls' around in a window, and bounces them off one another while doing so, and then measures how fast this critical and awe-inspiring process happens per second).

What's going on? Is that gremlin in my keyboard back to haunt me? Or have I been staring at the screen too long and I'm mis-reading the numbers? Or maybe age is creeping up on me and I'm just getting things wrong more often, as a prelude to slipping on the ice and breaking my hip.

Circular Reasoning

The fundamental problem with the current test is that it is limited by the resolution (the time between events) of the timing mechanism used by the application. In this case, the timing mechanism is a Flash Timer object that is scheduled to call our code at frequent intervals (supposedly far more frequent than the frame rate we're seeing). So while the application purports to test the number of frames that it can render per second, it actually tests the number of times per second that the Timer sends out events. In the situation where Flex can render the balls much faster than the Timer frame rate, this means that the frames-per-second (fps) reported has very little to do with the speed at which the balls are being moved or rendered.

By producing an alternate test that runs on AIR (which takes Flex out of the browser and runs Flex applications in a standalone window on the desktop), we remove a constraint where the Timer resolution tends to be more limited in the browser than in standalone applications. Since the Timer can call our ball-moving routine more often in this AIR version, and since we were taking less than a frame's worth of time to move the balls before (although that wasn't reflected in the fps result), we get a higher effective frame rate and better results.

So the AIR version is closer to the ideal, where we're actually measuring more of what the test was presumably intended for; moving and rendering the balls.

But there's still a problem here, which has been implied by the wide variation in the results. The test is confusing three orthogonal issues and trying to combine them into one result (fps). The three things actually being tested are:

• Calculation speed: How fast can the runtime position the balls, including moving them, calculating collisions, and offsetting the collided balls? This comes down to VM and math performance, since it's just raw computations.
• Rendering performance: There is a certain amount of time spent each frame actually drawing the balls on the screen. That seems to be insignificant in our case, compared to the other factors here, but you would think that a graphics benchmark might actually care about this item them most. How fast can we draw the balls? I don't know, because we're spending most of our time just waiting around for Timer events and calculating the balls' positions. That's like timing how long it takes you to get dressed in the morning, but starting the timer the previous night when you go to sleep.
• Timer resolution: This is the primary issue called about above. I don't believe it was ever the intention for the benchmark to measure the rate at which the various runtimes could schedule callbacks. Because, well, that'd be a pretty dull graphics benchmark. But that's exactly what's happening here; we're not measuring graphics performance at all in the default test.

To add to the problem of these three factors getting each others' chocolate stuck in the others' peanut butter, you've got the problem that all of the different platforms (Flex, Java, .net, etc.) probably have different reliance on these three factors (are they faster at rendering but slower than computation? Do they have more limited Timer resolution? How do browsers affect these factors on each platform? Or hardware? Or operating systems?)

I think that if you wanted real results that you could use and compare from this benchmark, you'd have to de-incestuate the factors and measure them all separately. I could envision three competely separate tests coming out of this:

• Calculation speed: Let's have a non-graphical test that simply moves/collides the balls as fast as possible and reports how many times it was able to do this per second. No need to render here; we only care about how fast we can do the math and related setting of properties.
• Rendering performance: Render a given number of balls as fast as you can and report how often you can do it per second. In order to not run up against the Timer resolution issue, you'd have to adjust the number of balls so that you could be assured that you would be spending very little time waiting around between callback events. In fact, you could even try to render far more than could be done in a single frame, but then report a number like balls/second instead of frames/second.
• Timer resolution: Write a test that simply issues callbacks from the Timer mechanism, without doing any work in those callbacks, to see how many times this happened per second. (I wrote this test and posted a blog about it; see that post for more information on this no-op timer test).

With results from these three tests, it would be easier to understand where a platform was being bottlenecked: at the runtime, rendering, or resolution level.

At this point, I'd love to say, "...and here's that benchmark!" But I'm not going to, because I've already spent a pile of time just trying to understand the weird results I was seeing from Bubblemark. If someone else wants to take a whack at it, that'd be great. But at the very least, I hope this helps you understand and filter the results you see from the BubbleMark tests a little better.....

Moment of Reflexion

A comment on my previous posting asked for some explanation of what Flex is and how it relates to Flash.

I'm still intending on posting some technical content here and going over some sample code, but hey - I'm Flexible.

I was sort of assuming that everyone had heard of Flex and knew something about it. But since I'm new to the platform myself, and since I intend this blog to be for relative newcomers to the platform (at least for now, since that's where I'm at), it seems reasonable to define some terms. Here is my understanding of how these various Adobe products and platforms relate to each other. If I get whacked from someone in marketing, I'll replace them with something more correct, but I think this will do for now.

ActionScript

ActionScript is the programming language used to program Flash applications. The current version of the language is ActionScript3, which is based on ECMAScript (read: JavaScript), and should be quite familiar to most web programmers because of its similarity to JavaScript. ActionScript is more type-safe that JavaScript in general (or at least that's my impression from using it in the Flex environment); you need to declare most variables as having specific types at compile time. The base syntax is actually not too unlike Java. I've found the transition easy overall, although I still write my variable declarations backwards out of habit and then go back and fix them when the compiler pukes on my code. For example, the following declaration in Java:

    Object blah;

would be writtin in ActionScript as:

    var blah:Object;

I should write a macro in the IDE to do the switch for me, since I seem to have a hard time twisting my brain around after all this time to do it correctly.

Flash

Flash is a graphical library and runtime that enables animated graphics applications. There is an authoring tool for creating Flash content, a language for programming Flash applications (ActionScript3), a class library of graphics and other useful objects, and a VM that enables Flash applications to perform well. From the start, Flash was about creating animations, and the tools and libraries for Flash all reflect that time-based model; the Stage, MovieClip objects, and frame rates are all part of a normal Flash application.

The rendering in Flash takes place via a "display list" (also known as a scene graph, or retained-mode graphics). Instead of issuing direct rendering calls in your code which get executed immediately (like they do in immediate-mode APIs such as Java 2D), you place rendering requests on a display list which gets processed and displayed which the next frame is rendered. These requests then stay on that display list until removed.

The use of a scene graph is a subtle point, and perhaps one that most application designers couldn't frankly give a toss about. But it ends up being important to Flash and Flex developers, as it's important to understand what is being put on the display list and when that display list needs to be cleared and repopulated with other rendering requests instead.

For example, if you put the following rendering calls into the display list of a Flash object (where graphics is the graphics context of a Flash display object):

    graphics.moveTo(0, 0);
    graphics.lineTo(100, 100);

then you will get a line on the screen from (wait for it...) (0, 0) to (100, 100). Suppose you later want to move the line to be from (50, 50) to (150, 100). If you made these calls to the graphics object:

    graphics.moveTo(50, 50);
    graphics.lineTo(150, 100);

and ran your application, you would see two lines on the screen. You didn't actually change the line's position; instead, you simply added another line to the display list. The typical way to get what you want is to recreate the display list from scratch by clearing it first, and then add in the rendering calls you want:

    graphics.clear();
    graphics.moveTo(50, 50);
    graphics.lineTo(150, 100);

Flex

Flex is a GUI toolkit library that sits on top of the Flash library and runtime, much like Swing sits on top of Java 2D (for the Java folks in the audience). The addition of Flex did a few things to improve (at least for folks like me) the programming experience of Flash applications:

• GUI components: Prior to Flex, most Flash applications brewed their own components (buttons, etc.), which meant more work for the developer, less consistency between applications, and potentially buggy or unpredictable behavior. Who here has run across bizarre implementations of scrollbars in Flash applications that didn't look or behave at all like the scroll bars we know and love? Flex brings uniformity and functionality to this previously, er, more chaotic space.
• Programming model: As I alluded to before, the programming model of Flash is, well, different than most GUI developers might expect. Thinking about an enterprise UI in terms of MovieClips, frames, and display lists is, well, different. Flex abstracts all of that away and allows developers to write GUI applications for Flash in a much more traditional way. A button control is a button control, with events, skins, states, and so on; it's not a MovieClip added to a Stage or anything so unusual (at least to me, but maybe I just have Stage fright).
• Declarative programming: Flex also introduced the MXML language for declarative programming of applications. This XML-based language allows programmers (and design tools) to create static code that tells Flex how the UI should be layed out, and saves the actual dynamic programming logic via ActionScript for things that cannot be declared statically. There are various advantages to this model (from what I've seen so far): clear XML code that sets up the GUI briefly (as opposed to lots of code to do the same task), a logical hierarchy of XML tags that reflect the parentage of GUI containers and components, and interaction with design tools for ongoing creation and editing of the GUI in a WYSIWYG builder (such as the one in FlexBuilder). It's pretty cool to be able to go in and out of design mode, hand-coding stuff where appropriate, and dragging and dropping other things when possible. The fact that the GUI layout is persisted in XML means that the builder tool can handle edits much better than approaches that use actual code behind the scenes (typically, the code is not allowed to be edited or if you do edit it you probably cannot get the tool to read it in again).

Given all of these advantages and advances of Flex, GUI programmers should now find it much easier to write real, full, robust GUI applications that run on the Flash platform.

The output of a Flex build is simply a SWF file (the type of file that the Flash player will run); it ends up just being another Flash application. But inside of that application is the set of Flex components and capabilities used in your application, which then translate into Flash rendering.

The Flex SDK is free and open sourced; you can write Flex applications, compile them, and ship them all for free.

FlexBuilder

This IDE (a plugin for the Eclipse platform) is optional - you can use the Flex SDK for free as a command-line compiler, using whatever IDEs and code editors you want. The FlexBuilder tool does cost money, but gives you some advantages over hand-coded Flex applications, such as the GUI builder I mentioned above as well as all of the code editing features you would expect for both ActionScript and MXML (code hinting, and so on).

AIR

Think of Adobe AIR as the packaging of both web applications (HTML/JavaScript) and Flex/Flash applications for the desktop. Flash was written originally for the browser, and Flash content is predominantly browser-based today. Although you can run SWF files in a standalone SWF player, traditional Flash applications are really intended for use in web browsers. The same goes for Flex applications, since they are essentially Flash applications with more built-in capabilities.

But what if your users want to run your application offline? Or what if they want access to their local filesystem, a feature that browser applications typically don't allow because of security restrictions?

AIR enables all of this; it allows applications to be installed on the local operating system and accessed from the desktop just like the other applications that the desktop user runs. AIR applications can still access online information, in fact that is still an important model for these rich applications, but they can also run offline when appropriate or necessary.

AIR also bundles in more capabilities that are not currently a part of the Flash or Flex platform, such as a full HTML/JavaScript rendering engine. So even if you're a web developer, just writing AJAX applications with HTML and JavaScript, you can use AIR to deploy these applications to your user's local desktop.

BlazeDS, LiveCycle, etc.

I think I'll leave these product terms for another day. There's a host of Flex-related products and capabilities for the back-end. But frankly, I'm still figuring out the stuff on the client, and I'd really rather stick to what I know on this blog....

Know Free Launch

Adobe released a bunch of good stuff yesterday. It's pretty huge. Not mentioning it would be like forgetting to tell your wife you like her new haircut; tactless and just plain dumb.

• Adobe AIR 1.0: Bringing Flex, web programming, and Rich Internet Applications in general to the desktop.
• Flex 3: The latest release of the Flex platform, a powerful GUI toolkit sitting on top of the rich Flash rendering engine.
• The open-sourcing of both the Flex SDK and BlazeDS (back end services for Flex and AIR applications)

Meanwhile, I'm still putting together technical pieces on a Flex app I've written; expect that to start dribbling out here soon. Like the crumbs that multiply on my shirt as I eat my breakfast; at first it's just a couple of crumbs, but pretty soon you've got a whole scone there in its component parts.

Eventually, I want to be able to write pieces that help describe the work we’ve done on Flex, the features we’re working on, the future plans, and the inner details that help people understand how to use it most effectively.

But hey, I just got here; I don’t know any of that stuff. I’ve been developing graphics software for my entire career and hacking the internals of desktop Java for the last several years, and I am very familiar with desktop client technology in general, but the whole stack offered by Flex is completely new to me. I will, I’m sure, eventually learn all of the fun internals, and when I do I’ll execute on the plan above.

But until then, I’ll be coming up to speed on Flex, Flash, AIR, ActionScript3, and everything else I need to be able to help out on the Flex team. And as I do, I thought it would be useful to post interesting things that I discover, applications that I’m working on, and random thoughts on Flex development to hopefully help others on a similar learning curve.

Maybe you’re a Java developer, trying to learn about Flex. Or maybe you’re a Flash developer, interested in Flex’s GUI toolkit. Or maybe you’re new to desktop development and are checking out the various technologies available. Or maybe you did a Google search on “acid reflux”, ended up here and haven’t a clue what I’m talking about.

In any case, stay tuned for future postings here where I’ll talk about some of the things I’m working on and interesting things I find out about the platform as I cruise along.