This article was reposted here after I saw it on http://riaoo.com/?p=1658; the original content comes from http://blogs.adobe.com/xwlin/2010/04/flash_player_101_-_adobe_max_2009.html and http://blogs.adobe.com/xwlin/2010/04/flash_player_101_-_adobe_max_2009_1.html
Speaker: Lee Thomason (lthomaso@adobe.com) Translator: Lin Xiaowei (xwlin@adobe.com)
Preface
Time: the 21st of this month. Location: Beijing International Convention Center. Adobe is about to hold a two-day Flash Platform technology summit. I’ll have the honor of joining Ely Greenfield as co-speaker for the <
The 2009 Adobe MAX still has a Flash Player Internals session, but the speaker is Lee Thomason (another hardcore veteran — Flash Player architect, nicknamed Player Guy, said to be the number one Flash Player guy). This is a very in-depth topic, and the talk moves fast: in one hour he covers as many aspects of the player architecture as he possibly can. Here I’ve summarized and translated the gist of his talk for your reference.
Building Flash Player
In the past, mobile FlashLite and desktop Flash Player were two completely separate products, each developed and maintained by its own independent team. The FlashLite team had its own goals, product positioning and feature set, and decided its own project roadmap. This created a lot of tension and challenges between the two teams. So Adobe decided to merge the two teams — which means the upcoming Flash Player 10.1 for mobile and desktop will be built from the same codebase.
At MAX, Lee mentioned that roughly 20% of the Flash Player code is still platform-specific, code that special-cases Palm, ActiveX, OS X, Netscape and so on. There are so many differences between platforms that a 100% shared codebase is impossible, but that still means 80% of the Flash Player codebase is platform-independent.
One codebase, more player types.
A unified player codebase makes it easier for Adobe to set a consistent strategy across mobile, browser, desktop and other players, but it also brings new challenges for the Flash Player team and the developer community.
The first challenge is the processing power (mainly CPU) and memory size across platforms, which differ wildly and the gap keeps widening. An 8-core server CPU and a handheld Palm Pre have vastly different processing power, and a PC browser platform can offer several GB of memory while a handheld has only 20 MB — which pushes developers to build apps in completely different ways. Note: we’re not saying you don’t need to worry about memory usage on a PC browser, only that it matters far more on a handheld.
The second issue is the variation in screen size and pixel density that a unified UI has to deal with. Today, more and more applications run across devices, so it’s increasingly important to understand how to lay out pixels on a limited screen and how to use them efficiently. An LCD monitor’s pixel density (pixels per inch) is far lower than a handheld’s, which means a 40-pixel button looks big on your LCD but is too small to use on a Pre. Likewise, even two handhelds with the same resolution can have different pixel densities: the iPhone 3GS has a 3.5-inch screen at 320*480 and 163PPI, while the Palm Pre has the same resolution on a 3.1-inch screen at 186PPI. So if we design our UI purely by resolution, two buttons that the player considers the same size may work fine on the iPhone but be unusable on the Pre.
Another challenge comes from the variety of user interaction methods supported by different devices — some devices support keyboard and mouse, some support multitouch, and so on. Flash Player needs to understand different input devices, and we as developers need to understand what that diversity means for the apps we build. When building apps for multitouch devices, we need to remember that a finger is not a pixel-precise input device. The player has to give us the most accurate information possible, from the lowest-level raw data up to high-level user actions like swipes and taps.
The last challenge of unifying on a single strategy is that user expectations for an app change from device to device. Expectations around performance and usability shift depending on whether the user is currently on a phone or a desktop browser. On a phone, the moment a call comes in, the current app has to shut down within 10 milliseconds; if the player can’t do that, users will be disappointed.
Flash Player itself is not a magic bullet for every challenge
The only conclusion from all this is that we can’t expect Flash Player to solve every problem; in fact, as developers we need to understand deeply the platform our app ships on, and to know that a design that works well for one user/device combination may not work at all for another. As we start building apps for more and more radically different device platforms, we need to keep these factors in mind.
That’s part one of this topic. In part two we’ll look at the execution model and the player architecture, in part three we’ll discuss what improvements ActionScript’s second-generation virtual machine brings, and in part four we’ll dig into Flash video and the rendering system.
Understanding the execution model
The execution model is how Flash Player executes its instructions within each frame cycle. Behind the scenes Flash Player actually runs any number of threads, it’s just that AS doesn’t expose a multithreaded programming model to developers. That means, conceptually, we have to treat Flash Player as a single-threaded entity. The debate over the pros and cons of this single-threaded programming model never stops; I don’t want to say much about such a controversial issue, but please keep this fact in mind.
The Elastic Racetrack
The elastic racetrack is Flash Player’s frame execution model. It describes how code execution and frame rendering balance against each other within a single frame’s processing cycle. Flash Player 9 and AVM2 made some improvements to this model; this information is based on research into the event mechanism and the rendering model, and the complete model hasn’t been officially published.
The basic racetrack theory hasn’t changed: during the cycle in which Flash Player executes a frame, the first part of the time is used to execute code and the rest is used to render the objects in the display list. Each execution phase can extend its execution time as needed to run more code or do more rendering work, and the total length of the racetrack grows accordingly.
What changed from the previous model is what each phase looks like within a micro-cycle and how they add up to a frame.
AVM2 is driven by a marshal-level component in Flash Player called Marshal. Marshal is responsible for slicing time into the basic time slices that Flash Player works on. Here I want to clarify that Flash Player’s time slices have nothing to do with the frame rate at which a swf file runs; we’ll ultimately see how Flash Player combines these time slices into a frame. Running a swf compiled by Flex in Firefox on Mac OS, Marshal typically slices time into 19-20 ms slices — the slice size varies by platform and browser. For the sake of our discussion, let’s assume a slice size of 20 ms, meaning Marshal produces no more than 50 slices per second. Within each slice, five possible operations are performed in the following order:
- 1.Player event dispatch - for example Timer events, mouse events, ENTER_FRAME events, URLLoader events and so on.
- 2.User code execution - all code listening for the corresponding events from the previous step is executed.
- 3.RENDER event dispatch - calling stage.invalidate() during user code execution triggers this special event.
- 4.Final user code execution - user code listening for the special event from step 3 is executed at this point.
- 5.Player updates the display list.
Marshal repeatedly executes 20 ms slices like this, deciding what to do next as it goes. All the operations performed within a slice ultimately boil down to the two-section racetrack described above (code execution, image rendering), which is what we call a frame. User code and invalidate operations fill the code execution section, and render operations fill the rendering section of the racetrack. It’s worth pointing out that these operations can only happen within the time Marshal has allocated; if your user code is short, Marshal will still wait a while after the user code finishes before moving into the rendering phase.
To better illustrate which actions are executed how and how the elastic racetrack is created, see the example below, which describes how slices are handled in swfs running at 5fps, 25fps and 50fps respectively.
From the example above you can see that at different frame rates, the elastic racetrack in a frame cycle performs different operations — for a 5fps swf, each frame handles 10 user actions, 1 invalidate action and 1 render action; for a 25fps swf, each frame handles 2 user actions, 1 invalidate action and 1 render action; for a 50fps swf, each frame can only handle 1 user action, 1 invalidate action and 1 render action. A very important point to note is that certain events can only occur in certain specific slices — for example, the Event.ENTER_FRAME event can only be dispatched in the initial slice of a frame.
- Either the code portion or the rendering portion of a slice can run too long, making that slice longer than 20 ms — that’s what “elastic” means. To keep the playback rate close to the frame rate set when the swf was compiled, Marshal will choose to drop some slices.
- A swf file’s playback rate can’t exceed the rate at which the current Flash Player slices time. You can set a swf file to play at 120fps, but Flash can play it at most at 50 frames per second (the exact number depends on the current system settings).
- Code can be executed more often than the swf’s frame rate. Play a 1fps swf file: one frame takes 1 second, which is 50 slices, but in every slice mouse or timer events will fire, although only the last slice will render. You can also choose to render early by calling updateAfterEvent(), but only from within mouse, timer and keyboard event handlers. In that case, Marshal considers the current frame finished and moves into the next frame starting from the next slice. Finally, if a Sprite’s visual properties such as width, height change while the mouse passes over that Sprite, Flash will force a render.
- If the frame rate is not an integer factor of the number of slices per second, the rendering interval on that platform becomes irregular. For example, a swf with a 20fps frame rate running on a system with 50 slices per second: Flash Player’s behavior will be to play two frames every 5 slices, so the swf’s rendering frequency will be 2-3-2-3-2-3 (slices).
(This article may be updated as the original author adds more content.)