Windows Phone Performance - Part 5 of many: A Developer’s look at the Nokia Lumia 610

The plan for part 5 was to continue my WP Performance series with a discussion of List performance. However – as it usually happens - Life has came and changed the plan. First, Oren Nachman published a great article on The Windows Phone Developer Blog called “Listbox, Why Art Thou Blanking”, and I don't really have too much to add to what he wrote. But also, I happened to get a Nokia Lumia 610 phone to test some of the applications we are working on. The 610 (and other, so-called Tango devices) will have a big impact on the lives of Windows Phone developers, but most of have never seen one first hand. I cannot give you one, but I am trying to do the next best thing: I will share experiences with the device and the results of some measurements that I performed so that you can get a feel for the performance characteristics.

The Nokia Lumia 610

The Lumia 610 is the first “Tango” device. Tango is the codename for the Windows Phone 7.5 Refresh (build 7.10.8773.98), but you will not catch a Microsoftee call it Tango officially. As an OS update, Tango only brings a handful of small features to Windows Phone. The big change is that Tango has been optimized for phones that have as little as 256 MBytes of RAM available. And that is exactly what the Lumia 610 has. The other notable difference (from a developer’s point of view) is that the 610 has an 800 MHz CPU (the first generation devices had 1GHz, and most of the second generation phones have a 1.4-1.5 GHz CPU).

The Nokia Lumia 610

The Lumia 610 is a very good-looking device – especially the black-white combination that you can see above. It looks better than the much more capable Lumia 710. The Tango OS also performs nicely – you can barely say that the OS runs on a low-specs device, as long as you are using the native features of the operating system. The looks and low price, great in-store performance (where you mostly play with the OS) will probably make it a hit among those who are looking to enter the world of smartphones.

Things change dramatically when you start third party applications. At this point, you will notice that some applications take forever to load, and for more complicated apps, you will experience glitchy scrollings, blankings, slow response time and even occasional freezes for a second.

Application Loading / CPU performance

In the second part of this series (It's Full of Stars!), I dealt with application startup times. Let’s see how three phones measure up in this test: the Lumia 610, the LG Optimus 7 and the Lumia 800.

I tested the star drawing app on all devices. The LG Optimus 7 is a decent first generation Windows Phone, with 1GHz CPU. The Lumia 800 is a second generation device, with 1.4GHz CPU. Both of these have 512 MBytes of RAM. As I mentioned before, the Lumia 610 has an 800 MHz CPU with 256Mbyte RAM. Just for kicks, I also included the values measured on the Emulator.

The first column of the table shows how many stars (small ellipses) are drawn, and the other columns show the number of milliseconds it took to launch the application. The stars are drawn on the UI thread at startup, so this drawing delays the startup time of the app.

Silverlight Layout and Ellipse rendering performance on the Nokia Lumia 610, Nokia Lumia 800 and the LG Optimus 7.

The results are surprising, but clearly show how efficient the CPU in the Lumia 610 is. Despite the 20% lower CPU clock, the Lumia 610 could easily match the performance of the LG Optimus 7. It could even draw 20,000 stars, which the Optimus 7 couldn’t finish in 20 seconds, so the OS killed the app!

This is of course by no means a comprehensive test of the CPU. But it is a fairly good indicator on what you can expect from the Lumia 610 in Silverlight – the test app exercises the layout and rendering subsystems of Silverlight, which are key to any application.

GPU Performance

To measure the raw CPU performance, I used the “Fill Rate Test Sample” from the MSDN article Performance Considerations in Applications for Windows Phone. This does not test things like shaders, perspective transformations, etc – but it is a good first indicator. I gradually increased the number of rectangles on the screen, which in turn increased the fill rate. As we saw in part 4 of this series, The Power of the GPU, Fill Rate is an important measurement for how much work the GPU has to do, and a high Fill Rate makes the FPS (frame per seconds) drop. Without further ado, here are the results in table and chart format:

Fill Rate vs FPS on the Nokia Lumia 610, Nokia Lumia 800 and the LG Optimus 7

As you can see, the Lumia 610 not just matches, but seriously outperforms the first generation Optimus 7, delivering almost twice as many frames per second under heavy load! The Lumia 800 has 75% higher CPU and GPU speed than the 610, and it performs accordingly.

The FPS starts to drop at the Fill Rate of only 2.5 for the Optimus 7 (which is tuned to a maximum frame rate of 50 instead of the usual 60). At this point, the list scrolling becomes somewhat choppy – it begins to jump a little bit every second or so, while remaining completely smooth in between. This is a very low value – it barely gives you enough space to put a background image and a full screen of information on top of it. E.g. if you have a Panorama app where you want to put a semi-transparent background behind your content, you are already at the edge. The first cutoff points are much further for the 610 – at a Fill Rate of 4. This means that the 610 can manipulate 1.5 more screens worth of pixels without any visible glitch. Of course, the Lumia 800 wins here again, which can almost go up until a Fill Rate of 8 before the first signs of strain start to show.

The pain point (where the frame rate drops below 30 FPS) comes way too early on the Optimus 7 – at a Fill Rate of 5, we are already in the red zone. The 610 can handle 70% more pixels before dropping below 30 FPS, at a Fill Rate of 8.5, while the 800 can soar up to a Fill Rate of 15.

Memory

So, the Lumia 610 CPU is on par with first generation devices, and the GPU is much better. Why did I say then that the performance of the Lumia 610 is not perfect? It all comes down to lack of memory.

Microsoft has done an awesome job shoehorning the Windows Phone 7.5 Mango OS into 256Mbytes. The only feature they had to remove are the background tasks, and some media codecs (see the details at Handle Feature Reductions). But while apps on 512Mbyte phones have a lot of memory to play with, apps on 256 MByte phones only get 60 MBytes. Tango improved virtual memory (paging), which means that your application still can use up to 90 MBytes of memory, only the data above 60 MBytes gets swapped out. When this happens between two screens, all the user sees is some delay between the screens. But when it happens while you are still on one screen, the app seems to freeze and does not react to user input. This usually takes about a second, which is more than enough to cause frustration in the user.

There are some great posts by Mike Battista on Optimizing Apps for Lower Cost Devices that cover what you can do to help this situation. You can get to a fairly decently performing app following his directions. However, if you are using things like the WebBrowserControl or the BingMaps control (and can’t replace them with a call to the appropriate OS task), you are out of luck. No matter what you do, these controls take up too much memory. For example, a simple app only with only a WebBrowserControl can easily consume as much as 50-52 MBytes of RAM, which doesn’t leave much space until the swapping starts to happen.

Lumia 610 loading a complicated web page in a simple app, occupying 56 MBytes of RAM (which later goes down to 42) out of the available 60.

To see what the low memory means for a more complex app that uses the WebBrowserControl, check out this video. Note, that I do not load anything beyond a basic text file into the browser – if I did, the 610 would load up unbearably slow.

Summary

The Lumia 610 is a surprisingly powerful device that easily matches the first generation Windows Phones and even outperforms them when it comes to graphics. However, the lack of memory kills the joy – a slightly more complicated app can easily find itself being swapped out, resulting in slow app launches, responsivity issues and temporary freezes due to paging. Despite the fairly powerful GPU and CPU, it takes significant developer effort to match the user experience of apps when compared to first generation devices – if at all possible.