Yet more checkerboarding analysis

I’ve been spending a bit more time on refining the checkerboarding tests in Eideticker that I talked about last time. Most of my work has been focused on making the results as representative of a real world scenario as possible, to that effect I’ve been working on:

  • Changed the test case from a web site of my own concoction to a more realistic example (the taskjs.org site)
  • Use actual Android native events (via MonkeyRunner) to synthesize touch-based scrolling instead of simulating the event in JavaScript (which exercises a completely different codepath).
  • Fixing various synchronization issues to make results more repeatable. Before captures were of wildly variable lengths, which made the numbers extremely suspect. There’s probably still a few issues, but much less than before.

The end result of this is a framework that gives much more meaningful results. The bad news is that the results that I’m measuring don’t show a very positive picture for where we’re at with the native re-write of Firefox. Even relative to the version of mobile Firefox which is currently on the Android Market, we still have some catching up to do. Here’s some video of the “old” firefox in action:

And here’s the Native fennec (what we’re currently offering in nightly, with some minor modifications by me to change the way the “checkerboard” is drawn for analysis purposes):

The numbers behind this comparison:

Platform Percent checkerboarding over run of test
Old Fennec 2%
Native Fennec 57%

(by the way, this performance regression is filed as bug 719447)

I know there’s lots of great effort going into improving this situation, so I have hope that we’ll be doing much better on this metric in the coming days/weeks. The process for creating these videos/analyses is mostly automated at this point, so my plan is to create a small dashboard (ala arewefastyet.com) to measure these numbers over time on the latest nightlies. Stay tuned!

3 thoughts on “Yet more checkerboarding analysis”

  1. The font size seems different in the two videos. Wouldn’t the longer page cause a larger “percentage of checkerbording”?

  2. @AndersH: I hadn’t really thought of that, but yes, I suppose it could depending on how the drawing code inside Fennec works and prioritizes filling in regions. It’s very difficult to get an apples-to-apples comparison between browsers because of these sorts of differences. For example, my current automation currently generates 5 “swipe down” events to make sure we scroll right to the bottom of the page, even though only 2 are required to scroll to the bottom in the original XUL fennec.

    I expect that, much like Talos, Eideticker will be most useful for comparing Fennec nightlies against each other as we add individual features.

  3. I’m glad you guys are tracking this — after recently switching from an original Droid 1 to the galaxy nexus, I was surprised that there wasn’t really any improvement in the checkerboarding!

    That & the weird redraw flash when navigating to a new page make the whole browser feel unpolished. Hope they don’t prove too tricky to fix for the next release. :)

Comments are closed.