MIDP 2 Performance Frequency Distribution

Background
Previously, I looked at performance percentiles for MIDP 2 handsets over time and said I would provide an analysis of performance by handset vendor. Instead, I will stay at a higher level and look at some more descriptive statistics regarding MIDP 2 performance (JB2 scores) instead over the period January 2004 through July 2007.

The Analysis
The figure below shows the frequency distribution and cumulative frequency distribution of 609 handset JB2 scores.

The cumulative frequency curve approximates the percentile plot I presented in a prior post.

The distribution of JB2 scores has a strong negative skew (γ₁ = 1.07795). That is, more data is concentrated in the left tail than would be expected in a normally distributed data set. The distribution is also 'peaky' with kurtosis (γ₂ = 0.43981). This tells us that, as developers, we should be looking at the median rather than the arithmetic mean when making a decision regarding market share for our MIDlet.

In the JB2 data set, the arithmetic mean of 173.15 JB2 score is significantly greater than the median of 130. If we had naively targeted a score of 173 and expected a 50% market share, we would actually achieve a market share of approximately 35%. (Bear in mind also that the original Motorola RAZR has a JB2 score of only 49.)

Ranking of Release Date and Performance
In order to better visualise just how poor the relationship between release date and MIDP 2 performance is, the rank of the handset release date is plotted against the rank of the corresponding JB2 score. Ranks are normalised in the range 0 through 1.

This plot is another simple way of showing that there is no relationship between handset performance as measured by the JB2 benchmark and release date. The data set has a Kendall tau rank correlation coefficient of τ = 0.196 indicating a poor correlation.

Summary
What this and my previous posts tell us is that as developers we cannot build our mobile application with an expectation that handset performance is doubling every year. If we assume that the 609 handsets with a JBenchmark 2 score that were used in this analysis are representative of the entire population of MIDP 2 terminals, then we must continue to target low performance terminals.

As I said previously, this analysis is simplistic because it does not take into account market share, memory and other considerations. I will publish an analysis of these factors in coming weeks.