Top Android Smartphones Tested From HTC, LG, Samsung Electronics, Xiaomi
Over the course of my years in the IT industry, I have seen the ethos around device experience and performance as ever-changing. One thing always seems to crop back up every time we see a new compute platform arise as the primary platform whether it be workstations, personal computers, tablets, or smartphones. In the past few years, that has been squarely centered on mobile and mobile performance and how it’s quantified and advertised to consumers. This has ultimately led to the creation of the “8 core myth”, which may become the “10 core myth” if more SoC vendors continue to go down this path of more CPU cores. Ultimately, a smartphone is a heterogeneous compute platform with many different types of cores specialized for different purposes and for anyone to focus on one type and to market those as the measure of performance is misleading. We recently performed a live benchmarking video session on YouTube and wrote a 10 page research paper on the tests. We looked at smartphones from HTC , LG, Samsung Electronics and Xiaomi containing technology from ARM Holdings , MediaTek, Qualcomm, and Samsung Electronics.
These potentially misleading practices reached their peak around the time of Mobile World Congress, where virtually every smartphone manufacturer was touting having 8, 64-bit cores. I followed this up with a conversation about why having 8 and 10 cores in smartphones is such a bad idea. This was supported byan experiential review that I did of the LG G4, which is actually only a 6 core phone, which would lead many, based on today’s 8 core marketing that it would not keep up with other phones but it did and does. The LG G4 is the company’s flagship phone which competes easily with their competitors’ 8 core devices without breaking a sweat. After doing that, we started to realize that benchmarks are partly the culprits of the too many core madness and I addressed the problems with benchmarks and those that use them in facilitating this potentially misleading marketing. This led me to realize that we need to show all of these things together and that led us to create an idea to do a video and paper about the myth about CPU core counts in smartphones and how we will show that they are not relevant in the benchmarks and applications that matter most.
We at Moor Insights and Strategy recently did a live benchmarking video session on YouTube and Periscope to illustrate the problems that plague the smartphone industry today. We took some of the leading smartphones from some of the biggest smartphone manufacturers in the world and tested them against each other while disabling CPU cores to show how having so many cores in many cases does not affect overall system performance very much, if at all. We used leading industry benchmarks from Futuremark, Basemark and Sofica and combined those with real world application testing to show how little the performance was affected by having fewer than 8 or even 6 cores.
Since this live benchmarking session ended up being over an hour long and didn’t fully explain everything that was going on in our testing, we have also followed that up with a 10 page paper that explains exactly what benchmarks we ran, why we ran them, how they were run and what our results were in a fairly detailed manner.
If you’d like to read the paper and see our detailed results, feel free to click here to download it.
We hope that this analysis could not only positively change the behavior of carriers, retailers and reviewers, but also handset makers HTC, LG, Samsung Electronics and Xiaomi, and technology providers ARM Holdings, Intel, MediaTek, NVIDIA, Qualcomm, and Samsung Electronics.