Eyes on the iPhone 4: Is the iPhone's screen resolution superhuman?

Apple introduced what it called the "Retina Display" when it unveiled the iPhone 4, which is a high resolution screen of 960x640 pixels packed into a 3.5 inch screen. Other phones, such as the Nexus One, have increased pixel count by using what is known as a PenTile configuration of the pixel on the display, where each pixel is either a Red/Green unit or a Blue/Green unit. Apple instead used what we could consider the "normal" definition of a pixel that contains three colors but reduced the size of each pixel to a fourth of what is was on all the previous iPhone models, which is not an easy feat.

While this increase in resolution is certainly welcome, Apple decided to add a little bit of marketing buzz by claiming that the resolution of the display "is so high that the human eye is unable to distinguish individual pixels". The president of DisplayMate, Raymond Soneira, sent out an email soon after the iPhone unveiling to a variety of news outlets saying that our resolving power is directly related to our distance from the object, which is in part due to the fact that our eyes have the ability to slightly alter in shape in order to accommodate different optical scenarios. Additionally, he stated that at the Apple recommended 12 inch distance that the iPhone's pixels would be distinguishable. A slew of bloggers have since commented on this issue, either blaming Apple for false advertisement or praising the screen because it is damn gorgeous anyway.

What I have found interesting is how one measures the resolving power of the eye, which most people have for some reason neglected in talking about our resolving power. The human eye is a complex system that has the ability to adapt to several situations, most notably to the different intensities of light by altering pupil size and to distance by altering shape. There is also the question of how the visual cortex in the back of the brain processes this information, and whether it makes assumptions and calculations on what we are perceiving. We currently have no way of calculating what kind of data processing is done on the light that hits the eye.

This visual computation problem, however, have never been a reason to stop scientists from using a laser to prod an object. A paper published by a researcher named Guirao in January of 1999 describes the most popular method of measuring the performance of the human eye: by shooting a laser into the eye and seeing what happens (see the picture below). This figure is displaying two techniques at the same time. The first is that a low power green laser (5mW at 543nm) is directed at the eye, bounces back out, and the result is captured by a camera. The motion of the eye is also tracked with an IR LED so that the scientists know that the laser is going into the pupil (which has been artificially dilated) and not just bouncing off the iris. I am still curious how they managed to get sixty people to agree to be in their experiment, especially their "group c" was made up of people who are 60-70 years old.

Ow. Apparently there is "minimal discomfort" according to Santamaría 1987.

The image processing of the laser light that comes back from the eye is where most of the magic occurs. The laser light focused onto the eye has known properties such as the diameter of the beam. By the time this light hits the camera, it has been distorted. Scientists then use what is called the modulation transfer function to measure the change, which roughly comes out to how much of the sharp shape of the beam is lost by the time it hits the camera. A significantly detailed paper on the mathematics behind this is described by the Santamaría reference below.

The data that comes relates how sharp the resulting image is in relation to what is called cycles per degree, which is just a measure of resolution that can be though of as similar to a washboard in pattern, where one bump plus one dip make a cycle. The lower the sharpness, the less one can see the difference between one cycle and the next. Then the researchers choose a sharpness value that is too low for the eye to distinguish to calculate the highest resolution we can see. This procedure has been performed on a variety of people and at a variety of pupil sizes, which has most notably found that old people have poor vision (shocker).

So are Apple's claims correct on not being able to see the individual pixels of the iPhone (at 12 inches)? It very much depends on the person. People older than 60 will not be able to see pixels even after they hold the phone right up to their face. Young people between 20 and 30 might be able to distinguish it if they have had great eyesight all their lives and have no other problems such as keratometic astigmatism, amblyopia, any retinal or ocular disease, diabetes or nervous system disorders, or have previously had surgery on their eyeballs. On top of that, these studies still do not take into account motion (I dare you not to move your hand while holding your phone), stereoscopic vision, or how the visual cortex processes such fine grained information. So Apple's claim appears to be valid except for a very, very small number of people who are probably between the ages of ten and fifteen. While I think it would be ridiculous to have an iPhone at this age, you might hear high pitched squeaking as the younger masses complain that their iPhones suck because they can see that Pikachu is actually made up of a bunch of blocks.



References:
Campbell and Green. Optical and retinal factors affecting visual resolution. The Journal of Physiology (1965)

Guirao et al. Average optical performance of the human eye as a function of age in a normal population. Investigative Ophthalmology & Visual Science (1999)

Santamaría et al. Determination of the point-spread function of human eyes using a hybrid optical-digital method. Journal of the Optical Society … (1987)

The Race to the Perfect Phone

The mobile phone is constantly advancing. The big software players in the tech industry seem to agree that the phone has become the most essential iteration of the personal computer. It's a behemoth opportunity to make an operating system platform that has the same mass-market adoption as Windows does for PCs. Each company has their champion: Apple has iPhone OS, Google has Android, Microsoft has Windows Phone, HP-Palm has WebOS, etc. Each operating system goes a long way to represent the fundamental ideology of its creator, and its creator's vision for the future of mobile computing.

They're all pursuing different routes to achieve market dominance, but the problem is that none of them are doing it quite right. In the last twelve months, I've had the opportunity to play with each of the major contenders in the race to the perfect smartphone, and I haven't found a clear winner. Here's a breakdown of what I liked and didn't like about each platform.

Blackberry OS

This is where I started my investigation a year ago. I found Blackberry hardware to be superb; an excellent keyboard and lots of extra hardware features. The OS is another story. I liked its straight-forward layout and capable messaging tools, but it often felt like an ambitious cell phone- - not always a smartphone. It had some of the ideas of a pocket computer: advanced customizable options, downloadable apps, a web browser, and even some rudimentary file management. But each asset felt like a poor add-on to a regular cell phone. The advanced settings were buried in an illogical network of menus. The apps lacked diversity and quality consistency. The web browser was awful. What it all revealed was a company that had a great understanding of phones, but a poor understanding of software. The race to the perfect phone has become a software race, and though Blackberry is leading the popularity race today, they're going to lose it if they continue on this gradual, add-things-as-we-go approach to their OS.

Microsoft's Windows Mobile 6.5

In some ways, this is a really underrated software experience. Considering Windows Mobile 6 is ten years old, it's amazing how much of it still holds up as a full-featured smartphone experience. High customization, decent software support, and still the best information management - assuming you use Outlook on your PC. The problem is that in those ten years, it didn't make enough progress - and now it feels incredibly dated.Windows Mobile was conceived as a stylus-operated platform. No part of the OS feels friendly to fingers, and not even HTC's impressive user-interface additions on my Imagio phone could fully solve that problem - it was incredibly frustrating to suddenly discover a menu that I could really only access with the stylus. The other major problem is that as a smartphone competitor, WM6.5 really doesn't integrate the internet very well into its experience: certain apps, social networking feeds, and web browsing were either poorly-implemented or simply unavailable to Windows Mobile. 

Palm's WebOS

This could have been a big deal. The initial demonstration of the WebOS platform was praised by journalists; Palm had created a beautiful, clean OS that took everything we liked about iPhones and gave it many of the things people still wanted: multitasking, less restrictive software support, and a physical keyboard. The whole 'cards' paradigm for multitasking apps in WebOS is brilliant - in my opinion, it's still the best multitasking solution of any smartphone. There were two problems that killed Palm and WebOS: Palm didn't have enough clout to seduce app developers to write apps, and the flagship Palm Pre phone had a horrible marketing strategy involving confusing television ads starring a creepy pale girl being worshiped by hordes of orange-clad Asians. No joke.

Google's Android

Android is doing so many things right that many consumers have placed their bet on Google winning this race. Google believes they can put out a very open operating system that each manufacturer can customize and tailor to suit their specific desires. Unsurprisingly, I found the 'plain' Android to be pretty...ugly. The fonts have an unprofessional Linux-distro feel, there's no color coordination anywhere in the user-interface, and the app organization sometimes felt cluttered to me.  Some manufacturers like HTC have shown the merit in Google's vision. HTC makes excellent phone hardware, and they also have great software developers who crafted the beautiful "Sense" UI - an advanced graphical skin on top of Android that offers a blend of unique functions and aesthetic improvements. The problem is that for every HTC, there's a Sony-Ericsson or LG -- manufacturers that just haven't figured out why it's important to take this extra step in the software experience. This is what could kill Android: it places far too much faith in the manufacturers of the phone hardware. Inthe past, manufacturers haven't had to worry as much about providing their own high-quality software on phones. With Android, they're suddenly all but required to dabble in software, and many are failing to provide that measure of competency. This is creating a splintering effect among Android phones, where some experiences are undeniably better-supported than others; it could be extremely frustrating to buy an Android phone, and then immediately see a sexier souped-up version of Android on a different phone, and then find out that Google's latest Android "Froyo" OS update will not be available on EITHER of these phones.

Apple's iPhone OS

This is a tricky one. I wanted to completely hate Apple's flagship device. But I have to admit, there are many things that Apple has figured out that other developers are still stumbling over. Simply put, it answered my desire to carry just one pocket device better than any other phone. It easily provides the best experience for importing music and photos to your phone. The web browser is pretty impressive, and the sheer breadth of the app catalog is astounding; the idea that I could be carrying a respectable port of Street Fighter IV on my phone is unbelievable. Most importantly, there's a certain visual logic and navigational cohesion throughout the entire operating system that makes even complicated tasks much simpler to perform than on other competitors; for example, conducting a three-way conference call was incredibly easy. BUT there are still some serious gripes I have with the iPhone. It needs needs needs multitasking. The camera features needs to be expanded immensely. Battery life could stand to be improved. The GPS navigation features weren't impressive. On a more personal note, taking it out in public made a false statement about my consumer habits that I didn't appreciate.

OS platforms to watch...
Like I said, there's no clear winner yet. And it's only going to get more hazy as the current OS platforms are updated and new competitors enter the race.

Windows Phone 7: Microsoft went back to the drawing board and came up with a total reboot of their vision for mobile phones. Windows Phone 7 is one to watch because it's being leveraged forced into the market by the richest software company on the planet - and it actually has some pretty innovative design philosophy. In particular, where most of the other competitors are organizing their interfaces with pages and pages of apps, WP7 is all about organizing the stuff you want from those apps into more manageable panoramic "hubs" of content. It's also got Xbox Live. Still, WP7 is pretty late to the party and that could hurt its adoption.

Android "Gingerbread" Update: We haven't heard much about it yet, but like I said: Android's doing a lot of things right.


Blackberry OS 6: Blackberry is at the top of smartphone sales, and although I found its current OS to be pretty weak, there's something to be said about a company that has earned the trust of both consumers and corporate users. Their upcoming OS 6 doesn't need to outperform any of the other competitors; it just needs to add enough to keep current users loyal to the brand.

iPhone OS 4: This could be a big deal, as it answers the multitasking problem in a pretty innovative way - assuming it works. It's also adding some other hefty features, including a social gaming network and some alleged video-chat functionality. Other companies should watch these improvements closely.