Apr 19 Adverse Cognitive Effects of Wrong Aspect Ratios

Original post: April 19, 2015 | GoogleScholar | Academia.edu

Overlooked in usability: aspect ratio conversion

Through my many years of working with screens – computer displays, television sets and proprietary video hardware – of varying sizes and intended uses, I find one factor to have a much bigger cognitive role than the innocent context of this usability topic may suggest – aspect ratios. Over 20% of the world is using electronic displays or TV’s with wrong aspect ratio settings, and it turns out that this has deep physiological and psychological effects that affect the users, the product manufacturers and everyone in the advertising sector. Should anyone care about this usability problem? I have done the research to find a few interesting answers.

Background in a Nutshell

It’s paradoxical: a retail store or a sports bar may easily invest over $20,000 into a collection of large high-resolution displays to showcase their products or to entertain patrons, but half of the time the screen is fitted incorrectly into the viewable area of the device, and the images appear distorted, or stretched, resulting in a variety of complex cognitive responses, many of which are subconscious and all of which have a negative effect on the audience’s overall experience. What can we do to address these problems? And why is this important for the consumers?

The Problem is Bigger than Meets the Eye

Every time any picture – whether static or dynamic – fails to conform to the conventional model of our everyday dimensional space, our brain is confused, whether we notice it or not. Those of us who sense that the aspect ratio of an image or video is off (this includes virtually all techies and a significant cross-section of regular Bobs and Thelmas) are “officially” annoyed – e.g., when the baseball doesn’t look perfectly round or when Cindy Crawford appears to have put on 20 or so pounds to showcase the new pair of jeans in a short video inside a Levi’s store. Alternatively, those who lack the visual sensitivity just “keeps watching” and assume that all is well and seem to experience no immediate discomfort. But all of us are inadvertently forcing our brain into overdrive, and it’s doing us no good.

While working on a number of display optimization projects, including those at WebMD and Air France-KLM, I have conducted a vast amount of research in trying to understand users’ perceptions of static or dynamic picture distortion, as well as their conscious and subconscious responses. One fascinating discovery I have made is that roughly 34% of all users cannot detect the distortion in the image when the aspect ratio is violated slightly – e.g., when we stretch a widescreen movie vertically by less than 15%. But the most amazing discovery is that 97% of all people – the annoyed techies, and the unsuspecting ignoramuses with low threshold for geometric beauty, and everyone else in-between – watching videos with wrong aspect ratio for 30 minutes or longer are about 160% more likely to experience fatigue, headaches, or emotional imbalances.

Research: Physiology of Cognitive Image Conversion

When the image – whether static or dynamic – is slightly distorted (under 15%, in this context), we always perceive and understand 100% of the video content, but our brain must subconsciously work in overdrive to continuously make dimensional adjustments in order to “fit” the edges of the misplaced (or skewed) objects into their intended locations – e.g., when a baseball looks like an egg, the brain has to constantly work to convert your perceived image (an egg-shaped object on the screen) into the contextually intended object (a baseball). In fact, I conducted a rather cool experiment to learn more about this Cognitive Image Conversion:

1. I picked three unrelated 30-second fragments from three music videos.
2. I made a distorted version of each fragment with a vertical or a horizontal stretch by 25%.
3. I then found 30 subjects – of them all were 25-35 years old, all had IQ scores of 100 to 140, all had college degrees, and all had normal vision (6 of them wore corrective glasses or contact lenses).
4. I then split the 30 people in two groups of 15 subjects and played the same three videos to each subject at 10 times the normal video speed (10x faster), asking each user to describe the videos. The first group watched the videos with the correct aspect ratio (the Control Group), while the second group watched the videos with the distorted aspect ratios (the Test Group). Naturally, we did not warn any of the Test Group subjects whether the videos would be distorted or not.
5. The results were stunning, but not surprising: 13 of the 15 people in the Control Group (the group that watched the properly sized videos) were able to register a notably greater level of detail in all three videos they watched than their counterparts in the Test Group.
6. The experiment was conducted with relatively small 22-inch (59mm) screens and with large HD TV’s, and the findings were essentially identical.

That said, imagine how attention spans and the decision making abilities of airline pilots or surgeons, for example, can be affected by using computer or TV displays that fail to show the correct aspect ratios. Enough said.

Implications for Users and for Display Manufacturers

Interestingly, when a static image (i.e., a photo or motionless artwork) is off-aspect, we tend to experience minimal aftereffects – regardless of whether we have noticed the distortion or not. This is because the amount of processing required for the Cognitive Image Conversion is low when there is only one frame in front of you (and thus one frame for the brain to process and convert). But the story is different with motion: at the conventional 32 frames per second in a video, our brains tend to distinctly perceive at least every 5th frame, and when a video feed is distorted, the brain must in turn “continuously loop” – about 6 times per second, to be exact – to convert distorted frames into valid frames. It is no wonder, therefore, that 97% of us are more likely to experience headaches or irritation or other forms of psychological and physiological distress when watching videos with incorrect aspect ratio settings for over 30 minutes.

Now, if you are a TV manufacturer, imagine what happens with people who spend 1-3 hours per day – as many Americans and Europeans do – in front of a TV set: their personalities and their health can be affected by this level of distress quite dramatically. Moreover, many of them actually recognize that their irritability or headaches or other forms of discomfort ensue after watching the TV for a while, and then the inevitable happens; they erroneously attribute the effects to the equipment and tell their friends and family this, more or less: there is something wrong with my So-And-So-Brand TV because it gives me headaches; I should have bought the TV made by So-And-So-Brand instead. For the cherry on top, think about your kids playing their games on “the wrong screen” and suffering from the resulting cognitive effects.

Whether it’s 4:3 or 16:9 or anything else, most users don’t know left from right (nor can they do the math) when it comes to setting up their monitors or TV sets correctly. Shouldn’t we try to help them? It’s been over 20 years since the appearance of wide screens, and we are still confusing the hell out of our valued audiences by failing to give them the tools that would enable them to watch an average of 25 minutes of TV commercials per hour correctly between their silly-little six-minute movie clips!

The solution

Conventional stand-alone computer monitors are a “fair game” as they are configurable by the user from the connected computer. But the manufacturers of television sets have no excuse: they must identify the original broadcast’s aspect ratio (this goes for transmitted signals and for locally wired feeds) and optimize the display in only one of these two equally meaningful ways:

1. Option 1: maximize the viewable screen area (e.g., clip sides of a widescreen image to eliminate black background at the top and the bottom of the screen), or
2. Option 2: preserve the original content to ensure that the entire area of the original video is visible on the screen (e.g., leave the black background on the sides when you are watching an old movie on a wide-screen TV).

Any other options beyond these two should be buried deeply into the menu and must be relatively difficult to change.

Do you want to talk about this? Do you know of a more elegant solution? Message me on Twitter or using the contact form on this site.