3D TV Technology and the Human Vision

3D TV technology renders an impression of depth while displaying an image over a 2D surface. Yet the human brain knows that a 3D object cannot be contained within a flat surface. This affects our way of seeing 3D content and may even leads to undesirable effects - including disorientation and headaches.

In this article, we discuss these issues; we also see how the difference between human vision and 3D television technology impacts the way film producers have to shoot content for 3D presentation with the aim of minimizing some of these ill effects.

In most simple terms, 3D TV is a display technology that helps you experience TV programs, movies and video games in a stereoscopic effect. It is based on what is referred to in 3D imaging technology as stereopsis, or separation between two full-size but slightly different images of the same scene - one to be viewed by the left eye and the other by the right eye. This separation is known as parallax, and the amount of parallax in 3D TV content determines the aggressiveness of the 3D experience.

This illusion of 3D however is created by 3D TV by displaying the 3D content as a two-dimensional flat image!

Main differences between 3D TV Technology and the Human Vision

While 3D TV technology relies on the use of two slightly different images to build up a 3-dimensional representation of the content being displayed on the screen in a similar manner to the way human vision works, yet there are a few distinct differences in the way we see a 3-dimensional object and the way 3D TV technology works.

Sequential Presentation of Image Content:

3D TV technology does not actually display the two separate images exactly simultaneously in the way we see, but these appear intermixed with one another in what is referred to as field-sequential 3D. This means that each of the two sets of full-size images intended for the left and the right eyes are displayed intermixed in a sequential order - one after the other rather than simultaneously.

The blurred 3D image as seen without 3D glasses A 2D representation of the blurred 3D image shown above Images taken from a stereoscopic skydiving footage - Credit : Dzignlight Studios

The whole process requires the user to wear special LCD shutter glasses - known as 3D glasses, for the two separate images to properly combine in the viewer's mind to create the illusion of a single 3D image. Without these glasses, the image on a 3D TV appears as a doubled and blurred 2-D picture of the intended 3D representation. 3D shutter glasses consist of two active LCD lenses that synchronize with the frame content on the screen - often via an infra-red link, to allow the left eye to see the images intended for the left eye only and the right eye to see the right-eye images only.

This is done by blocking the respective eye in sequence. The viewer would not see any flicker as the process is repeated at 120 per second to generate a full 1080p 60 Hz 3D image for each eye.

This means that the system is effectively delivering two simultaneous streams of 1080p60 high definition content even though the source material on the Blu ray disc is in most cases recorded in 24 fps. This is possible thanks to the latest developments not only in 3D HDTVs, but also in HDMI, and a whole new breed of Blu ray players.

In fact, the latest version of HDMI - HDMI 1.4 - is essential as it is the only HDMI version supporting enough data throughput to deliver two simultaneous streams of 1080p60 over a single HDMI interconnect. And the recently finalized  Blu ray standard - designated Blu-ray 3D^TM - have been designed to support the new sequential 3D TV in full HD - delivering two simultaneous 1080p signals (one 1080p signal for each eye) to comply with Full HD 3D TV.

However, not all that glitters is gold. The fact that original movie content has always been recorded in 24fps means that any 120Hz refresh rate HDTV will have to use 2:3 pull-down processing to match the 60Hertz frame rate of the 3D TV content with that of the 24 fps 3D movie content.

A 3D illusion over a 2-D Surface - totally unnatural!

Yet, there is an even more significant difference in the way we see a 3-dimensional object in space in comparison to the way 3D TV technology creates an illusion of a 3-D image. We say illusion because the system is just presenting 2D images on a flat surface at a fixed distance away from the viewer's eyes. This is rather unnatural for the brain. Why?

The left/right eye separation - called interpupillary distance, is responsible for the left and right eyes to see a slightly different image of the same object. For the average adult, this separation corresponds to approximately 2.5 inches; for a child, this is closer to 2 inches.

When seeing a real 3-D object, the eyeball accommodates to focus the eye on the object in space, meaning that both eyes have to converge slightly because of this interpupillary distance to be able to see the same point on the object. This also means that in a real world situation, the brain is directing the eyes converging muscles and the focusing muscles to the same point in space. At the same time, the left/right eye separation also creates a parallax - or separation between the two slightly different images as seen by the left and the right eye.

Parallax, and eyeball accommodation or focusing, are the two most important cues the human brain uses to determine relative image depth. But there are others as well like facts the brain has learned from experience and that unconsciously puts into the equation when determining an object relative depth. For example, colors of distant objects get desaturated in comparison to a nearby object; or that closer objects occlude further ones.

Now, 3D TV technology and 3D filmmaking address only the issue of image parallax, or stereopsis. 3D technology generates the illusion of a 3D image by displaying two slightly different 2D images on the same flat screen surface at a parallax (image separation) that would vary depending on the depth of the 3D TV image being shown on the screen. The parallax introduced by 3D imaging systems is there to simulate the parallax generated by the left/right eye separation when viewing a real object in space.

This parallax in 3D TV technology determines the image depth relative to the screen - in or out of the screen - depending on where the parallax between the left eye and right eye views causes the viewer's eyes to cross and therefore to converge.

This means that while the eyes focusing muscles are focusing the left and the right eye on a flat screen - at a fixed distance away from the viewer, the eyes converging muscles are converging to a point that can be anywhere - in front or behind the screen. In 3D terminology, this disparity is referred to as fundamental disparity.

This is a totally abnormal situation for the brain since as stated earlier on, when viewing an object in space, the brain directs both the eye converging and focusing muscles to the same point. This abnormality for the brain is not much of an issue for the majority of viewers as for most people, it is convergence that mainly helps the brain determines depth, not focus. This is true as long as the parallax in 3D TV images is managed properly, otherwise, it may cause discomfort. In particular, since the eyes would never diverge, any positive parallax in excess of the viewer's interpupillary distance will result in a difficult viewing experience - one where the brain would find it hard if not impossible to fuse the two stereo images into one 3D image.

Furthermore, too great a disparity between focus and convergence for extended periods of time will eventually wear out the audience as the brain tries to direct converging and focusing muscles to the same point in space.

This means that since this disparity between convergence and focusing is directly related to the left-eye/right-eye images parallax or separation, the parallax must reside within what may be defined as the audience 'comfort zone'. Exceed this comfort zone and 3D viewing may turn out difficult to watch.

Studies carried out by both In-Three Partners in 3D and Prof. Martin S. Banks, Professor of Optometry and Vision Science at Berkeley show that this comfort zone ranges from about negative 10 inches of parallax to a maximum of positive 2.5 inches for an adult, or a maximum parallax range of 12.5 inches. This parallax range applies to movie theater viewing. Images with a negative 10 inches of parallax will cause the viewers eyes to converge at 20% of the distance from them to the screen, i.e. a viewer 50 ft away from the screen in a movie theater would see the nearest converging point 10 ft away from him.

On the other hand, a positive parallax of 2.5 inches would be interpreted by the brain as the object residing at infinity since the axis of vision for the left and right eye would be parallel for the average adult with an interpupillary distance of 2.5 inches.

Children and 3D Viewing

The comfort zone for 3D viewing, and therefore the supported parallax range for the brain to correctly fuse the left and right eye images into a single 3D representation, is related to the viewer's interpupillary distance.

Since interpupillary distance in children is narrower than in adults, the 3D comfort zone for children is also narrower - ranging from approximately negative 8-inches to positive 2-inch for a child with an interpupillary distance of 2-inches.

This also means that the 3D children see is always more aggressive since the maximum parallax range supported by a child is 10-inches. Furthermore, a parallax which is within an adult's comfort zone, may be outside the child limit; should this happen, the 3D viewing experience would eventually turn out to be difficult and uncomfortable for the child.

This means that any 3D content created for the whole family should take into account the narrower interpupillary distance for children and therefore, the supported reduced parallax range for comfortable viewing.

3D Viewing, Convergence, Focusing, and ill effects

We have already stated earlier on that the disparity between the eyes convergence point in space and focusing may lead to a difficult viewing experience. Hence, this explains why there is the need to properly manage the separation, or parallax between the left and right eye images.

While most viewers will not suffer any ill effects after a brief orientation - lasting typically a few seconds - till the brain manages to fuse the two stereo images in one, a too aggressive 3D effect resulting from too much of a difference between convergence and focusing may in some individuals lead to severe disorientation and headaches.

Yet, there is more to a comfortable 3D viewing experience than just maintaining the parallax between the two stereo images within the audience comfort zone detailed above.

Abrupt changes in camera movement can easily turn out to be equally annoying. It will always take time - known as 'hunting time' in 3D terminology - for the viewer's eyes to adjust and converge to a different point in space. And the fact that the convergence point in space is difference from the eyes focusing point does further complicate the whole issue for the brain.

If the transition between shots entails abrupt large convergence changes, the eyes would not converge in time with the result that in those instances, what should appear as a 3D image would eventually appear as a blurred 2D image.

For the same reason, a too frequent dramatic camera movement would also result in an extremely tiring 3D viewing experience.

Equally annoying can be the passage of objects on the screen from left to extreme right or vice versa with a negative parallax, i.e. where the point of convergence is in front of the screen and therefore objects appears closer to the viewer than the screen surface. As the object view becomes partly hidden by the screen border at either extreme side, the eye further away from the object would see less of the object than the other eye. This causes a visual disparity between the left eye and the right eye - a disparity which would be interpreted by the mind as a problem in the 3D view. This may eventually lead to the viewer totally disengaging him or herself from the action on the screen.

3D Content Production - a totally new approach

These difficulties call for a totally different way of shooting 3D content. For example, the issue with the image disparity referred to in the previous paragraph is partially corrected by cropping both the images for the left and right eye so that both eyes see a 'balanced view' of the object. But this approach would then result in the cropped content to appear to float off the screen through what is often called a virtual window.

Another approach to correct problems arising as a result of abrupt changes in convergence between shots is depth matching - where shot to shot parallax or separation is adjusted so that the eyes would not be forced to change convergence too fast.

Depth grading is also another important technique in 3D TV technology and 3D content production. In this case, the internal depth of a group of objects or scene is changed so that the viewer’s eyes are left converged at the same distance to the screen at which the primary subject of interest in the next shot appears. This makes shot-to-shot transitions more comfortable for the audience.

Depth grading is also used in 3D viewing for artistic reasons such as when objects are extremely far away from the viewer. These would result in zero convergence. But zero convergence in 3D viewing means that image subjects like cloud formations would appear flat as there is nothing to give depth information.

This is normal in that our perception of depth decreases with an increase in distance. But a flat expanse during a 3D movie presentation may turn out to be disappointing for the audience. Hence, the parallax is often altered to help create more of a 3D impression than we actually experience in real life.