Digital Images: Foveon X3 versus Bayer
An image quality study based on SD9 samples
by Mike Chaney

Please note: All images (c)2002 Digital Photography Review, http://www.dpreview.com.
Images used with permission of original photographer.
This page is not sanctioned by dpreview.com

As many people try to get a handle on the image quality differences between Foveon's full color image sensor and traditional Bayer sensors, I decided to do a short study of a few samples. Since the SD9 captures all three color channels at each photo site, it is possible to "Bayerize" a few samples by discarding all but one color at each pixel to create a standard Bayer mosaic pattern. That pattern could then be "fed" to Qimage Pro in the form of a raw Bayer image, allowing Qimage Pro to recreate the image as if it were shot on a Bayer camera. In doing so, we can effectively compare a 3.4 MP image from a Foveon X3 sensor to what we would expect from a 3.4 MP Bayer sensor. The samples on this page should be used for purposes of illustration (of concepts) only. Since Bayer sensors capture images very differently from full color sensors like the Foveon X3, it is important for viewers to understand the differences between the two technologies in order to understand how to evaluate images once reviews of full color sensors start to appear on digicam review sites. Since Bayer and full color sensors differ so significantly in their ability to capture different types of detail in an image, it is important to understand that comparing Bayer sensor images with full color sensor images requires examination of more than just details on a black and white resolution chart! Hopefully this page will illustrate what to look for when comparing image resolution and quality across different sensor designs. Until now, the methods of comparing image quality from camera to camera were less important because all cameras used the same type of single color Bayer sensor. Now that full color sensors are making their way onto the market, we need to understand what we've been missing (using single color sensors) and what to expect with full color sensors. The process of comparison that was used on this page is simple:

Here is a visual depiction of the process of "Bayerization" as described on this page, and an illustration of how a full color sensor "sees" the world versus a Bayer sensor:

What an X3 sensor sees What a Bayer sensor sees Bayer after interpolation
Original SD9 crop
This is what the SD9
"sees" on the sensor and
also represents the SD9
final image*
The original after
"Bayerization". This is
what a Bayer sensor
"sees", A.K.A. the raw
image.
The image reconstructed
from the "Bayerized"
version. This is the final
Bayer image after being
processed (no AA filt.).

* Not including white balance and final color adjustment.

A note about validity and practical application of the samples here

Please note that the process that I am calling "Bayerization" involves manipulating data from a full color device to produce a theoretical Bayer sensor. We can do this (in theory) since a Bayer sensor is analogous to a full color sensor with 2 of 3 color channels omitted at each photo site. Due to differences in the optimization of each design however, actual/practical results will vary from the examples on this page, but should generally lie somewhere between the two Bayerized versions displayed. More information regarding the difference between the Bayerized samples and Bayerized samples with AA filter is given below.

In writing to Phil Askey about the information on this page, he had reservations about the actual comparisons regarding my Bayerized samples that did not include at least a simulated AA filter. AA filters (short for antialiasing filters), A.K.A. "low pass filters" are used in nearly all Bayer cameras. The AA filter is basically a "blurring" filter that is placed somewhere between the lens and the image sensor (usually right on top of the sensor). This blurring filter is used to eliminate aliasing of high frequency spacial detail due to the lens being able to resolve detail higher than the Nyquist frequency, which can result in color aliasing. This blurring filter is needed basically to reduce color distortions in the Bayer design due to its single color capture per pixel. As a result, and due to popular request, I have updated this page to include samples that simulate what a normal Bayer sensor would see if it were equipped with an antialiasing filter such as those found in most dSLR's. With this information in mind, here is a description of the samples presented on this page:

In the end, actual tests of the SD9 camera will be the only true way to evaluate image quality. After comparing the images on this page, you may now be armed with more knowledge about what to look for and how to compare images. As you can see, there's a lot more to quality than just resolving black and white horizontal/vertical lines.

Update (10/03/02) - Tuning the Simulated AA Filter

Peter posted a good idea on the DPreview forums about tuning the simulated AA filter that I used in these samples. The idea is to use a crop from a Foveon based image of a resolution target that can be compared directly to a good Bayer sensor with the same resolution. If my Bayer-DeBayer code produces a result very close to a real Bayer sensor, we would have more assurance that my simulated AA filter is not too weak, not too strong. Here's the thread on DPreview. In performing this test, I did find that my AA blurring filter was slightly over aggressive and needed to have less blurring effect to match what we would expect from an actual Bayer based camera. As of 10/03/02, I have replaced all Bayerized-with-AA samples on this page with the retuned versions, including the crops at the bottom of the page. Note that while I did end up retuning my simulated AA filter slightly, it did not have a very noticeable effect on the Bayer-with-AA samples. The retuned Bayer-with-AA samples do, however, appear slightly sharper than the original samples that I posted here. The crops below show the original Foveon image (left), my Bayerized simulation with AA filter (middle), and what an actual Bayer camera records (Canon D60) on the right. In addition, you can see from the bottom 200% zooms that my Bayerized-with-AA crop closely matches the detail available in the same crop from an actual Bayer camera. While this is not a complete proof of concept for my Bayerization techniques, it at least indicates that my Bayer-with-AA samples do a reasonable job of simulating an actual Bayer sensor with respect to detail and sharpness.



Samples

Sample 1: Portrait

Link to Phil's SD9 original

Link to Bayerized version

Link to Bayerized version with simulated AA filter

 

Sample 2: Landscape

Link to Phil's SD9 original

Link to Bayerized version

Link to Bayerized version with simulated AA filter


Conclusion

I'll leave any final conclusions regarding "resolution equivalents" or other absolute comparisons to the reader, as they really have little merit anyway until actual controlled tests can be performed. As for the samples on this page, simply download versions and zoom/examine side by side. Frankly, I was surprised at how well the image reconstructed after being Bayerized with no AA filter, however, since nearly all single color [sensor] cameras have an AA filter over the CCD, the samples that include the simulated AA filter will probably be closer to what you can expect from actual Bayer based cameras. Overall, the images held up reasonably well considering the Bayerized image started with only 1/3 the amount of captured information, however, there are some obvious areas where the Foveon X3 (full color sensor) technology excels, particularly in areas of high frequency detail. Some notable areas are shown below. The first two examples show some loss of high frequency detail in the Bayerized version while the third example shows loss of chrominance (color) detail in the parking lot gate as the Bayer interpolation algorithm "removed" color information because the sampling frequency was too low to obtain true color. Due to the increased low level detail and lack of artifacts, it is likely that the Foveon sensor design will produce images that can withstand significantly more processing, including the ability to resample images to larger sizes and produce large prints.

In conclusion, some things to look for when comparing Bayer and Foveon sensor images:

Cuff (200%) Bayerized Bayerized w/AA filt.

 

Watch (200%) Bayerized Bayerized w/AA filt.

 

Gate Bayerized Bayerized w/AA filt.