This week, I have to choose which area of chip design that I am going to involved in, and I decided to work on the accelerator for chip design. I had chosen to work on the accelerator for Image Sensor, to accelerate the image sensor output into a raw image that looks real, which is a challenging task for me.
Understanding the Image Sensor
In order to understand how image sensor produce color, I had read a lot of information about image sensor and color. It is interesting to know that the typical color is obtained from a Color Filter Array (CFA), which consists of a set of arrangement for each pixel. The most common CFA is the Bayer CFA which consists of a color combination of 2 green pixels, 1 red pixel and 1 blue pixel. It is really amazing to know that those beautiful images that finally are formed out as JPEG format in our phones or camera, are actually came from an interpolation algorithm. I found that Cambridge in color website is particularly useful in understanding all the fundamentals about the color and image theory. Besides, there are also many different CFA that had been innovated by the top image sensor manufacturers such as FujiFilm, ON semiconductor, and Sony. Each of them are trying to innovate and to improve the image sensor quality in their own way.
The Path Between Image Sensor and Raw Image
To accelerate the output of the image sensor into a raw image, and based on my current understanding about image sensor, there are two steps that need to be involved in, which is the Color Interpolation and Color Correction Matrix. Color interpolation which is also named as “Demosaicing Process”, is to convert the single color pixel information into three color pixel information. This is because each pixel originally only has one single color information (either red, green or blue from Bayer CFA), color interpolation actually interpolate and restore the RGB color for each pixel. As I read from this documentation by ON semiconductor themselves, these color interpolation algorithm are said to be the camera manufacturer trade secret which would not be published. However, the color correction matrix coefficients could be obtained from ON semiconductor themselves, and they may be improved from there.
Furthermore, I also studied about the generation of signal for the CCD from the documentation, and hopefully by next week I will be able to simulate the signal with each demosaicing algorithms, and identify the best algorithm to be used.
0 Comments