Reduction of Predictable Noise in Digital Images
Mar 2, 2016 20:07:59 #
Noise
In digital images, noise is a random variation of either brightness or colour, or both. However, the definition and origin of noise should be considered first. Noise is any undesired signal. This definition, per electrical engineers, applied to radio signals. An example is the static one hears on an AM radio when the radio is not tuned to a station, or tuned to a station with a very weak signal. What is ultimately important is the amount of signal vs. the amount of noise.
Signal to noise ratio (SNR) is just that. How much signal is present with respect to how much noise. When applied to cameras, it is the physical measurement of the sensitivity of the imaging system, whether that be film or digital. Common practice is to measure the SNR in decibels (db) of power. In reality, to simplify this term, it simply relates how much signal is present to how much noise is present. A SNR of 1 indicates equal amounts of signal and noise. Signal in our photographic usage would refer to how much image light is present with respect to how much noise.
A broadcast signal of sufficient strength will over power the static or noise signal such that the broadcast is the predominant sound from the radio. The signal a radio station broadcasts, it is analogous to light of the image we are taking with a digital camera. Similarly, a portion of an image which is brighter will overpower and mask the noise in that part of the final image.
Though inaudible, image noise is equivalent to the static heard on AM radios. It is unfortunate that this terminology carried over from radio broadcasting and found its way into digital imaging, but it did. The reason is that noise in digital imaging has a negative connotation. Though not exactly the same, grain in film images was an accepted part of photography. Noise, having that negative connotation, seems undesirable in digital images. Noise levels in digital imaging, if present in sufficient levels can render an image unrecognisable. There are, however, useful applications of noise. Well touch on those later on.
Most photographers have a concept of what noise is, but are not completely aware of the types of noise and, possibly, that certain types can be reduced, some predictably.
Types of noise
Gaussian Noise
Gaussian Noise arises during the acquisition of the image and is caused by insufficient illumination of the subject, and/or high sensor temperature, and/or transmission of the image (circuit noise).
Typically this type of noise is of a Gaussian distribution, additive, and independent at each pixel. It is also independent of the signal intensity and caused, most likely, by thermal noise (Johnson-Nyquist noise). This also includes the noise of capacitors resetting.
Each pixel is amplified as part of reading the sensor information. This produces a constant level of noise in the dark areas of the image. In colour cameras there is more amplification of the blue channel than the green or red. Thus, there can be more noise in the blue channel.
Salt and Pepper Noise
Impulse or spike noise is sometimes referred to as salt-and-pepper noise. This presents itself in an image by the presence of dark pixels in bright areas and bright pixels in dark areas of the image. This noise is caused by analog-to-digital converter errors, also bit errors in transmission. This can be mostly eliminated using a variety of techniques, those being Dark Frame Subtraction, Median Filtering, and Interpolation around dark/bright pixels. This noise is unique to each image.
Shot Noise
Dominant noise in dark areas of an image are typically caused by statistical quantum fluctuations. This is simply the variation of the number of photons sensed at a given exposure level. This noise is also known as photon-shot-noise. This noise has a root-mean-square value which is proportional to the square root of the image intensity. It follows that the noise level at a specific pixel is mutually independent of other pixels. Shot noise follows a Poisson distribution. This approximates Gaussian distribution except for very low intensity levels.
Additionally, photon-shot-noise is not the only source for this noise. Dark leakage current, also known as dark-shot-noise or dark-current-shot-noise, can also contribute to noise patterns within an image. Dark current is evidenced most at hot pixels within an image sensor. The dark charge of both normal and hot pixels can be removed by using various techniques.
Quantization Noise (or uniform noise)
Quantization of the pixels of an image into a number of discrete levels will create noise, referred to as quantization noise. Its distribution is approximately uniform. It can be dependent of the signal, but it will be independent of the signal if other sources of noise are large enough to cause dithering. This can also occur if dithering is explicitly applied.
Film Grain
Photographic film grain is a signal-dependent noise. It is similar in statistical distribution to shot noise. If the grains are uniformly distributed, that is an equal number per unit area, and if each grain has an equal and independent probability of developing a dark silver grain after being struck by and absorbing photons, the number of such dark grains in an area will be random. The distribution will be binomial. Where in areas the probability is low, the distribution will be close to the classic Poisson distribution of shot noise. One can use a simple Gaussian distribution to adequately model this condition.
Anisotropic Noise
This refers to noise sources that are dependent on the orientation of the image. Image sensors are, sometimes, subject to row or column noise.
Digital Cameras
Low light levels require more exposure either by slower shutter speeds, larger f-stops, or both. Additionally, higher gain (sensitivity or ISO) can be used. Also the combination of all of the fore mentioned could be used. Using slow shutter speeds can lead to a significant quantity of salt and pepper noise. This is due to photodiode leakage currents. Again, this salt and pepper noise can be reduced by using certain techniques touched on in this article.
Effects of Sensor Size
Heres some bad news for adherents of using less than full frame sensors. The size of the sensor area is the largest determinant of signal levels that will determine signal-to-noise ratio. These apparent noise levels assume the aperture area is proportional to the sensor area, or the f-stop or focal-plane illuminance is held constant. What this means is that for a constant f-number, the sensitivity of an image sensor scales approximately with the sensor area. Larger sensors typically create lower noise in images when compared to smaller sensors.
Where images are bright enough to fall in the shot noise restricted level, and are scaled to the same size on the screen, or printed the same size, pixel count makes little difference to perceptible noise levels. The noise depends primarily on the sensor area. It does not depend on how this area is divided into pixels.
For example, the noise level produced by a 4/3 sensor at ISO 800 is approximately equivalent to the noise level produced by an FX sensor at ISO 3200. As an aside, the ability to produce quality images at high ISOs is a major factor in the desirability of FX DSLR cameras, which use larger sensors than compact point and shoots. Examples exist that show a FX DSLR producing less noise at ISO 400 than a point and shoot at ISO 100.
Sensor Fill Factor
An image sensor is divided into rows and columns of photosites, which collect light from a given area. Not all sensors areas are used to collect light. This is due to other circuitry. A sensor with a higher amount of area for collecting light, that is fill factor, will produce better ISO performance based on sensor size.
Sensor Heat
Electronic equipment produces heat while it is operating. Temperature will have an effect on the amount of noise present in a digital image. As an aside, digital cameras will produce more noise in hotter weather than in colder weather. Longer exposures will also produce heat.
Image Noise Reduction
Now that the various types of noise, and causes of noise have been outlined, can anything be done to reduce any of these noise sources? Let's consider two topics.
Algorithms
Most algorithms used in converting data collected by an image sensor have some form of noise reduction. There may be many methods, but each must determine whether the actual variations in pixel values represent noise or a detail of photographic importance. The result is an attempt to diminish the noise while preserving the image.
This algorithm approach is not perfect. A trade off needs to be considered. The need to remove noise while preserving fine detail that may have a characteristic similar to noise is required. Some cameras have settings that permit the adjustment to control just how the camera approaches noise reduction as ISO values are increased.
Not All Noise is Bad
Large amounts of visible noise within an image is almost always undesirable. However, there are cases when a certain amount of noise is desired. Though degrading the signal-to-noise ratio, noise added for the purpose of dithering will improve the image visually. Noise is also helpful in the prevention of discretization artefacts, or colour banding (posterization).
Reducing Some Noise
Now that the various sources and forms of noise have been outlined, is there a way to reduce noise, but without negatively affecting the quality of the image? Reduction of any noise would, most likely, be beneficial. In that context, a proposed methodology follows.
Consider a source of noise that has not been presented until this point in this presentation. A certain noise is somewhat predictable, [b]Fingerprint Noise[/] also called fixed pattern noise. This is noise that is unique to each sensor. Fingerprint noise can be reduced in order to improve the image to some degree.
One must also realize the difference between two types of fixed pattern noise. Pixels that appear brighter on long exposures are called hot pixels. Pixels that always appear brighter are called stuck pixels. Knowing this we can reduce noise produced by stuck pixels for all images, and hot pixels in images which are exposed in such a way that those pixels will be predictably present.
Since each sensor has a unique finger print noise pattern all that would be necessary is to determine the the location of the unique pixels that form the "print pattern" and adjust each one appropriately. This could be done by photographing an evenly illuminated white surface and read the value of each pixel within the image produced.
Considering that a light meter will meter a scene and provide a shutter speed / f-stop combination which will render that scene with values half way between 0 and 255, we can do an exposure at a given ISO and each RGB value, for each pixel, theoretically, should be 127. Those pixels that are fingerprint pixels will vary from the value of 127 noticeably.
Proposed Methodolgy for Reduction of Fingerprint Noise
(Nikon D700 and similar RAW file format cameras)
1. Obtain as pure a white surface as possible, such as a high quality white paper.
2. Evenly illuminate that surface.
3. Start with an ISO of 100, successive images will be made at ISO settings of 400, 800, 1600, 3200, 6400, 12800, and 256000.
4. Manually set the focus at infinity.
5. Place the camera as close as necessary in order to fill the frame with the uniformly illuminated white paper.
6. Meter the scene to set the exposure.
7. Take the image.
8. Read through the file to ascertain the X,Y position and RGB values of each pixel.
9. Place an arbitrary error factor that will accept a limited variation above or below the value of 127.
(This value may require adjustment, initially. However, using a 97% or 6 sigma statistical deviation should provide a substantially usable value)
10. Once the allowable limits are determined, read through the file making note of the X,Y locations of pixels whose RGB values vary greater than the allowable error factor. Also note the numeric values of those pixels that exceed the allowable limits both greater and less than.
11. Do this some number of times for each exposure setting to assure the locations and values remain relatively constant from image to image.
12. Once the RGB values and X,Y locations of those values have been determined, the adjustment values can now be subtracted or added to those locations on any image made at that particular ISO and reduce the fingerprint noise of any image subsequently photographed at that particular ISO.
Though various exposure settings at a particular ISO may produce slightly different fingerprint-noise patterns, they should be similar enough that the application of the correction factor will still be suitable.
In digital images, noise is a random variation of either brightness or colour, or both. However, the definition and origin of noise should be considered first. Noise is any undesired signal. This definition, per electrical engineers, applied to radio signals. An example is the static one hears on an AM radio when the radio is not tuned to a station, or tuned to a station with a very weak signal. What is ultimately important is the amount of signal vs. the amount of noise.
Signal to noise ratio (SNR) is just that. How much signal is present with respect to how much noise. When applied to cameras, it is the physical measurement of the sensitivity of the imaging system, whether that be film or digital. Common practice is to measure the SNR in decibels (db) of power. In reality, to simplify this term, it simply relates how much signal is present to how much noise is present. A SNR of 1 indicates equal amounts of signal and noise. Signal in our photographic usage would refer to how much image light is present with respect to how much noise.
A broadcast signal of sufficient strength will over power the static or noise signal such that the broadcast is the predominant sound from the radio. The signal a radio station broadcasts, it is analogous to light of the image we are taking with a digital camera. Similarly, a portion of an image which is brighter will overpower and mask the noise in that part of the final image.
Though inaudible, image noise is equivalent to the static heard on AM radios. It is unfortunate that this terminology carried over from radio broadcasting and found its way into digital imaging, but it did. The reason is that noise in digital imaging has a negative connotation. Though not exactly the same, grain in film images was an accepted part of photography. Noise, having that negative connotation, seems undesirable in digital images. Noise levels in digital imaging, if present in sufficient levels can render an image unrecognisable. There are, however, useful applications of noise. Well touch on those later on.
Most photographers have a concept of what noise is, but are not completely aware of the types of noise and, possibly, that certain types can be reduced, some predictably.
Types of noise
Gaussian Noise
Gaussian Noise arises during the acquisition of the image and is caused by insufficient illumination of the subject, and/or high sensor temperature, and/or transmission of the image (circuit noise).
Typically this type of noise is of a Gaussian distribution, additive, and independent at each pixel. It is also independent of the signal intensity and caused, most likely, by thermal noise (Johnson-Nyquist noise). This also includes the noise of capacitors resetting.
Each pixel is amplified as part of reading the sensor information. This produces a constant level of noise in the dark areas of the image. In colour cameras there is more amplification of the blue channel than the green or red. Thus, there can be more noise in the blue channel.
Salt and Pepper Noise
Impulse or spike noise is sometimes referred to as salt-and-pepper noise. This presents itself in an image by the presence of dark pixels in bright areas and bright pixels in dark areas of the image. This noise is caused by analog-to-digital converter errors, also bit errors in transmission. This can be mostly eliminated using a variety of techniques, those being Dark Frame Subtraction, Median Filtering, and Interpolation around dark/bright pixels. This noise is unique to each image.
Shot Noise
Dominant noise in dark areas of an image are typically caused by statistical quantum fluctuations. This is simply the variation of the number of photons sensed at a given exposure level. This noise is also known as photon-shot-noise. This noise has a root-mean-square value which is proportional to the square root of the image intensity. It follows that the noise level at a specific pixel is mutually independent of other pixels. Shot noise follows a Poisson distribution. This approximates Gaussian distribution except for very low intensity levels.
Additionally, photon-shot-noise is not the only source for this noise. Dark leakage current, also known as dark-shot-noise or dark-current-shot-noise, can also contribute to noise patterns within an image. Dark current is evidenced most at hot pixels within an image sensor. The dark charge of both normal and hot pixels can be removed by using various techniques.
Quantization Noise (or uniform noise)
Quantization of the pixels of an image into a number of discrete levels will create noise, referred to as quantization noise. Its distribution is approximately uniform. It can be dependent of the signal, but it will be independent of the signal if other sources of noise are large enough to cause dithering. This can also occur if dithering is explicitly applied.
Film Grain
Photographic film grain is a signal-dependent noise. It is similar in statistical distribution to shot noise. If the grains are uniformly distributed, that is an equal number per unit area, and if each grain has an equal and independent probability of developing a dark silver grain after being struck by and absorbing photons, the number of such dark grains in an area will be random. The distribution will be binomial. Where in areas the probability is low, the distribution will be close to the classic Poisson distribution of shot noise. One can use a simple Gaussian distribution to adequately model this condition.
Anisotropic Noise
This refers to noise sources that are dependent on the orientation of the image. Image sensors are, sometimes, subject to row or column noise.
Digital Cameras
Low light levels require more exposure either by slower shutter speeds, larger f-stops, or both. Additionally, higher gain (sensitivity or ISO) can be used. Also the combination of all of the fore mentioned could be used. Using slow shutter speeds can lead to a significant quantity of salt and pepper noise. This is due to photodiode leakage currents. Again, this salt and pepper noise can be reduced by using certain techniques touched on in this article.
Effects of Sensor Size
Heres some bad news for adherents of using less than full frame sensors. The size of the sensor area is the largest determinant of signal levels that will determine signal-to-noise ratio. These apparent noise levels assume the aperture area is proportional to the sensor area, or the f-stop or focal-plane illuminance is held constant. What this means is that for a constant f-number, the sensitivity of an image sensor scales approximately with the sensor area. Larger sensors typically create lower noise in images when compared to smaller sensors.
Where images are bright enough to fall in the shot noise restricted level, and are scaled to the same size on the screen, or printed the same size, pixel count makes little difference to perceptible noise levels. The noise depends primarily on the sensor area. It does not depend on how this area is divided into pixels.
For example, the noise level produced by a 4/3 sensor at ISO 800 is approximately equivalent to the noise level produced by an FX sensor at ISO 3200. As an aside, the ability to produce quality images at high ISOs is a major factor in the desirability of FX DSLR cameras, which use larger sensors than compact point and shoots. Examples exist that show a FX DSLR producing less noise at ISO 400 than a point and shoot at ISO 100.
Sensor Fill Factor
An image sensor is divided into rows and columns of photosites, which collect light from a given area. Not all sensors areas are used to collect light. This is due to other circuitry. A sensor with a higher amount of area for collecting light, that is fill factor, will produce better ISO performance based on sensor size.
Sensor Heat
Electronic equipment produces heat while it is operating. Temperature will have an effect on the amount of noise present in a digital image. As an aside, digital cameras will produce more noise in hotter weather than in colder weather. Longer exposures will also produce heat.
Image Noise Reduction
Now that the various types of noise, and causes of noise have been outlined, can anything be done to reduce any of these noise sources? Let's consider two topics.
Algorithms
Most algorithms used in converting data collected by an image sensor have some form of noise reduction. There may be many methods, but each must determine whether the actual variations in pixel values represent noise or a detail of photographic importance. The result is an attempt to diminish the noise while preserving the image.
This algorithm approach is not perfect. A trade off needs to be considered. The need to remove noise while preserving fine detail that may have a characteristic similar to noise is required. Some cameras have settings that permit the adjustment to control just how the camera approaches noise reduction as ISO values are increased.
Not All Noise is Bad
Large amounts of visible noise within an image is almost always undesirable. However, there are cases when a certain amount of noise is desired. Though degrading the signal-to-noise ratio, noise added for the purpose of dithering will improve the image visually. Noise is also helpful in the prevention of discretization artefacts, or colour banding (posterization).
Reducing Some Noise
Now that the various sources and forms of noise have been outlined, is there a way to reduce noise, but without negatively affecting the quality of the image? Reduction of any noise would, most likely, be beneficial. In that context, a proposed methodology follows.
Consider a source of noise that has not been presented until this point in this presentation. A certain noise is somewhat predictable, [b]Fingerprint Noise[/] also called fixed pattern noise. This is noise that is unique to each sensor. Fingerprint noise can be reduced in order to improve the image to some degree.
One must also realize the difference between two types of fixed pattern noise. Pixels that appear brighter on long exposures are called hot pixels. Pixels that always appear brighter are called stuck pixels. Knowing this we can reduce noise produced by stuck pixels for all images, and hot pixels in images which are exposed in such a way that those pixels will be predictably present.
Since each sensor has a unique finger print noise pattern all that would be necessary is to determine the the location of the unique pixels that form the "print pattern" and adjust each one appropriately. This could be done by photographing an evenly illuminated white surface and read the value of each pixel within the image produced.
Considering that a light meter will meter a scene and provide a shutter speed / f-stop combination which will render that scene with values half way between 0 and 255, we can do an exposure at a given ISO and each RGB value, for each pixel, theoretically, should be 127. Those pixels that are fingerprint pixels will vary from the value of 127 noticeably.
Proposed Methodolgy for Reduction of Fingerprint Noise
(Nikon D700 and similar RAW file format cameras)
1. Obtain as pure a white surface as possible, such as a high quality white paper.
2. Evenly illuminate that surface.
3. Start with an ISO of 100, successive images will be made at ISO settings of 400, 800, 1600, 3200, 6400, 12800, and 256000.
4. Manually set the focus at infinity.
5. Place the camera as close as necessary in order to fill the frame with the uniformly illuminated white paper.
6. Meter the scene to set the exposure.
7. Take the image.
8. Read through the file to ascertain the X,Y position and RGB values of each pixel.
9. Place an arbitrary error factor that will accept a limited variation above or below the value of 127.
(This value may require adjustment, initially. However, using a 97% or 6 sigma statistical deviation should provide a substantially usable value)
10. Once the allowable limits are determined, read through the file making note of the X,Y locations of pixels whose RGB values vary greater than the allowable error factor. Also note the numeric values of those pixels that exceed the allowable limits both greater and less than.
11. Do this some number of times for each exposure setting to assure the locations and values remain relatively constant from image to image.
12. Once the RGB values and X,Y locations of those values have been determined, the adjustment values can now be subtracted or added to those locations on any image made at that particular ISO and reduce the fingerprint noise of any image subsequently photographed at that particular ISO.
Though various exposure settings at a particular ISO may produce slightly different fingerprint-noise patterns, they should be similar enough that the application of the correction factor will still be suitable.
Apr 12, 2016 23:17:52 #
rmalarz: Thanks. Having a background in electronics, I generally followed your discussion of noise and how to minimize it. I leave it to you and others of your caliber to effect the technical changes for actually reducing image noise. Of course, as always, the laws of physics will govern in this matter.
Apr 13, 2016 01:06:03 #
anotherview wrote:
rmalarz: Thanks. Having a background in electronics, I generally followed your discussion of noise and how to minimize it. I leave it to you and others of your caliber to effect the technical changes for actually reducing image noise. Of course, as always, the laws of physics will govern in this matter.
anotherview, I'm not going to be able to affect any technical changes for the manufacturers. All I can hope to do is arrive at some method to work with those features and produce the best images I can.
Again, you're quite welcome.
--Bob
Apr 13, 2016 09:27:35 #
Good morning. All the same, the manufacturers may find useful information in your writing for their technical changes.
rmalarz wrote:
anotherview, I'm not going to be able to affect any technical changes for the manufacturers. All I can hope to do is arrive at some method to work with those features and produce the best images I can.
Again, you're quite welcome.
--Bob
Again, you're quite welcome.
--Bob
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.