Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
For Your Consideration
The Fallacy of the "JPEG+Raw" option
Page <<first <prev 4 of 8 next> last>>
Dec 22, 2016 00:15:26   #
neilds37 Loc: Port Angeles, WA
 
Uuglypher wrote:
Thanks for your support, Neil.
I' ve broach the topic within a number of threads over the past few years, but I've become more and more irritated at various comment I read about the relative merits of 8-bit depth, vs. 12-bit depth vs. 14-bit depth vs. 16-bit depth formats regarding the relative numbers of tones of gray and hues provided by each format...with no mention that those figures are true only when the entire available dynamic range is utilized in the exposure.

We all should know from experience with the JPEG-adjusted in-camera histogram frame (the width of which represents the entire dynamic range allotted to JPEG) that the brightest stop (the right half of the histogram frame) provides 50% of the available tonal and color spectra. The next brightest stop ( half of the remaining half of the frame) provides 25% ...and so on...until we get over to the far left where the darkest stop may provide only 4 or 8 almost black gray tones.

Well, the same applies, with the same proportions but vastly different tonal and color numbers to the maximum DR available for the different bit-depths of raw image data captures... which the camera manufacturers cannot accurately state...because of "process variance" in production of silicon-based semiconductors (including our cameras' sensors). Each camera thus must be individually tested for how many more stops of exposure or parts thereof are available for raw exposure.

So...all this put together suggested that a thread was overdue dealing with specifically how use of the JPEG + RAW option is a very misleading "convenience" as regards the image data quality captured.

Best regards,
Dave
Thanks for your support, Neil. br I' ve broach the... (show quote)


Much like Linda, I was only shooting both for the rapid convenience of the jpg. Now that I have the capabilities of Faststone figured out it's as easy to view/routine edit Raw as a jpg, leaving Lightroom/PSE for the heavy-duty edits. Navigation in FS is left-click, next image, right-click, previous image.

Reply
Dec 22, 2016 03:58:25   #
kymarto Loc: Portland OR and Milan Italy
 
Uuglypher wrote:
[uquote=kymarto]Loss of 50 to 75% or more of tonal or color spectra? That sounds like a gross exaggeration. Please tell us how you arrived at that figure.


Quote:
Hi, Toby,
Thanks for joining in.

Given that the brightest available stop of exposure contains half (50% , if not more...) of its bit-depth's contribution of tonal and chromatic spectrum (depending on how much of the lower reaches of the DR was used...) -and that most cameras (about 70% of those tested) have AT LEAST one full stop of ERADR (if not more)... then for each stop of available exposure sacrificed , there is loss of 50% or more of its bit-depth's contribution. If two full stops of ERADR are sacrificed, the the loss is 75%. If two and 2/3 stops of ERADR are sacrificed, the loss is beyond 75%. And if the choice was also to expose to the left, well, you do the math.

Thus, if I were to fully expose my raw captures according to the JPEG histogram, Each image would have testified to sacrifice: one and 2/3 stops worth of tonal and chromatic spectra (somewhere between 50% and 75%) provided by their 14 bit-depth. To my mind, I'd rather not waste that much of the potential contribution of the sensor of a camera, 1/2 to 2/3 of the price of which supposedly went to the cost of the sensor.

This is all based on the twice-as-much /half-as-much relationship between full stops upon which we base so much of our exposure theory and practice.

The 2004 white paper by Bruce Fraser ""RAW Capture, Linear Gamma, and Exposure" is quite illuminating.
http://www.adobe.com/digitalimag/pdfs/linear_gamma.pdf

Dave
Hi, Toby, br Thanks for joining in. br br Given ... (show quote)


Hi Dave,
OK, that sounds reasonable. I certainly agree that if you're shooting jpg+raw and exposing to keep the jpgs from clipping you are losing a big chunk of what the raws can do. However I must in good conscience point out a couple of things.

The first is that this is all a bit logarithmic: just as +3dB doubles the sound power but really doesn't seem like a large jump in volume to the ear, so too in the image: underexposing by a full stop and then raising the values to look like what it would have been had you exposed one stop more really doesn't look that much different in normal viewing--often even two stops of underexposure don't really seem to affect the final image that much, even though you are losing then your 75% of tonal values, as it were. Certainly on very close viewing the differences will be noticeable, but generally speaking what is there will by far outweigh what is not--just as the difference between what we can see when we light a single candle in a dark space (as compared to no candle) is MUCH greater that the difference between one candle and two. This is certainly not a reason not to use the full dynamic range of the sensor, but that brings me to my second point.

Here we must consider cost/benefit questions. The benefits of using the full dynamic range are obvious (though perhaps not as great in reality as the numbers might seem to indicate, as per the above)--but what are the costs?

Here we have to take a step into the real world and consider situations. Under controlled conditions, where time is not critical, I fully agree that ERADR is absolutely recommendable. BUT... but...In the field, with changing conditions with time-critical subjects, one has to balance the possibility of blowing highlights past ERADR and losing the acceptable exposure, against keeping a bit of headroom for the inevitable changes in maximum brightness in the scene that will inevitably occur.

Working professionally in audio, I might point out that standard practice in setting the zero reference level for recording is either -12 or -14dB, even sometimes -20dB. Heavens! We are losing something like 97% of our dynamic range by the way you count, and yet no one will ever notice the difference unless they listen to absolute silence in an anechoic chamber. And even if there was a noticeable difference, one would have to balance that against possible peak distortion if one were to raise the zero level, one instance of which would ruin a recording.

I think you see my point. ERADR is not a panacea, and can actually be more than counterproductive in many normal shooting situations. Even if one shoots for jpg exposure in a combined jpg+raw situation, the raw file will contain substantially more information, which can go into making a much better image that what is available in the jpg in any case, with the added insurance of some headroom if necessary. As good as with ERADR? Clearly not, but one must be aware of the difference, and weigh which is the better compromise in any given situation.

Reply
Dec 22, 2016 06:37:07   #
JohnFrim Loc: Somewhere in the Great White North.
 
Uuglypher wrote:
Thanks for your support, Neil.
...
We all should know from experience with the JPEG-adjusted in-camera histogram frame (the width of which represents the entire dynamic range allotted to JPEG) that the brightest stop (the right half of the histogram frame) provides 50% of the available tonal and color spectra. The next brightest stop ( half of the remaining half of the frame) provides 25% ...and so on...until we get over to the far left where the darkest stop may provide only 4 or 8 almost black gray tones.
...
Thanks for your support, Neil. br ... br We all sh... (show quote)

I just want to confirm with you that the JPEG-adjusted in-camera histogram frame uses a sensor linear tone mapping for the X-axis and not a "gamma-adjusted zone mapping" (I may have that terminology wrong, but I think you know what I mean). I always thought the histogram could be divided into equal-width portions representing exposure increases, typically imagining about 5 bands along the X-axis.

Reply
 
 
Dec 22, 2016 08:07:10   #
steve_stoneblossom Loc: Rhode Island, USA
 
Uuglypher wrote:
Hi, Steve,
Not for the almost twelve years that I've been shooting either Raw captures by ETTR or EBTR or occasional JPEGs ( out of necessity)

I recall that back when I was trying JPEG+raw the histograms were roughly comparable in the computer after download. You can see the pre-normalization and post-normalization histograms of ETTR and EBTR images earlier in this thread.

Do you have some of interest to the discussion?

Dave

When someone claims that a particular methodology results in a 50% to 70% degradation in quality, who wouldn't be interested? I, on occasion*, have shot raw + JPEG, although I no longer do.
I will have to catch up with the discussion sometime in the near future. As for now, I must say it seems to me that such a large variation would be readily evident in a histogram.

*On the frequent days we had 2 or more clients getting married on the same day, and my wife and I were not on the same job, she always asks me to send images from my client for her to see during our set-up. As I cannot forward raw images via WiFi, I would shoot raw + JPEG. However, I now find it easier to just shoot these "glimpses", if you will, with my iPhone.

Reply
Dec 22, 2016 08:32:01   #
Frank2013 Loc: San Antonio, TX. & Milwaukee, WI.
 
kymarto wrote:
Hi Dave,
OK, that sounds reasonable. I certainly agree that if you're shooting jpg+raw and exposing to keep the jpgs from clipping you are losing a big chunk of what the raws can do. However I must in good conscience point out a couple of things.

The first is that this is all a bit logarithmic: just as +3dB doubles the sound power but really doesn't seem like a large jump in volume to the ear, so too in the image: underexposing by a full stop and then raising the values to look like what it would have been had you exposed one stop more really doesn't look that much different in normal viewing--often even two stops of underexposure don't really seem to affect the final image that much, even though you are losing then your 75% of tonal values, as it were. Certainly on very close viewing the differences will be noticeable, but generally speaking what is there will by far outweigh what is not--just as the difference between what we can see when we light a single candle in a dark space (as compared to no candle) is MUCH greater that the difference between one candle and two. This is certainly not a reason not to use the full dynamic range of the sensor, but that brings me to my second point.

Here we must consider cost/benefit questions. The benefits of using the full dynamic range are obvious (though perhaps not as great in reality as the numbers might seem to indicate, as per the above)--but what are the costs?

Here we have to take a step into the real world and consider situations. Under controlled conditions, where time is not critical, I fully agree that ERADR is absolutely recommendable. BUT... but...In the field, with changing conditions with time-critical subjects, one has to balance the possibility of blowing highlights past ERADR and losing the acceptable exposure, against keeping a bit of headroom for the inevitable changes in maximum brightness in the scene that will inevitably occur.

Working professionally in audio, I might point out that standard practice in setting the zero reference level for recording is either -12 or -14dB, even sometimes -20dB. Heavens! We are losing something like 97% of our dynamic range by the way you count, and yet no one will ever notice the difference unless they listen to absolute silence in an anechoic chamber. And even if there was a noticeable difference, one would have to balance that against possible peak distortion if one were to raise the zero level, one instance of which would ruin a recording.

I think you see my point. ERADR is not a panacea, and can actually be more than counterproductive in many normal shooting situations. Even if one shoots for jpg exposure in a combined jpg+raw situation, the raw file will contain substantially more information, which can go into making a much better image that what is available in the jpg in any case, with the added insurance of some headroom if necessary. As good as with ERADR? Clearly not, but one must be aware of the difference, and weigh which is the better compromise in any given situation.
Hi Dave, br OK, that sounds reasonable. I certainl... (show quote)
While this might not be the real right way to do it. I live in my viewfinder when shooting most of my stuff, as it's rarely static, I have learned about how many marks to the right I can let my in camera meter go before I know I'm going to have blown highlights so that's how I mostly shoot to the right, the light is constantly changing with my scene as I move the camera to catch it so I think you make a very good point here Toby.

Reply
Dec 22, 2016 09:41:11   #
steve_stoneblossom Loc: Rhode Island, USA
 
steve_stoneblossom wrote:
When someone claims that a particular methodology results in a 50% to 70% degradation in quality, who wouldn't be interested? I, on occasion*, have shot raw + JPEG, although I no longer do.
I will have to catch up with the discussion sometime in the near future. As for now, I must say it seems to me that such a large variation would be readily evident in a histogram.

*On the frequent days we had 2 or more clients getting married on the same day, and my wife and I were not on the same job, she always asks me to send images from my client for her to see during our set-up. As I cannot forward raw images via WiFi, I would shoot raw + JPEG. However, I now find it easier to just shoot these "glimpses", if you will, with my iPhone.
When someone claims that a particular methodology ... (show quote)


OK, I've now had time to read through more thoroughly.

Firstly, I would concur with kymarto in his concern over "in the field, on the fly" conditions vs a more controlled environment, and the potential to lose any perceived gain by virtue of the variance in dynamic range from shot to shot. For me, personally, the occasion to set up for a shot is rare, and if I were to attempt to expose for ERADR, I would prefer to err to the side of caution, exposing to the right less than the maximum possible, with the hope of gaining some without losing whatever gain I might have realized on the other end.

A question remains for me, though. Whether you're shooting raw, JPEG, TIFF, DNG, raw + JPEG, or another format, your camera produces a JPEG version, which is what you see on your LCD screen and what is used to produce the histogram. What is it about the act of saving the JPEG that reduces the quality of the related raw image? Which is why I asked about comparing histograms of raw only and raw + JPEG shots. Can you prove that by shooting raw + JPEG, the camera(s) handle the raw file differently than if you shoot raw only and the camera produces the JPEG without saving it?

Reply
Dec 22, 2016 09:46:32   #
selmslie Loc: Fernandina Beach, FL, USA
 
Uuglypher wrote:
The subject of the "JPEG+Raw" option arose in another thread. Rather than hijack that thread to continue this secondarily introduced topic, it seemed altogether meet and proper to start a new thread for discussion thereupon. ...

I realize that ETTR/EBTR/ERADR are subjects that are near and dear to your heart so please don't take this personally.

Throughout this thread you have not mentioned that the full DR of the sensor is not available unless you are at base ISO. Even a camera with a wide DR at base ISO like the D810 drops to under 11 Ev at ISO 1600 and about 8 Ev by ISO 12,800 (according to DxOMark).

Neither have you mentioned the relationship between the scene's DR and the sensor's capacity. Most daylight outdoor scenes present a useful DR of about 5 stops (Zones III through VIII) within which you can see detail. Below and above that range detail starts to drop off. Having additional DR in the raw file may not provide any benefit other than allowing you to pay less attention to the exposure knowing that you will be adjusting the image in post.

In a post I made to the Further on ISO invariance thread I demonstrated that there often is a wide range of possible exposure variation that can still produce an acceptable image. Specifically, that post showed no difference between an image exposed at EC -2 and EC +2 after adjusting the "Exposure" (gain) in post processing.

Scenes with a wider DR will, of course, require more careful exposure to avoid blowing the highlights. However, such wide DR scenes are unusual since they may involve artificial light (night scenes, fireworks, etc.) or scenes with a combination of subjects like indoor scenes with a view to the outside through a door or window.

Regarding the usefulness of having both a raw and a JPEG version of the same image, it's really a personal choice. Sometimes the JPEG is perfectly suitable and you don't need the raw version, sometimes it is a simple matter to view the JPEG to cull out the losers (both the raw and JPEG) if the capture was not good for reasons other than exposure.

I see no reason not to shoot JPEG only or raw only depending on the circumstances.

Reply
 
 
Dec 22, 2016 11:38:06   #
Uuglypher Loc: South Dakota (East River)
 
Hi, Toby,
My point is not that it is convenient to use your camera's maximum DR in all situations, but that when one wishes to capture the maximum possible image data quality, every effort should be made to use that maximum available DR.

It is true that one stop differences are not appreciated as "twice as bright" or "half as bright" ( see the JPEG example presented) but the effect on basic image data quality (signal-to-noise ratio) is very real...absolute.

Also, your trepidation about clipping highlights is one of the great advantages of EBTR, should one chose to use it. Knowledge of the ERADR of one's camera is a great insurance policy against blown highlights!

Also, Scotty, note that the series shown could easily be extended at both ends to provide a DR range of far more that than your suggested five stops upon which you base much of your other statements.

Best regards,
Dave


(Download)

Reply
Dec 22, 2016 11:47:37   #
Uuglypher Loc: South Dakota (East River)
 
JohnFrim wrote:
I just want to confirm with you that the JPEG-adjusted in-camera histogram frame uses a sensor linear tone mapping for the X-axis and not a "gamma-adjusted zone mapping" (I may have that terminology wrong, but I think you know what I mean). I always thought the histogram could be divided into equal-width portions representing exposure increases, typically imagining about 5 bands along the X-axis.


Hi, John.
Sure, you can divide the histogram frame into whatever is the appropriate number of equal segments to represent the DR, but to what purpose? It would not represent the actual contribution of each brighter stop to image data quality, which is what we ought be concerned with.

And remember...some or all benefits of using raw capture are realized only with use of some or all of the increased DR available for that raw capture!
See the attached graphics.


(Download)


(Download)

Reply
Dec 22, 2016 11:53:34   #
Uuglypher Loc: South Dakota (East River)
 
steve_stoneblossom wrote:
OK, I've now had time to read through more thoroughly.

Firstly, I would concur with kymarto in his concern over "in the field, on the fly" conditions vs a more controlled environment, and the potential to lose any perceived gain by virtue of the variance in dynamic range from shot to shot. For me, personally, the occasion to set up for a shot is rare, and if I were to attempt to expose for ERADR, I would prefer to err to the side of caution, exposing to the right less than the maximum possible, with the hope of gaining some without losing whatever gain I might have realized on the other end.

A question remains for me, though. Whether you're shooting raw, JPEG, TIFF, DNG, raw + JPEG, or another format, your camera produces a JPEG version, which is what you see on your LCD screen and what is used to produce the histogram. What is it about the act of saving the JPEG that reduces the quality of the related raw image? Which is why I asked about comparing histograms of raw only and raw + JPEG shots. Can you prove that by shooting raw + JPEG, the camera(s) handle the raw file differently than if you shoot raw only and the camera produces the JPEG without saving it?
OK, I've now had time to read through more thoroug... (show quote)


Hi, Steve,
The purpose of this thread is to point out that use of the "JPEG+Raw" option means that the raw capture you get at the end was exposed according to the DR allocated to the JPEG file...no more than that. If you are shooting the "JPEG +raw" option, compare the histograms of each SOOC, and then compare the SOOC and post -tonal normalization histograms of the EBTR exposure of a prairie dog I posted earlier in this thread.

Dave

Reply
Dec 22, 2016 12:01:32   #
Uuglypher Loc: South Dakota (East River)
 
Frank2013 wrote:
While this might not be the real right way to do it. I live in my viewfinder when shooting most of my stuff, as it's rarely static, I have learned about how many marks to the right I can let my in camera meter go before I know I'm going to have blown highlights so that's how I mostly shoot to the right, the light is constantly changing with my scene as I move the camera to catch it so I think you make a very good point here Toby.


Hi, Frank,
You will, of course, expose as you are most comfortable. The point I make is that the appearance of the JPEG-adjusted thumbnail your camera presents is useless for determining proper raw capture exposure. Your in-camera histogram may be the accurate means of determining the ETTR exposure, but to realize the benefits of raw capture without fear of clipped highlights you need to add your camera's PRE-DETERMINED ERADR ! That thumbnail will then look washed out with blown highlights...which the image would be if you were shooting JPEG FILES! But if you are shooting raw, that washed out thumbnail does NOT indicate overexposure.

Dave

Reply
 
 
Dec 22, 2016 12:03:10   #
kymarto Loc: Portland OR and Milan Italy
 
steve_stoneblossom wrote:
OK, I've now had time to read through more thoroughly.

Firstly, I would concur with kymarto in his concern over "in the field, on the fly" conditions vs a more controlled environment, and the potential to lose any perceived gain by virtue of the variance in dynamic range from shot to shot. For me, personally, the occasion to set up for a shot is rare, and if I were to attempt to expose for ERADR, I would prefer to err to the side of caution, exposing to the right less than the maximum possible, with the hope of gaining some without losing whatever gain I might have realized on the other end.

A question remains for me, though. Whether you're shooting raw, JPEG, TIFF, DNG, raw + JPEG, or another format, your camera produces a JPEG version, which is what you see on your LCD screen and what is used to produce the histogram. What is it about the act of saving the JPEG that reduces the quality of the related raw image? Which is why I asked about comparing histograms of raw only and raw + JPEG shots. Can you prove that by shooting raw + JPEG, the camera(s) handle the raw file differently than if you shoot raw only and the camera produces the JPEG without saving it?
OK, I've now had time to read through more thoroug... (show quote)


I seriously doubt that any camera handles raw files differently if you opt to save a jpg as well. I think Dave's point is that if you expose for the jpg as shown in the viewfinder--to not clip highlights--you will be losing the extra highlight information contained in the raw but absent from both the jpg and its representation in the viewfinds

Reply
Dec 22, 2016 12:11:37   #
Uuglypher Loc: South Dakota (East River)
 
Scotty,
Your caution to use the lowest possible ISO is well made. However, regardless of the ISO you employ, use of your camera's maximum allotment of raw-accessible dynamic range will ALWAYS provide better image data quality than not doing so.

Edit:
also, for those who have higher favorite ISOs, it is wise to determine the available ERADER at those higher ISOs. It is not the straight-line relationship that has been often proposed, and can be quite irregular.

The bottom line?
Raw image data capture with proper raw capture exposure is a totally different imaging medium from JPEG image file captures.

The proper raw capture exposure is one that produces the brightest image possible by coming as close as possible to clipping highlights without actually doing so. That provides an image data file guaranteed to permit accomplishment of your previsualized image, and accomplishment of a great variety of other creatively artistic tonal and chromatic variations as well!

the JPEG image data is, comparatively, "baked in" a very limited range of variations.

There is never a downside, regardless of the scene DR or the ISO employed to capturing raw image data with the maximum available DR! So when it is convenient, that's what aficionados of EBTR always do.

Just the facts.

Dave

Reply
Dec 22, 2016 12:12:49   #
kymarto Loc: Portland OR and Milan Italy
 
An interesting side note here: I recently shot a prime-time documentary for German TV using a Canon C300 Mk II camera. One of the settings for gamma is called Clog2, which is monstrously flat, actually highly compressed. I was instructed to shoot so that the highlights always looked blown (as the LUT (look up table) of the camera cannot accurately represent the range of tones in the viewfinder as they can be brought out in grading, and so always clips the highlights in live view). This was monstrously difficult, as I have always shot video by eye. The only way I could operate was by keeping my zebras at a 5% warning for clipping, and only having a hint of them in the viewfinder at any given time. It's like flying by instrument...

This is a camera designed for professionals. It would be easy enough for mirrorless and digital camera makers to implement something similar--so that you were really getting a better indication of your ERADR in real time--and I would welcome it. But honestly, it is very difficult to get used to such an absolutely washed out, dismal image for general shooting and composition. Of course the gamma curve here is not raw information, but comes a lot closer to representing your true dynamic range than the oversaturated and contrasty image that most manufacturers choose to make amateurs feel good about what they are seeing.

Reply
Dec 22, 2016 12:16:50   #
Frank2013 Loc: San Antonio, TX. & Milwaukee, WI.
 
Uuglypher wrote:
Hi, Frank,
You will, of course, expose as you are most comfortable. The point I make is that the appearance of the JPEG-adjusted thumbnail your camera presents is useless for determining proper raw capture exposure. Your in-camera histogram may be the accurate means of determining the ETTR exposure, but to realize the benefits of raw capture without fear of clipped highlights you need to add your camera's PRE-DETERMINED ERADR ! That thumbnail will then look washed out with blown highlights...which the image would be if you were shooting JPEG FILES! But if you are shooting raw, that washed out thumbnail does NOT indicate overexposure.

Dave
Hi, Frank, br You will, of course, expose as you a... (show quote)
What I'm trying to say Dave is by trial and error (camera to computer) I have learned how far I can push my in camera meter and not blow my raw files so to speak. May not be scientific but it gets me close with the type of shooting I currently do. I basically shot various scenes and notated the meter marks then loaded the images and adjusted in the computer. Gave me a good idea of how far right of center I can let my meter get....might not be totally optimal data I can capture but much more than before I read you articles so for that I would like to thank you.

Edit: I rarely if ever look at an image on the camera.

Reply
Page <<first <prev 4 of 8 next> last>>
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
For Your Consideration
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.