kymarto Loc: Portland OR and Milan Italy

I probably won't have time to do this, but I will post some examples from an HDR series. To answer your question, I generally start linear with ACR and then spice to taste non-linearly in PS. Here are three examples. First is at the nominal camera exposure, then +1, then +2 EV. On top is the original as ACR sees the RAW, then posted to my taste.

Later I will post my Nikon D800 vs. Canon 5D3 shadow noise test.

(Download)


(Download)


(Download)


(Download)


(Download)


(Download)

kymarto Loc: Portland OR and Milan Italy

OK, here is my noise test. Shot with Nikon 880E with 14-24 and the Canon 5D3 with the 16-35, both at 16mm @f5.6. The patches are taken from the center of frame. Shots were exposed at -3EV and then shadows raised linearly to approximate a normal exposure. I did a slight bit of adjustment to color balance and exposure to equalize differences, but it was very minor.

At the top is the Nikon D800E at 100 and 800 ISO, at the bottom the Canon 5D3 the same. If you download and open the file at full size you will be seeing them at 150% magnification.

What is immediately clear is how noisy the Canon is at 100 ISO. In fact, it is arguably worse than the Nikon at 800 ISO, at least in noise in the shadows. The Canon tends to be noisier in the shadows and the Nikon in the highlights, but the differences are pretty plain. This is why I shoot for myself with the Nikon.

(Download)

Uuglypher Loc: South Dakota (East River)

Thanks, kymarto,
From these three examples I have to conclude that if the second images of the second and third exposures were the best you could get from attempted tonal normalization, that your D800E has less than one stop of ERADR.

But, were it to have 2/3 stop of ERADR, that could be useful.

Your noise comparisons were surprising, and should warm the hearts of D880E lovers everywhere ...and swell the collective heart of all the Sony workers who designed and made that sensor!

Thanks, and best regards,
Dave

kymarto Loc: Portland OR and Milan Italy

Hi Dave,

I think that the D800 has a bit more than one stop--the "normal" image was already overexposed in the sky, but I would have to do some more formal tests to confirm that. It definitely has more room in the shadows than the highlights, though.

I found this:

http://www.martinzimelka.com/pages/D800_5.html

Uuglypher Loc: South Dakota (East River)

xxxxxxxxxxxx

Hi, kymarto,
thanks for the link.
Martin Zimelka's blog is, in substance and detail, a great testimony to the benefits of EBTR.
He states:
"Below, you’ll find a scene which I overexposed by roughly three stops. The amount of recoverable highlight detail is very also impressive. I would consider the practical limit of highlight recovery to be around 2 stops, otherwise colour transitions start looking artificial. This sample image was also a ISO 100 file."

He uses the term "overexposed" but we must realize that it is "overexposed" only in terms of what appropriate exposure would be for the dynamic range accessible to a JPEG file, not taking into account that the "capture of record" in the illustrated case is RAW data utilizing the ERADR, for which the exposure is correct, NOT "overexposed".

Also, he is doing his pp in Lightroom, which uses ACR for its raw conversion.

The only caution I recommend in interpreting his report is to realize that the actual amount of ERADR can vary among individual cameras of the same model. Cameras thus must be evaluated for their ERADR (Extra Raw-Accessible Dynamic Range) on an individual basis. I suspect that will continue to be the case until the sensor manufacturers determine to be more candid concerning the ERADR of their sensors.

I might add that although I seriously question my ability to base exposure decisions on my consistency of recognizing 1/3 stop differences, it becomes significant in testing a camera's ERADR. Happily, the decision endpoint in that task is the relatively easily percieved "clipped highlights or no clipped highlights" distinction!

Best regards, and thanks much for your interest and for your substantive participation in this discussion.

Dave

Uuglypher Loc: South Dakota (East River)

quote=kymarto]Hi Dave,

I think that the D800 has a bit more than one stop--the "normal" image was already overexposed in the sky, but I would have to do some more formal tests to confirm that. It definitely has more room in the shadows than the highlights, though.

I found this:

http://www.martinzimelka.com/pages/D800_5.html[/quote]

Hi again, kymarto,
I meant to add ....

RAW vs.JPEG Exposure?

I have found this to be a useful conceptual framework within which to discuss the merits of ERADR (Extra RAW-Accessible Dynamic Range) and EBTR (Expose Beyond the Right).

JPEG exposure?
Same as reversal color film: "expose for highlight detail, process for the shadows." The histogram and the thumbnail image extracted from the originally captured Raw data are helpful guides to JPEG exposure.

RAW data exposure?

Neither the histogram nor the thumbnail give any reliable information relating to exposure....
except for...
the clipping signal !

Although the lit arrowhead or the "blinkies" don't indicate when a RAW capture is clipping highlight detail, they do provide a convenient " touchstone" indicating that the exposure has been reached to which to add the pre-determined ERADR (extra RAW-Accessible Dynamic Range).

Thus, the easiest path to determining a camera's ERADR is to start with the exposure that elicits the clipping signal, and add exposures increasing by 1/3 stop each until an exposure is reached in which actual highlight clipping has occurred.

In other words, for optimal RAW data exposure: expose for maximum brightness with no clipped highlights, then tonally normalize by linear processing.

Dave

EBTR

(Download)

after tonal normalization

(Download)

kymarto Loc: Portland OR and Milan Italy

My problem with these concepts is that, like with audio headroom, there is good reason to leave some unless you are in a situation in which you have full knowledge and control of the luminosity of every object in your frame. When we record, we sacrifice a little signal to noise for the odd transient that might exceed max modulation and distort.

With modern digital equipment--both photo and audio, the dynamic range and noise floor are already way below those of their analog counterparts. While I appreciate the effort to maximize performance, my personal philosophy, since mostly I shoot in the field and don't spot meter every highlight, is to shoot a little conservatively so that I don't risk blowing any highlights that are not specular reflections. I find with the D800 that (at least at low ISOs) I am mostly always able to pull clean detail out of shadow areas, so it becomes much less desirable to risk my highlights by exposing to the right.

Uuglypher Loc: South Dakota (East River)

[kymarto]My problem with these concepts is that, like with audio headroom, there is good reason to leave some unless you are in a situation in which you have full knowledge and control of the luminosity of every object in your frame. When we record, we sacrifice a little signal to noise for the odd transient that might exceed max modulation and distort.

With modern digital equipment--both photo and audio, the dynamic range and noise floor are already way below those of their analog counterparts. While I appreciate the effort to maximize performance, my personal philosophy, since mostly I shoot in the field and don't spot meter every highlight, is to shoot a little conservatively so that I don't risk blowing any highlights that are not specular reflections. I find with the D800 that (at least at low ISOs) I am mostly always able to pull clean detail out of shadow areas, so it becomes much less desirable to risk my highlights by exposing to the right.[quote]
xxxxxxxxxxxxxxcccc

you state:
"While I appreciate the effort to maximize performance, my personal philosophy, since mostly I shoot in the field and don't spot meter every highlight, is to shoot a little conservatively so that I don't risk blowing any highlights that are not specular reflections."

I agree. I have a camera that, although it has one and 2/3 stops of (technical)ERADR, it also demonstrates some aberrant hue and value shifts at one and 1/3 stops. With that camera I am definitely conservative and use only one full stop of its (technical) ERADR.

xxxxxxx

you state:
"When we record, we sacrifice a little signal to noise for the odd transient that might exceed max modulation and distort."

Does "...the odd transient that might exceed max modulation and distort." refer to the distorted sky colors accompanying the blown clouds in one of your buddhist statue images following tonal normalization shown earlier in the thread?

xxxxxxxxxxxxx

you state:
"I find with the D800 that (at least at low ISOs) I am mostly always able to pull clean detail out of shadow areas, so it becomes much less desirable to risk my highlights by exposing to the right.

As regards absolutely not risking blown highlight detail I think we are on the same page. The premise I've long observed is the point made repeatedly by Jeff Schewe, that to assure the highest possible image data quality in RAW capture, expose for the brightest possible image with absolutely no blown highlights (except, of course., for specular highlights which, by definition, have no detail). Am I correct that you ascribe, rather, to the idea that superior shadow detail can even result from underexposure and tonal normalization by moving the raw converter's "Exposure" slider to the right?

A corollary of Schewey's (and my) practice is that all RAW image data exposed to result in the brightest possible image with no clipped highlights will be of the highest possible S:N ratio, including the data in the "darker end" of the histogram " light pile" (as well as the bright end). Which begs the question of how the quality of purposfully underexposed shadow detail ( with its lower S:N ratio) could be quantitively greater and qualitatively superior to the shadow detail captured further to the right...indeed, possibly even beyond the right (the particular camera's ERADR permitting)?

My concern is with conclusions drawn from exposures made with cameras, the precise ERADR of which has not been accurately determined. And given that two accurately tested D800s have had ERADRs differing by as much as one full stop suggests that guessing at the actual "headroom" of an individual, untested camera is fraught with possible error...and risk of "...the odd transient that might exceed max modulation and distort."

kymarto, I do appreciate your patience and your candid observations and findings, as well, and most important, your willingness to help assure clear communication between us.

Best regards,
Dave

kymarto Loc: Portland OR and Milan Italy

Hi Dave,
I'm very surprised that two D800s would vary to that degree. What physical manufacturing variances could account for such a discrepancy? I would have to wonder how accurately tested they were. Still, that is a subject that has not been much explored AFAIK.

Yes, the odd transient would definitely mean something like sky colors in the Daibutsu image. As I understand it, recovering luminosity is one thing, recovering reasonably accurate chroma information another.

The eye will notice false colors and tonality much more in light areas than in dark areas. I'd much prefer to have a little room to play in the highlights and have shadows that are a bit noisier and/or with flatter tone curve than vice versa. ETTR in any case tends to compress the tone curve in the highlights IME. I would prefer to emphasize contrast in the highlights and let the shadows flatten out a bit than the other way around.

Also, shadows clipped to black, if tonality runs all the way to black (instead of clogged shadows with a definite demarcation between the darkest real tone and black) are not anywhere near as disturbing as highlights clipped to white.

With older digital cameras, I liked the ETTR concept, since shadows brought up more than about 2EV were extremely noisy (Canon's still are) and often clogged at the low end, which is very unpleasant. With the D800, I have found that I am able to bring shadows up extremely well with minimal extra noise or tonal unevenness, so I am now finding much less real use for the concept, even though I know I am not getting the maximum S/N. The difference is hardly noticeable, and I am getting a higher percentage of good shots than if I were skating on thin ETTR ice.

Here are two examples. Some experiments I shot last year at the beach in Japan near me. In both cases I post the original image, as it came out of ACR at default, and then after my tweaking. In both cases you can see how much tonal gradation I was able to pull out of severely underexposed areas, and pretty clean at that. Had I exposed any more to the right, I would have had much more blown out highlights that would have been unrecoverable (even more unrecoverable). (Note that "modified 1" is slightly cropped on the right as compared to its original.)

original 1

(Download)

modified 1

(Download)

original 2

(Download)

modified 2

(Download)

Uuglypher Loc: South Dakota (East River)

Hi, kymarto,
I'm on the road to Madison WI..270 horrendously icy miles to go. Will respond after I get a good night's sleep.
THANKS For your patience.

Dave

Uuglypher Loc: South Dakota (East River)

Hi Kymarto

You wrote:
"I'm very surprised that two D800s would vary to that degree. What physical manufacturing variances could account for such a discrepancy? I would have to wonder how accurately tested they were. Still, that is a subject that has not been much explored AFAIK."

As regards testing for ERADR I agree that it must be done in a uniform manner. The exposure at which the JPEG-adjusted histogram and histogram warnings (the "blinkies") suggest the first hint of highlight clipping in a JPEG file is taken as the base exposure. Beyond that exposure, a nine-exposure series of additional exposures, each 1/3 stop brighter than the previous one, is captured. When, during tonal normalization, the first indication of highlight clipping is found, the preceding exposures constitute the ERADR.

I, too, was surprised to find different amounts of ERADR in two or more cameras of the the same model.

Three Nikon D800 have yielded:

Nikon D800: 1 and 1/3 stops (exposure series confirmed by EXIF and evaluated by DLG)
Nikon D800 2 and 1/3 Stops (exposure series confirmed by EXIF and evaluated by DLG)
Nikon D800 2/3 Stop (reported by John Shaw...experienced and reliable re: EBTR)

the other model for which I have similarly reliable ERADR data is the Nikon D7100
Nikon D7100 one stop,
Nikon D7100: 2/3 stop

I admit to total ignorance as to why such variation should be encountered and am eager to be disabused of that gap of knowledge.
Xxxxxxxxx
You wrote:

"Yes, the odd transient would definitely mean something like sky colors in the Daibutsu image. As I understand it, recovering luminosity is one thing, recovering reasonably accurate chroma information another. "

When I see that sort of dyschromic aberration it has, so far, invariably been in exposures past the point of first evidence of highlight detail clipping.

Xxxxxxxxxxxxxxx

you wrote:
"The eye will notice false colors and tonality much more in light areas than in dark areas. I'd much prefer to have a little room to play in the highlights and have shadows that are a bit noisier and/or with flatter tone curve than vice versa. ETTR in any case tends to compress the tone curve in the highlights IME. I would prefer to emphasize contrast in the highlights and let the shadows flatten out a bit than the other way around. "

WITH LINEAR CAPTURE AND LINEAR PROCESSING, INCLUDING TONAL NORMALIZATION, I'M AT A LOSS HOW TO FIND A "TONE CURVE."..CAPTURED LINEARLY AND PROCESSED LINEARLY THE STRAIGHT-LINE RESPONSE CHARACTERIZES THE DIGITAL IMAGING PROCESS. I DO AGREE THAT dyschromiax CAN OCCUR IN THE UPPER REACHES OF THE RESPONSE AS ACTUAL CLIPPING IS REACHED AND EXCEEDED, BUT HAVING TESTED MANY CAMERAS FIRST-HAND AND EVALUATED MANY CORRECTLY MADE TEST EXPOSURE SERIES, THE ONLY TIME I'VE NOTED DYSCHROMIA/COLOR ABERRANCIES , as stated above, HAS BEEN IN THE BRIGHT END OF THE SERIES AFTER THE FIRST EVIDENCE OF HIGHLIGHT CLIPPING FIRST APPEARED.

Xxxxxxxxxxxxxxxx

You wrote:
"Also, shadows clipped to black, if tonality runs all the way to black (instead of clogged shadows with a definite demarcation between the darkest real tone and black) are not anywhere near as disturbing as highlights clipped to white."
I AGREE TOTALLY.

Xxxxxxxxxxxxxx

You wrote:
"With older digital cameras, I liked the ETTR concept, since shadows brought up more than about 2EV were extremely noisy (Canon's still are) and often clogged at the low end, which is very unpleasant. With the D800, I have found that I am able to bring shadows up extremely well with minimal extra noise or tonal unevenness, so I am now finding much less real use for the concept, even though I know I am not getting the maximum S/N. The difference is hardly noticeable, and I am getting a higher percentage of good shots than if I were skating on thin ETTR ice."
"Here are two examples. Some experiments I shot last year at the beach in Japan near me. In both cases I post the original image, as it came out of ACR at default, and then after my tweaking. In both cases you can see how much tonal gradation I was able to pull out of severely underexposed areas, and pretty clean at that. Had I exposed any more to the right, I would have had much more blown out highlights that would have been unrecoverable (even more unrecoverable). (Note that "modified 1" is slightly cropped on the right as compared to its original.)"

Your two examples are striking. I can't imagine the PP steps necessary to both restrain the highlight clipping obvious in the originals while, at the same time, extracting exceptional tonal spectrum (and exceptionally numerous cusps of detail) from the seemingly hopeless deeper shadow areas. I'm amazed that you've managed to restrain highlight clipping while simultaneously revealing such a plenitude of shadow detail.

And you call it "tweaking".
Masterful!
I have to think there's some skillful masking involved to so effectively deal with both ends of the dynamic range.
C'mon.....Howdjadooit?

And thanks, Kymarto, particularly for contributing to and positively expanding the scope of this conversation! It better points up the need to devote more attention to the relation of observed color distortions (in hue and/or value, and /or saturation) and their relation to the proximity of the precise clipping points of particular channels.

I hope I am not alone in frequently feeling inadequate to the task of perceiving the crucial functional aspects of exposure, particularly those that must be considered in achieving the goal of maximizing image quality via maximizing image data quality through EBTR. I'm sure O'm not alone in wishing for participation by some of the originators of the conversation on ETTR and EBTR

In the course of discussing various aspects of EBTR I've encountered much inconsistency in terminology. The most inconsistent terminology seems to deal with the color/chromatic "distortions", "infidelities","warping", "aberrancies" and "shifts" and "dyschromasias" ( with apologies to medical lexicologists), that occur in the bright end of the DR associated with augmentation by EBTR and proximity to the onset of - or recently passed threshold of - highlight clipping.
Although attempting to standardize terminology in any realm is fraught with difficulty, any suggestions regarding possible means of standardizing that terminology would be appreciated.

As knowledge of, interest in and discriminating use of EBTR increases, I hope that more users will participate in discussions such as this one.

Dave Graham

kymarto Loc: Portland OR and Milan Italy

Hi Dave.

I don't do all linear processing. I start with ACR, but then I also like the shadows/highlights function of PS, in which choosing the parameters is quite important to getting a pleasing look. I usually don't do masking, but I am not above doing dodging/burning or using the ACR adjustment brush.

I do a lot of HDR these days. Go have a peek at my galleries on toby-marshall.com

kymarto Loc: Portland OR and Milan Italy

Sorry -- mistake :(

Uuglypher Loc: South Dakota (East River)

Hi, Toby,

I understand better our different experience with the color disturbances associated with the brightest regions and brightest stages of image file processing. that is why I do absolutely no tonal or chromatic processing in a non-linear environment and am almost fanatically insistant on not adding the accurately determined ERADR until the accurate JPEG FILE hoghlight clipping exposure has been identified....and, truth to tell, rarely if ever use the last 1/3 stop of ERADR!

If you do decide to go through the ERADR determination drill with your D800, I'd sure like to know the results...and I'd be glad to have a look at your exposure series to see how close we agree if you'd like.

I've spent some enjoyable and instructive time on your website, Toby. It's really impressive.
Thanks again for your patient, perceptive, and thoughtful participation in this thread. I'm certain I'm not the only one of the well-over 2000 viewers this thread has had to have benefitted.

Best regards,
Dave Graham

Apaflo Loc: Anchorage, Alaska

It is virtually impossible to edit images in a non-linear environment!

When a RAW converter demoisaics the raw sensor data it generates an RGB image that is gamma corrected (non-linear). That is what you edit, not the raw sensor data that is linear. It is possible to save the RGB data to a TIFF file and then use something like ImageMagick tools to convert it to linear data, but even then an editor may change it silently.

Whatever, differences between the headroom as you are measuring it cannot be attributed to the camera, as such. It's JPEG image generation that is different, due to configuration. Different White Balance, saturation, brightness or contrast (or Picture Controls) change it. But worse than that, it changes with the colors of the scene being shot! That means simply that what you are measuring might be right for setting up one shot, but wrong for the next shot of a different scene.

The only meaningful way you could measure it would be if the camera is set up with UniWB and the right color correction filter (to get 1.000 for all WB channel multipliers). That would make the headroom constant, but of course all of the camera generated JPEG images would be greenish looking too.

As long as the histogram is generated from an image using any white balance with multipliers other than 1.000 it will not be an accurate representation of the raw sensor data, and there is no simple way to calibrate that does not use at least corrections for each color intensity and for each multiplier (six variables).