burkphoto Loc: High Point, NC

What post-processing does is allow a degree of tonal range compression (really, dynamic range compression of the digitized sensor data).

This is EXACTLY the same concept as used in audio recording and broadcasting, where dynamic range compression "squashes the audio" so that you can hear it above background noise in a car or noisy household, or "fit" a vocal into a dense musical mix. You are taking a linear input and reducing the maximum volume while raising the minimum volume. The same detail is there, but you are lessening the loudness or brightness differences between adjacent sound waves or pixels. Since I do both audio and image processing, I see the parallels. The processes and tools are almost totally analogous.

Yes, JPEG is just a storage medium. Its form of compression just reduces file size. Unless used to the extreme, it has no noticeable effect on dynamic range. The fact that it is storing an 8-bit file format is what limits the dynamic range.

The sensor data is digitized with between 12 and 14 bits of resolution. HOW the data is "fit into" the 8-bit range is very important. If you just slice the middle out of that range of linear response and make a file out of it, it will be very contrasty. If the camera's processor is set to apply an 'S' curve to it, much as film would, you use more of the tones in the 12-14 bit file. Post-processing will allow much more brightness range compression to taste.

'HDR' is a misnomer. The output of an HDR process is usually a print or an 8-bit file. HDR refers to the scene, not the output. It's a method for reducing scene brightness range to a reproducible brightness range. Because it uses several exposures of the same scene at different levels, the combination can record more detail at the extremes.

Getting natural-looking output is the challenge. I have seen far more BAD HDR attempts than good ones. I jokingly call bad HDR, "high definition reality," rather than "high dynamic range." It takes considerable finesse to create output from an HDR process that does not look weird. Just look at real estate photos on Zillow if you want to see a lot of cliché HDR work.

Ysarex Loc: St. Louis

Need to be careful here. Referring to 12 and 14 bit camera ADCs and 8 bit JPEGs altogether without qualification might cause someone to get the idea that you're comparing equivalents -- 12 is to 14 is to 8 as if those bits were a uniform unit of measure. You're not implying that right? For example it's possible that a camera that stores 14 bits of resolution nonetheless has a DR capture capacity less than a camera that stores 12 bits. These bit depths are not equivalent units of measure. And in fact lets use a 12 bit camera that can capture more DR than a 14 bit camera and then process the output from that 12 bit camera to include all the DR captured and store that result in an 8 bit JPEG -- we'll have captured a wider DR range and stored it (processed) in 8 bits than is possible to capture with the 14 bit camera.

As you noted in your first sentence our images undergo tonal range compression to fit an output target. It's a requirement and all of our digital photos are processed either in camera or on the computer to that target. It's in your next two sentences so let's jump there.

Yep, we have a standard output target -- usually a print or an 8-bit file. The reason we use an 8-bit file (JPEG) is because it's print/output sufficient. As such the 8-bit file isn't a limiting impediment rather it's more than adequate to the task. And that's the point.
Take digital and JPEGs and bit depths out of the discussion and let's just go back to making a print in the darkroom. The OP's original problem is as old as photography itself. We had the exact same problem with film -- unless the foreground is sunlit the DR between the foreground and sky is going to exceed what we can easily capture and print using film. Back in the day we jumped all kinds of hoops to solve the same problem -- including sandwiching multiple exposures. The limiting factor was the tonal range we could stuff into the print. Nothing's changed. We figured out long ago that 8-bits of digital storage was output/target sufficient. It still is. So that's not what's limiting the OP's attempt to better deal with high DR scenes.

burkphoto Loc: High Point, NC

1) Dynamic range of the sensor IS independent of bit depth generated from its analog signal. But manufacturers are loathe to use more or fewer bits to quantize a signal than are needed to capture the dynamic range without compressing it, at least in raw files. So there is often a rough parity between sensor dynamic range and bit depth stored in a raw file. In the early days of digital imaging, we had 10-bit processing. The bit depth increased as sensors with increasingly more dynamic range were developed, and especially as processors with faster throughput were developed.

2) An analog to digital converter can quantize output from a sensor that has 14 stops of dynamic range into a 14-bits-per-channel file without significant dynamic range compression. But if a sensor has a 12-13 stop dynamic range, a 12-bit AD converter is usually specified.

3) Yes, an 8-bit file contains all the dynamic range it is usually possible *to print.* But the original raw data contains much more uncompressed data. Maybe it’s less confusing to say the 14-bit file has finer color gradation from black to channel saturation in each color.

3) Converting a 14 bit file of tonally uncompressed data requires throwing away intermediate data points between tonal values. If you are going to have a scene brightness range of 14 stops “stuffed” into a container for about six stops, a lot of the intermediate values must be changed to the closest higher or lower value that an 8-bit file can define. The result is a simulation of reality with much less brightness range.

4) In the case of raw to JPEG in-camera conversions, how that data is converted is a function of camera menu presets and conversion algorithm design. With typical defaults, the entire dynamic range captured is not preserved, but only some middle portion of it is favored. Many cameras’ conversion defaults are awful. (As an aside, I’ll mention that Apple and other major smartphone vendors have greatly advanced the technology of in-camera raw to JPEG conversion.)

When post-processing a raw file to 8-bits per channel, a whole host of factors can be controlled by the user. The result can be a conversion that more naturally mimics the original scene by preserving detail throughout. The response curves from black to full color saturation in each channel can be bent to whatever shapes serve the intended emphasis of the photographer.

What additional bit depth CAN give you for printing is finer color gradation for a wider color gamut. Sending 16-bit output to high end inkjet printers enables better color gradation throughout, with smoother gradients where highlights transition to shadows. This requires eight or more pigment inks and the finest coated inkjet papers. It also requires a wide gamut working profile and direct color conversion from that profile to the printer/paper/ink profile in use.

Wallen Loc: Middle Earth

Your keep going back explaining an answer that is not being contested.
The issue at hand is not print/output sufficiency after processing the image.
It is getting a JPEG SOOC similar to a Post adjusted RAW

With reference to SOOC JPEG, Pulling is done because they will not be there because of JPEGS limitations.

Yes it is. Bit depth & Jpeg compression.
Bit depth is not a measurement of dynamic range, but bit depth limits how wide a dynamic range it can represent. If Jpeg compression is not lossy, i will agree with you.

Given as proof that your previous statement is false. Camera processing power do not limit the quality of Jpeg. And my exact words are "it would not always make the photo better".

Firstly, which better photo? I am speaking about in-camera HDR SOOC JPEG, you are speaking of Post processed image saved as JPEG. We are not talking about the same subject.
Second, you are again insisting a process. We are not discussing print/output sufficiency after manual post processing. The issue is how to get a high dynamic range JPEG SOOC. Dont change the subject.

Having no access to those cameras, I'll take you word for it even though I'm very much in doubt.
Save is the end of process and will not affect nor alter what a camera did beforehand.
That aside, TIFF (Tag Image File Format) is like a container file that can contain other files, meta data and even file compression. It might be that the TIFF is just storing within it the same JPEG image. If it does, why expect a difference in dynamic range?

Ysarex Loc: St. Louis

Yes, and as such 8 bit digital storage is output sufficient.

Which must be tonally compressed to then become possible *to print.* No other option exists. Nobody makes and sells ultra DR print paper that holds three more stops of tonal dynamic range.

You're identifying the limiting factor when you say; "possible *to print.*" We identified that limiting factor a very long time ago (long before you and me). And when digital showed up we determined that 8 bit storage was, as you say, possible-to-print sufficient. And so the OP is not being stymied by the fact that JPEG is an 8 bit storage format. That's the point.

burkphoto Loc: High Point, NC

The stymie is that cameras attempt to process wide bit depth sensor output to a lower bit depth file, insufficiently. I'm not arguing the idea that 8-bits is somehow insufficient for everyday prints and displays. It's fine for that.

It's the conversion from linear output to something less than linear that is the issue. One size does not fit all. You would need all the sliders in Lightroom Classic crammed into the camera to achieve something much better than current cameras generate.

Ysarex Loc: St. Louis

Agreed.

Yes, and the fact that that conversion is not optional -- it must be done. And it must be done to a standard output target (print). And that is the limiting factor, not that JPEGs are 8 bit.

Ysarex Loc: St. Louis

That's what the OP would like to do.

The 8 bit format of a JPEG and JPEG compression are not a problem that limits our ability to do that. Your assertion otherwise is incorrect. We are limited by the processing options built into the camera image processor. Both the camera image processor and our raw processing software must produce a final image to the same target output.

Below is a photo of a high dynamic range scene. The first photo on your left is the camera JPEG with all the picture controls set to default. The middle photo is also a camera JPEG (same exposure) but two picture controls were changed -- most importantly Active D lighting. The camera processor is working with the same raw data that becomes available on the computer.

The image on your right is from the same raw file that produced the camera JPEG but processed in C1 and saved directly to a JPEG.

The camera and C1 used the exact same raw data and both output directly to JPEGs. How did the JPEG the camera saved limit the DR in a way that the JPEG from C1 did not?

(Download)

rwm283main Loc: Terryville, CT

Ysarex, I think you demonstrated quite well with your last 3 photos, and how they were processed, what the camera is capable of doing. And, I believe your middle photo comes very close to what the RAW file looks like.
I’ve been following along these past few days and I have to thank everyone for commenting and contributing to my initial question. I’ve learned a lot from all of you.
I’m now experimenting with the Dynamic Range settings that the Fujifilm camera has to offer. There are two on my camera. One can do it all automatically or I can select the other which gives me some control (so far this is the one I prefer). It’s been very overcast the past couple of days so I haven’t been able to test my new settings. Saturday looks promising.
Once again, thank you all for such great information.
Keep shooting everyone.
Rich

Ysarex Loc: St. Louis

There are three things you can do to get your X-T2 to modify how it processes the scene DR to the output JPEG.
1. The film simulations have different contrast/tone curve responses. It's not a big difference but it counts. Use a lower contrast film sim like Astia.
2. You have the two adjustments Highlight tone and Shadow tone. You can pull in both ends.
3. And then there's the DR function. DR100 is the default and does nothing -- they should label it off. DR200 is low and DR400 is high. Both reduce the DR in the processed image -- DR400 is stronger.

The DR modes require that you raise the ISO above base. For DR200 you have to raise the ISO to at least ISO 400 and for DR400 to at least ISO 800. What the camera is going to do then is withhold one or two stops of ISO lightening that would normally be applied to the sensor output and so the raw file. This is intended to protect highlights from being blown out. NOTE: For many people it overprotects highlights resulting in overall underexposure -- not good. You'll want to experiment with how much +EC you can get away with.

Then with that ISO lightening withheld from the raw file the camera applies it instead via the image processor and at the same time lowers the contrast of the tone curve. End result is highlights protected and shadows lifted. For some of what you shoot it may be enough.

I fear with the X-T2 Fuji had not yet implemented a live view preview of the effect of the DR modes. You'll have to shoot them and then chimp the result if so.

profbowman Loc: Harrisonburg, VA, USA

Just to remind all of us that the jpeg version of a photo is not necessarily a second-rate format, I took Ysarex's three images and adjusted the first jpeg straigt from the camera to look like the other two. This shows that this photo also has the data of the other two because I could not add features that were not there.

A couple months ago we had a thread here showing that as humans we can see eight-bits of color depth at best and probably something closer to six bits. So, to me debating about whether 12 or 14 are better i s a useless argument. --Richard

Wallen Loc: Middle Earth

Read the OP again, "JPEG Camera setting for Optimum Dynamic Range" The OP knows how to process a RAW file and only wants camera settings so the JPEG SOOC be similar to a processed RAW.

We are not talking about our ability. Our ability to what? process RAW? That is not being discussed and except for merging, neither I am not talking about post processing an 8 bit format JPEG compressed image. Those are not a problem. Only you are stating it so and going around and around explaining that it is not a problem and is sufficient.

Agreed. In-camera processing options are limited, but processors which are blank slates is another matter.

Yes

Again i'll just trust you on the images and here is my point of view.

Picture 1 shows the dynamic range of the scene is wider than the dynamic range of JPEG
Picture 2 shows what happens when the automated processing feature of the camera is used.
Picture 3 shows what post processing adjustments can achieve.

None of those 3 increased the dynamic range of JPEG. The processing before conversion to JPEG just inserted details in places where there is little or none. Going back to converting high pitch dog whistle and bat click sounds to what we can hear, does not make our ear hearing increase its bandwidth. It just brought it to a pitch we can distinguish.

Ysarex Loc: St. Louis

And the 8 bit structure of a JPEG is sufficient to store that same target output so it's not a limiting factor. Glad we agree now.

Ysarex Loc: St. Louis

You miss the point. It's not about what we can see before the data is processed. The 12 and 14 bit raw formats are used to store the sensor output before processing. What we look at are processed images. The sensor output is linear and stored in the raw file the data is still linear which is inappropriate for viewing -- it doesn't look very good.

We use JPEGs to store processed images. You're comparing the linear data in a raw file with the processed data in a JPEG. That's an inappropriate and fallacious comparison.

The linear raw data must have a tone curve applied to be in anyway usable or viewable.

While processing the data we benefit immensely from having more data to work with. Once the raw data is processed then it's appropriate to save it as a JPEG.

The JPEG of mine that you previously modified had already been processed from the raw data. Try your stunt again with unprocessed linear data. The first image below is from a 14 bit raw file processed and saved as a JPEG.

The second image below is the linear raw data (WB & demosaiced only) saved in 8 bit as a JPEG. Good luck with the basket under the table.

(Download)


(Download)

RonDavis Loc: Chicago, IL

ALL, thanks for a GREAT DISCUSSION......I learned and confirmed a lot ...especially this point for digital: "Place the brightest diffuse highlight at the sensor threshold and click". (The opposite is true for film......which I don't shoot anymore).....

Now let's get to that "Honey Do" list! Happy Holidays!