Ysarex Loc: St. Louis

Correct.

Joe

TriX Loc: Raleigh, NC

Yep, you can disable the in-camera sharpness and noise reduction settings (they are on by default). I actually like this in DPP. While it is limited as an editor, later versions are becoming more comprehensive, and once I tune the in-camera parameters to my liking, both raw and JPEGs look essentially alike, saving me the usual NR and sharpening in LR/PS (and the in-camera NR is pretty good). It’s also a decent sorter, although having to do the export into PS (which creates a TIFF) is an extra step. Right now, DPP sorting and raw conversion is the first step in my workflow, but I may create an appropriate preset in LR/PS and reevaluate.

Ysarex Loc: St. Louis

Absolutely. In fact you can't even make the same adjustment. Try this: Open a raw file in ACR and set the WB temp. value to 5400 degrees K. Open the image in Photoshop and ACR will convert the raw file and send an RGB image to Photoshop. Now in Photoshop access the Camera Raw Filter and add 350 degrees K to the WB value.

You can't because you no longer have access to the raw data.




I suspect you're referring to the common myth that file formats like JPEG and TIFF are image files and a raw file is not -- it's just numbers. All files stored in a computer are numbers. Ultimately they all look like this:

0110101010110101110101000111010100100001011011011011100010110110100010101010111010010101101010100010110101010101010101011101010110101101010101010001010101101010100101101011010110111011110101010110101, etc.

They all have to be decoded and a raw file can likewise be decoded. Without being demosaiced a raw file doesn't look like much but the structure is basically similar to a TIFF file and you can see the raw data as an image if you want. Below is an example. I made one concession and lightened the whole thing a little otherwise you'd have a hard time seeing anything. I of course had to reduce the size and save this as a JPEG to post here but I enlarged and inset the statue's right eye so you could see the Bayer array in place. Raw data files are image files. They're not RGB image files and that critical difference means they will not respond equally to processing methods.

Joe

PHRubin Loc: Nashville TN USA

I would never have guessed that DPP did that. (I rarely use RAW). One reason I love this forum!

gvarner Loc: Central Oregon Coast

And so I looked it up to learn more. Per Wikipedia, "Normally, the image is processed by a raw converter in a wide-gamut internal color space where precise adjustments can be made…" and then it’s converted to JPEG or TIFF or whatever format you want for the final product. So it’s initially converted to something that can be viewed and edited and then to other formats for further editing or printing.

Blenheim Orange Loc: Michigan

Thanks. DPP is the first step in my workflow, too.

Mike

Ysarex Loc: St. Louis

That last sentence could use some clarification. The raw file is opened and converted to a viewable preview for the user. We have to be able to see to process the image. Preview is the operative word here -- note that DPP even has a menu item: Preview that you can use to adjust preview parameters. While we make edits to the image the raw converter (DPP, ACR/LR, C1, Capture NX-D, et al.) makes changes to the preview image to simulate what we're going to get but what the raw converter is actually saving in the background is an instruction list or recipe. DPP saves that recipe back to the raw file but will save it to disk if you wish as a .DR4 file. At this stage in the process the raw converter is not creating and working on what will become your final image and it does not save an image if you save and close DPP.

Here's a partial DPP recipe for one of my Canon CR2 files:

ExifTool Version Number : 11.49
File Name : img_0777.dr4
Directory : C:/Users/ysare/Desktop
File Size : 4.2 kB
File Modification Date/Time : 2019:06:09 15:43:26-05:00
File Access Date/Time : 2019:06:09 15:43:26-05:00
File Creation Date/Time : 2019:06:09 15:43:26-05:00
File Permissions : rw-rw-rw-
File Type : DR4
File Type Extension : dr4
MIME Type : application/unknown
DR4 Camera Model : PowerShot G7 X Mark II
Rotation : 0
Angle Adj : 0
Work Color Space : sRGB
Raw Brightness Adj : -0.5
White Balance Adj : Manual (Click)
WB Adj Color Temp : 2500
WB Adj Magenta Green : 0
WB Adj Blue Amber : 0
WB Adj RGGB Levels : 2924 6826 6826 4239
Gamma Linear : No
Picture Style : Shot Settings
Contrast Adj : 1
Color Tone Adj : 0
Color Saturation Adj : 0
Monochrome Toning Effect : None
Monochrome Filter Effect : None
Unsharp Mask Strength : 0
Unsharp Mask Fineness : 2
Unsharp Mask Threshold : 2
Shadow Adj : 0
Highlight Adj : -2
Sharpness Adj : Unsharp Mask
Sharpness Adj On : Yes
Sharpness Strength : 0
Tone Curve Color Space : RGB
Tone Curve Shape : Curve
Tone Curve Input Range : 0 255
Tone Curve Output Range : 0 255
RGB Curve Points : (0,0) (255,255)
Tone Curve X : 255
Tone Curve Y : 255
Red Curve Points : (0,0) (255,255)
Green Curve Points : (0,0) (255,255)
Blue Curve Points : (0,0) (255,255)
Tone Curve Original : Yes
Tone Curve Brightness : 0
Tone Curve Contrast : 0
Auto Lighting Optimizer : Standard
Auto Lighting Optimizer On : No
Luminance Noise Reduction : 1
Chrominance Noise Reduction : 1
Color Moire Reduction : 2
Color Moire Reduction On : No
Shooting Distance : 100%
Peripheral Illumination : 100
Peripheral Illumination On : No
Chromatic Aberration : 100
Chromatic Aberration On : Yes
Color Blur On : Yes
Distortion Correction : 100

That's all that DPP is saving. When you decide to export a JPEG or TIFF (File -> Convert and Save) DPP runs the recipe against your raw file data and creates an output RGB file.

Joe

Blenheim Orange Loc: Michigan

All image files are converted to something that can be viewed by some program. Canon's raw files and their DPP program are proprietary and what they do is a secret. But I think it is incorrect to assume that "it’s initially converted to something that can be viewed and edited." I don't see that phrase in the Wikipedia article. The files need to be converted to some other format only so that other programs can read and display them. But there is no inherent reason why raw files could not themselves be the "final product."

Later in the article you will find this:

"Many raw file formats, including IIQ (Phase One), 3FR (Hasselblad), DCR, K25, KDC (Kodak), CRW CR2 CR3 (Canon), ERF (Epson), MEF (Mamiya), MOS (Leaf), NEF (Nikon), ORF (Olympus), PEF (Pentax), RW2 (Panasonic) and ARW, SRF, SR2 (Sony), are based on the TIFF file format."

That strongly suggests that raw files are not so radically different from other image file formats. My guess (and we can only guess) is that what DPP is displaying when you open a raw file is much more likely a TIFF file variant rather than to a JPEG file (it is most certainly is not a JPEG). Many programs can display TIFF files. Why can those programs not display a Canon raw file? Because Canon is intentionally preventing that to force you into their system, as are the other manufacturers. Maybe the manufacturers are response for all of the ideas people have a about raw files being some sort of magical "negative" that "can't be viewed" without their special software, or without extensive difficult and time consuming "processing."

And then there is this about the proprietary (and monopolistic) nature of raw formats:

"Different manufacturers use their own proprietary and typically undocumented formats, which are collectively known as raw format. Often they also change the format from one camera model to the next. Several major camera manufacturers, including Nikon, Canon and Sony, encrypt portions of the file in an attempt to prevent third-party tools from accessing them."

And then there is this about why that is a problem:

"This industry-wide situation of inconsistent formatting has concerned many photographers who worry that their valuable raw photos may someday become inaccessible, as computer operating systems and software programs become obsolete and abandoned raw formats are dropped from new software. The availability of high-quality open source software which decodes raw image formats, particularly dcraw, has helped to alleviate these concerns. An essay by Michael Reichmann and Juergen Specht stated "here are two solutions – the adoption by the camera industry of A: Public documentation of RAW [sic] formats; past, present and future, or, more likely B: Adoption of a universal RAW [sic] format". "Planning for [US] Library of Congress Collections" identifies raw-file formats as "less desirable file formats", and identifies DNG as a suggested alternative."

This statement in the Wikipedia article is misleading, I think:

"Cameras that produce raw files save these settings in the file, but defer the processing. This results in an extra step for the photographer, so raw is normally only used when additional computer processing is intended."

This is simply not true in my direct experience.

Later, under "drawbacks" there is this:

"The time taken in the image workflow is an important factor when choosing between raw and ready-to-use image formats. With modern photo editing software the additional time needed to process raw images has been greatly reduced but it still requires an extra step in workflow in comparison with using out-of-camera JPEGs."

Sure. It is an "extra step" - what, like 5 seconds? - if JPEGs are all you want. Back before digital cameras we scanned prints into a lossless format, and then sized, edited and as the very last step converted to JPEG. That was not a big deal then, and it is not a big deal now. Conversion to JPEG is the least of the worry and the quickest and easiest step imaginable. The idea that allowing the camera to convert images to JPEGs is somehow saving anyone some great amount of time or effort is just nonsense in my opinion.

In my experience, the workflow is much faster working with raw files than it was working with JPEGs. "Processing??" "Converting??" Click, done. I am not saying that this will be true for everyone, nor am I suggesting that there is no place for using JPEG files.

And then we have the benefits of working with raw files:

* Many more shades of colors compared to JPEG files - raw files have 12 or 14 bits of intensity information per channel (4096-16384 shades), compared to JPEG's gamma-compressed 8 bits (256 shades).

* Higher image quality. Because all the calculations (such as applying gamma correction, demosaicing, white balance, brightness, contrast, etc...) used to generate pixel values (in RGB format for most images) are performed in one step on the base data, the resultant pixel values will be more accurate and exhibit less posterization.

* Bypassing of undesired steps in the camera's processing, including sharpening and noise reduction

* JPEG images are typically saved using a lossy compression format (though a lossless JPEG compression is now available). Raw formats typically use lossless compression or high-quality lossy compression.

* Finer control. Raw conversion software allows users to manipulate more parameters (such as lightness, white balance, hue, saturation, etc...) and do so with greater variability. For example, the white point can be set to any value, not just discrete preset values like "daylight" or "incandescent". Furthermore, the user can typically see a preview while adjusting these parameters.

* The color space can be set to whatever is desired.

* Different demosaicing algorithms can be used, not just the one coded into the camera.

* The contents of raw files include more information, and potentially higher quality, than the converted results, in which the rendering parameters are fixed, the color gamut is clipped, and there may be quantization and compression artifacts.

* Large transformations of the data, such as increasing the exposure of a dramatically under-exposed photo, result in fewer visible artifacts when done from raw data than when done from already rendered image files. Raw data leave more scope for both corrections and artistic manipulations, without resulting in images with visible flaws such as posterization.

* All the changes made on a raw image file are non-destructive; that is, only the metadata that controls the rendering is changed to make different output versions, leaving the original data unchanged.

* To some extent, raw-format photography eliminates the need to use the HDRI technique, allowing a much better control over the mapping of the scene intensity range into the output tonal range, compared to the process of automatically mapping to JPEG or other 8-bit representation.

The article is pretty good, I think. Thanks.

Mike

Blenheim Orange Loc: Michigan

Thanks for that clear explanation.

Mike

Ysarex Loc: St. Louis

They pretty much are. Nearly all of the image file formats that we use for photos display a pixel with three different independent values -- in other words three channels per pixel. That can be R,G,B or it can be Y,Cb,Cr or even C,M,Y,K (four values) but for a pixel to be a color like human skin or navel orange etc. we need those three values. Each pixel in a raw files stores only one value. Raw files have the sensor CFA still in place for all intents and purposes.

On the previous page I was able to display a raw file that had not been demosaiced (CFA interpolated off) to make the point that the image really is there, but that's entirely a novelty. In that form the image is useless. It looks awful and can't be worked with until the demosaicing is done.



TIFF and JPEG are file formats. What DPP is displaying is an RGB image which it's not going to save as a file. It's a preview image for the user to see what they're doing with processing. When you're finished working DPP will create a thumbnail JPEG of that preview image to update the file browser. We don't know if for example under the hood the engineers have programed DPP to work with RGB preview images or YCbCr or Lab preview images. That doesn't matter to us as it shows us an RGB image on our RGB displays.




Lots of other software besides DPP will open and display CR2 and CR3 files. I used non Canon software on the previous page to open a CR2 file. I do all my processing work with Capture One which handles my Canon Cr2 files just fine. But any software that opens Canon raw files for you to examine and/or process will demosaic that file in order to show it to you. And then it will display an RGB image, not your raw data.

Joe

Blenheim Orange Loc: Michigan

Good stuff. Thanks. Very informative.

Welcome aboard.

Mike

bleirer

So when tiff and some others are said to be lossless, what is the meaning of that in this context? One can still change the color temperature for example, up or down on a 0 to 100 scale when editing a tiff file. Is something lost in IQ when you do this vs. reopening a raw file, or is it just another way to do the same thing?

bleirer

One thing i will add to the OP, if you use Lightroom you can find the picture styles from your camera under the profiles tab, near the top. Only works for raw files. Click there and then the little menu to the right to display the profiles. This is not remembering the settings you set in camera, but you can scroll through the picture styles that were available in camera and choose one. If you do nothing the default is Adobe color, which is a overall good profile. This is independent of the other sliders, but a camera style could then be saved as part of a preset and even automatically applied if desired when you import a picture from the camera.

gvarner Loc: Central Oregon Coast

When JPEG file’s are opened, then edited, then saved again, the process compresses the original data to make the file smaller, it throws some of the data away. TIFF and other types of files are not compressed on saving, thus they are called lossless. I don’t know why the creators of the JPEG process decided it was a good idea to throw data away. Maybe they thought they were doing everybody a favor because disk storage was so limited way back in the Dark Ages.

DirtFarmer Loc: Escaped from the NYC area, back to MA

I would point out that bandwidth is still limited in many situations so large image files can benefit from some compression.