Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
For Your Consideration
Practical HDR (a VERY long post)
Sep 16, 2015 22:41:43   #
kymarto Loc: Portland OR and Milan Italy
 
I've had a long road with HDR. The first time I came across the idea was in a Shutterbug magazine--must have been in the 80s. I found an article about a shoot that was done by a photographer, taking multiple exposures on film and then painstakingly masking them together to come up with something resembling HDR.

I was hooked. I had always been frustrated with the limited dynamic range of film, compared to what I could see with my eye. It was invariably: crush the shadows or blow the highlights. I would have loved to come up with what that photographer had achieved, but the film process was prohibitively complex.

Digital to the rescue! With the advent of Photomatix, the original HDR program, we suddenly had ways of tonemapping--combining the elements of different exposures in one frame. Trying this out opened up a whole new world of possibilities for me, starting around 10 years ago. Still, there were (and are) problems with the combination of exposures in HDR.

The main problem is ghosting. Things moving between frames will be rendered as partial or doubled or stuttered elements. This meant, until recently, shooting only static subjects or painstakingly retouching on a pixel level.

Another issue is "haloing". Basically the HDR program has trouble with places where dark meets light. If you ask the program to create a sharp transition at contrast edges it creates a very shallow, bas-relief type look. In order to make the image appear more natural, the radius of transition is increased, but this creates halos at edges--where light areas are dark around transitions, and dark areas light. This creates a very unnatural and unpleasing effect, and is, I think, one of the main reasons that many people don't like HDR (apart from us overdoing it).

Modern cameras have a huge dynamic range: it is literally now possible, using highlight and shadow adjustments, to create an image with the DR of a five ISO bracketed HDR using a single raw frame. Of course this solves both haloing and ghosting problems, and more and more I am ending up using this technique to increase tonal range, in lieu of HDR. However this has its limits, and once dynamic range exceeds a certain point, HDR is the only answer.

Also, even in less contrasty situations, using HDR tonemapping controls gives us much more opportunity to tweak the image to taste, including more or less tonal compression and different modes of tonal compression,. Therefore I often will shoot an HDR bracket series even if I think I will not create an HDR. This gives me both the option to make one if I think it might be good, plus I have a series of single images, of which I can choose the best exposure, should I choose to process the image as a normal digital photograph.

So when I chance upon a situation, I often shoot an auto exposure bracket series. Back at home, if I really find that I don't want to do HDR, I throw the extra frames away.

IN THE FIELD--

When shooting a series, the first question is how many frames to shoot. The general recommendation I have seen is to shoot three frames two stops apart, giving a bracket series of +-2 EV. My answer is: it depends on the dynamic range.

First of all, in a normal sunny situation, without a lot of reflective highlights, you don't even need to do a bracket series, much less in cloudy or hazy conditions. You can pull enough out of these situations in a single raw frame to cover almost all situations. I find that people who make HDRs in these conditions could generally save themselves a lot of trouble, since all that is achieved is some oversaturation and small tonal variation that could easily be done in an image editor alone.

The first situation in which I even consider HDR is in bright sun with deep shadows, and in this case +=2EV is almost always enough.

A note here: after shooting a bracket series, CHECK YOUR FRAMES. For an HDR to be successful, the lightest frame should have good detail in the deepest shadows, and most importantly, the darkest should have absolutely no highlights blown out (except a bright light or the sun). In almost all cases, I shoot a bracket series of +-3 EV, and often +-4 EV. Memory is cheap. There is nothing worse that coming home to find that there are areas of bright highlights which are blown and thus the shot is ruined. Shadows are a bit less critical. The HDR program will automatically bring up shadow values, and the worst that will happen if your lightest frame is too dark is that you will have some extra noise in the shadows. This is not ideal, but it will not kill the shot. Best is to shoot an extreme range and throw away the ends that are too light and too dark. I've lost too many good shots now to do otherwise.

There is a final point I'd like to mention, and that pertains to whether it is better to make a bracket series with fewer frames with the exposure values wider apart, or more with narrower exposure variances.First of all, you don't ever need to have steps of less than 1 EV from an exposure standpoint. 2 EV is also fine. All the HDR programs can create smooth values from the darkest to lightest values in the bracket series with exposures 2 EV apart.My Nikon has a maximum exposure step of 1 EV per frame in a bracket series. This is just totally stupid. It means to get a +-4 EV series I have to shoot 9 frames, instead of 5 with 2 EV intervals. HOWEVER, there is an advantage to shooting at 1 EV intervals, and that has to do with deghosting, which will be discussed later.

SHOULD I USE A TRIPOD?

Yes, if you can, but if you don't have one all is not lost. Modern HDR programs can automatically align images, and some shift will not be a major problem.

There is one caveat here, and it is parallax. If you have objects both close to and far from the camera, the shift on the sensor will vary--more for the close objects and less for those distant. So you will often get strange effects in an HDR shot handheld without stabilization: closer objects will be doubled or ghosted, and certain areas will appear slightly out of focus or hazy, even while overall the frame is in alignment. So do brace yourself if you can. Even a monopod is very helpful.

Use the lowest ISO possible, as HDR tends to accentuate noise. And this often means that your lightest frame is at a very slow shutter speed--another good reason to use a tripod if you can.

HOW DO I PROCESS THEM?

I'm not going to go into the actual steps, but basically an HDR program creates a 32 bit image which contains all the tonal values in the bracket series. This is the "well" from which the HDR program takes information and then maps tonal values onto a 16 or 8 bit image that can be displayed. This is usually called a radiance file, with the extension .hdr. I definitely recommend keeping these as "negatives" from which you can make different versions of the final image without recombining everything.

Note: If you want to use Lightroom or ACR to make easy HDRs, using the controls in those programs, then you will have to save a 32 bit floating point tiff, because they don't accept radiance files. PS and Lightroom also have a full HDR functionality now, and will do everything from combining to mapping. While I find the results pretty good, I don't generally use it because of deghosting limitations. It is fine if nothing is moving in the frame. Very unfortunately, Adobe will not work with .hdr or 32 bit tiffs made in other HDR programs.

WHICH PROGRAM SHOULD I USE?

I have tried them all, mostly, and I have definite preferences, though they all have slightly different functionalities and ease of use and advantages and disadvantages, so your mileage might vary. I'll go through a couple.

PHOTOMATIX
My program of choice--if I could have only one--would hands-down be Photomatix. It is by far the most versatile of all, and is the ONLY program with selective deghosting, which makes perhaps 50% of my HDRs possible. It also has a number of tonemapping algorithms, of two different varieties.

Basically there are global tonemappers and local tonemappers. A global tonemapping algorithm makes its mapping "decisions" based on the entire frame. Therefore there is no haloing at all, but it is also the most tonally-limited way to tonemap. If well done it can be very natural-looking, so it is often used in real estate shots, for instance, where one doesn't want an "artistic" look. I basically don't use this algorithm (Tone Compressor) at all. The results lack punch, and contrast tends to be low, with weak tonal differentiation in shadows and highlights (even if they are not crushed or blown out).

Local tonemappers look at adjacent pixels and groups of pixels, and tonemapping decisions are made on the basis of these relationships, as well as on the entire frame. This gives much more flexibility in normalizing diverse tonal values, as well as options for bringing out details in areas of similar tonal values. There are two local tonemappers in Photomatix, "Details Enhancer" and "Contrast Optimizer".

"Details Enhancer" was the original algorithm in Photomatix, and it is still hard to beat for certain things. It gives the best enhancement of microcontrast in shadows and highlights (called "Microsmoothing") available in any program. The grain of dark wood pops out, as does subtle shading in highlights such as clouds.

But this comes at a price. This mapper tends to have terrible haloing. Large expanses of the same tonality, such as walls of buildings and skies, have dark and light areas. Highlights can be veiled and hazy. Smooth colors can become grainy and coarse. Of course the strength of the effect can be reduced, but then the pictures seem to become increasingly flat and dull. This is not always bad for photorealism, but it is throwing away what "Details Enhancer" does best.

In version 5, Photomatix introduced "Contrast Optimizer". I find this a revelation. There is still very good tonal differentiation in the shadows and highlights, but it is smooth, and haloing is almost nonexistent. There are a number of controls to give you a very good result straight out of the program, without needing to resort to radical post processing and retouching to come up with something halfway decent. It is definitely more tame than "Details Enhancer", but gives a very pleasing result in the majority of cases, and it still has quite a range of compression and strength, from almost unnoticeable HDR to pretty full on painterly effects.

And there is another reason that Photomatix is my program of choice--"Selective Deghosting".

Many programs, including Photomatix, have an "Automatic Deghosting" option. This works by giving preference to a selected frame in the bracket series. It works on the whole frame, and you generally increase the strength until all the ghosts are gone. This comes at a price.

First, the program must deemphasize the availability of tonal values contained in the other frames, so the final result suffers--more noise, less smooth tonality, more limited dynamic range centered around the chosen frame--a generally inferior look. It also means that for moving objects, the position of all such objects in the frame is taken from a single frame. Thus if a darker frame is chosen to be the base frame for deghosting, then dark objects will have less tonality and more noise. If a lighter frame is chosen, bright objects might well be blown out.

What Photomatix does is to give you the possibility to select areas manually that are ghosted in a given image, and give you the possibility to choose a different base frame for each (it chooses by default a base frame for each selection, but often not too wisely--so you can and often should manually override).

In some of my HDRs I select 10 different areas to deghost, with a different base frame for dark areas and light areas, or to choose a particular configuration of moving objects. For instance there might be some branches blowing in the wind on a tree in the sun, so I can choose a darker frame for this, In another part someone is walking in the shade, so I can select this and choose a lighter frame. And there is a lake with waves, so a third selection can be made of this area. All the rest of the image uses the entire tonal range of all exposures, only the selected areas are weighted to the selected base frame,. This is absolutely revolutionary in terms of HDR functionality.

Photomatix also does a very good job of matching the tonality of chosen areas to areas where no deghosting is applied. The caveat is that if very dark areas are brough up, there will be more visible noise in the selected areas, and if a too-light frame is chosen, the result will be flat and greyish. But generally these problems can be completely avoided by choosing the base frame with the best exposure for a given ghosted area.

Now let's go back to the bracket series for a moment: should you shoot more frames at 1 EV intervals, or leass at 2 EV intervals?For deghosting, more is better, IMO. It is like having a faster frame rate--you have more chances to find the best position for moving objects in selective deghosting within the range (about 3 EV) where deghosting gives a good result in terms of tonality. Therefore, even if I could shoot in 2 EV intervals, I would always choose (at least with anything moving in the frame) to shoot more shots at 1 EV intervals.Photomatix also works relatively quickly, has robust alignment and a good workflow, and the broadest range of effects in any HDR program. I use it now 90% of the time.

As an added bonus, you can save your 32 bit file as a 32 bit floating point tiff. This can be directly opened in Lightroom or ACR which gives a quite different tonemapped look, in a way that is very smooth and user friendly. I often make three versions of an HDR: two in the local tonemappers in Photomatix and one in ACR (sorry I don't use LIghtroom), and then mask and blend them together selectively to get the best of all worlds.

PHOTOSHOP and LIGHTROOMI have tried them and found the new HDR functionality highly improved over earlier versions. However I don't like the automatic deghosting, although the alignment rocks. I could recommend this if you don't have stuff moving in the frame.

SNSHDR
This is a little-known program written by a lone coder in Poland. It gives extremely clean looking output that is very pleasing and mostly photorealistic, with no traces of haloing. It has some unique features, including masking and selective contrast control depending on hue, and the controls work in real time. Not only that, it is cheap, with the "Home" version (everything but batch processing) costing less than half of price of Photomatix. Its alignment and deghosting leave a lot to be desired (although I eagerly await the promised new version). The good news is that it will open radiance files made in Photomatix, so I often do this as well to see which I like best, or in order to mask them together.That pretty much concludes my list of "Likes". There are a few others, with which I have never had much success.

OLONEO PHOTO ENGINE
This gives decent results, but I don't like the deghosting and I find the controls limited and limiting.NIK HDR EFEX PRO III had high hopes for this one, and it contains some unique functionality that can be very useful--control points--where a lot of things can be adjusted in a selected area (contrast, brightness, saturation, hue, etc) before the final image is rendered. Unfortunately I find the final result to clip highlights and be a bit too hard-edged, with some haloing. The controls I find moronic, with a bunch of preset values that are not continuously variable, and some "looks" that are all limited variations on a theme. Some people really like the look, and it is decent for moderate HDRs of limited dynamic range, in my experience, but not good at all for more extreme stuff.

HDR EXPOSE 3
My biggest disappointment. It is the most sophisticated of all--with 32 bit processing at all stages, which should eliminate artifacts, and it has some useful functions. In practice I have never gotten a good result from it. To be fair, it is designed for photorealism, but I find the images too bland, without good microcontrast, and still with some haloing problems in the kind of images I make. For many people it might be perfect, but it has never pleased me and it disappoints me every time I try it again, hoping that I have missed something. Good stuff aside, I just find the actual tonemapping options too tame and too limited.


I know this is kind of hard to follow without examples. I'm happy to answer any questions or try to clarify or expand on certain points. I'm considering making a series of posts on single topics contained in here with photo examples, and perhaps we can find a way to share bracket series that we can tonemap together, trying different settings. In practice, while it is better to use raw files or uncompressed tiffs, jpgs can also be successfully used to create HDRs, so we could post jpg series and everybody could try their hand at making an HDR with them. I could give practical hints with examples that way ;)

Reply
Sep 17, 2015 14:30:24   #
Uuglypher Loc: South Dakota (East River)
 
Toby,
That's a super introduction to your intended interactive-instructive thread! I personally could not care less where it is posted...I'll follow it wherever it is.....but would it not more appropriately be posted in the Post-Processing Section, that being the Section in which one would normally look for such a thread?

Dave

Reply
Sep 17, 2015 14:48:31   #
minniev Loc: MIssissippi
 
Uuglypher wrote:
Toby,
That's a super introduction to your intended interactive-instructive thread! I personally could not care less where it is posted...I'll follow it wherever it is.....but would it not more appropriately be posted in the Post-Processing Section, that being the Section in which one would normally look for such a thread?

Dave


I for one would hope this topic isn't moved. I think the members of this section have such an interest in collaborative learning - whether it's about HDR, or textures, or ETTR, or whatever. And it seems the learning aspect is growing, something I'm excited about, particularly with the new Guest Speaker series St3v3M has announced. Perhaps a cross-link could be posted so that everyone can benefit? The discussion that grows out of this post would certainly fit the kinds of discussions we've been enjoying on this section. And I so appreciate Toby's work in putting this together.

Of course if it were re-located I'd chase it down :-)

Reply
 
 
Sep 17, 2015 22:24:09   #
MattPhox Loc: Rhode Island
 
Super interesting.

Reply
Sep 17, 2015 23:13:26   #
St3v3M Loc: 35,000 feet
 
Uuglypher wrote:
Toby,
That's a super introduction to your intended interactive-instructive thread! I personally could not care less where it is posted...I'll follow it wherever it is.....but would it not more appropriately be posted in the Post-Processing Section, that being the Section in which one would normally look for such a thread?

Dave

I'm excited the post is here, but it's a good point and to be fair I've sent the Moderators a PM to let them add a link to their section if they like! S-

Post-Processing Digital Images http://www.uglyhedgehog.com/s-116-1.html

Reply
Sep 18, 2015 05:07:48   #
Billyspad Loc: The Philippines
 
Toby Im glad your here fella. Can you clear up one point for me. I used to import Raw files into Photomatix but the version I have will not handle NEF files from a Nikon 7200.
So Im importing Jpegs. What difference do you feel this makes as Photomatix converts the files prior to producing a Radience file anyway. Im a fan of Trey Ratcliffe who according to a video he produced always shoots in jpeg for HDR work.
Would welcome your view on this.

Reply
Sep 18, 2015 09:41:39   #
kymarto Loc: Portland OR and Milan Italy
 
Billyspad wrote:
Toby Im glad your here fella. Can you clear up one point for me. I used to import Raw files into Photomatix but the version I have will not handle NEF files from a Nikon 7200.
So Im importing Jpegs. What difference do you feel this makes as Photomatix converts the files prior to producing a Radience file anyway. Im a fan of Trey Ratcliffe who according to a video he produced always shoots in jpeg for HDR work.
Would welcome your view on this.


I surprised myself by performing a raw vs jpg experiment and finding that there did not seem to be a big difference between the HDR results, even though jpgs obviously have a much more limited dynamic range than raws. I think the answer comes because while an individual jpg may have a more limited range than a raw, when you combine a bracket series into a 32 bit file you obviously are extending the dynamic range.

My guess is that you will do better with raws if you are on the edge with the bracket series--in other words if the dynamic range of your scene just slightly exceeds that of your bracket series, the extra headroom of the raw may just save the day. But if your bracket series comfortably covers the DR of the scene (be it in raw or in jpg), then the result should be similar.

It's important to realize that actually HDR programs cannot use raw files, they all have a raw converter that converts raws to tiffs, which then get combined into the .hdr file for tonemapping. In fact Photomatix recommends converting your raws to tiffs in a separate raw converter and combining those, because the built-in converter is just generic--dcraw I think.

I tried some experiements, converting raws to tiffs in PS and in DxO and using those in Photomatix, as compared to letting Photomatix do the work. There were some slight differences, but I did not feel that the Photomatix result was inferior, so usually I don't bother, but in your case, you could convert to tiff and combine those, instead of using jpgs.

And there are a couple of good reasons to do so. First, no matter what you say, jpg is a lossy format, and I really see no reason to lose anything at that early stage of the game. Second, if you convert to tiff (or dng), you have a chance to perform corrections before you combine, and there is value in that.

For instance, in PS you have automatic lens corrections, which will fix distortion and vignetting issues. You are much better off to do that before you create an HDR, rather than after. More importantly, pre-conversion offers you a chance to minimize color fringing, which is a real problem in HDR.

What happens is that chromatic aberrations--color fringes at contrast borders in images--become intensified in the HDR process. If you can eliminate them beforehand it can make a big difference.

Before the new fringing control in ACR and Lightroom, I used to run all my raw files through DxO Optics Pro and perform defringing, and then convert to dng (smaller than tiffs), and let Photomatix work with those. Now I am going to experiment with defringing in ACR and converting to tiff, and sending those to Photomatix.

FWIW, I also tried recovering highlights in the tiffs (when my darkest bracket was a bit too light), but this doesn't seem to help anything in terms of the final HDR, but by correcting chromatic aberrations and vignetting and distortions, and possibly doing a bit of presharpening, you can positively affect the outcome of the HDR from the beginning.

I haven't yet experimented with more radical corrections (like heavy sharpening, or boosting clarity or contrast), but certainly those would have an effect on the HDR. Realize too that your camera is already performing a lot of those kinds of corrections when it outputs a jpg. So if you are going to use jpgs for combining in HDR, I would suggest neutral settings to give you the flattest base from which to begin. And if your camera already corrects aberrations and vignetting in jpg, so much the better!

BTW Photomatix just did an update to v5.1. It might process raws from your camera now. The upgrade is free for v5 users, and if you are not using v5 you are missing some major improvements.

Reply
 
 
Sep 18, 2015 09:57:51   #
Pixelpixie88 Loc: Northern Minnesota
 
Once again, thank you Toby for putting this together.
I am saving it to bookmarks and it will be a very helpful learning tutorial for me!

Reply
Sep 18, 2015 23:27:21   #
Billyspad Loc: The Philippines
 
Good morning from the South Seas Toby. I do use version 5 of Photmatix but a copy that was "borrowed" from the net.
So of course a consequence of that it will not update and accept the latest NEF files. Oh the trials and tribulations of being a software pirate. It will accept Tiff or DNG.
An advantage of my old Pentax was it produced PEF or DNG files. From this new fancy Nikon which I dearly and sincerely love I shall have to convert the NEF files to an acceptable format before importing into Photomatix.
Thank you for your input.

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
For Your Consideration
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.