selmslie Loc: Fernandina Beach, FL, USA

DXOMARK (DXO) and Photon to Photos (P2P) do a good job comparing two or more sensors if we assume that the cameras being compared were tested with the same methodology and logic. But the methodology can change over time. But no matter how it is done there are other issues.

DXO does not use the same definition of noise to compute the SNR as is does for the DR.

P2P does not address SNR at all.

P2P does not agree with the DR results found by DXO. This may begin with the definition of noise.

Both appear to be placing a lot of weight on the presence of dual gain sensors in determining DR.

Neither of them addresses the aesthetic value of noise levels in determining DR or SNR.

The following post I will include a link to a description of several of the issues I found after considerable testing of several cameras.

selmslie Loc: Fernandina Beach, FL, USA

Here are the details: Camera Dynamic Range and Signal to Noise Ratio

TriX Loc: Raleigh, NC

You should include Bill Claff (bclaff), who is a member of this forum. He runs photons to Photos and wrote the white papers. Super knowledgable and a hell of a nice guy.

Personally, I trust the P2P DR figures above those of DXO. Although I understand that DXO’s DR figures are “normalized”, from a purely electronic/digitizer perspective, there is no way that a 14 bit A/D can produce DRs of >14 bits, something that DXO reports regularly, is often repeated, and in my opinion, is not accurate.

Here’s an interesting perspective on DR vs SNR: https://www.lumenera.com/blog/understanding-dynamic-range-and-signal-to-noise-ratio-when-comparing-cameras#:~:text=The%20following%20graphically%20illustrates%20SNR,into%20account%20all%20noise%20sources.

MJPerini

I'm not entirely sure what the question is. I'm certainly grateful that were have access to DxO & P2P to compare cameras we don't own or as an aid to deciding which to buy. But things like the Aesthetic value of noise is not something that can be reliably quantified. If you find it pleasing, it is pleasing. I am not saying it is not a real question, but that a lab is probably not the best place to determine that.
I have certainly experienced differences in cameras where the quality of noise becomes a differentiating factor
But I guess what I am saying is that we all base camera selection on some combination of good lab measurements and the subjective experience that come from actual use. In my view that's a good thing.
Some lean more on the lab some more on experience,--however each of us does it is fine.

More and more I look at comparison graphs where there are lots of lines close together (Camera sensors are getting better and differences are getting smaller. So the fact that the line from camera A is a fraction higher than camera B, but tells you nothing about weather the difference is significant.
I buy cameras somewhat infrequently, and keep them for a long time. With modern cameras we are all PHOTOGRAPHER LIMITED rather than Equipment Limited. I also find I take better pictures with cameras that are second nature. That may not be true of everyone.
I mean this as a constructive response. We have all taken good pictures with bad cameras and bad pictures with good cameras, In my view the pictures are always the only measure. But I also understand if you are buying a new tool you might as well buy the most helpful one.
I hope this helps

selmslie Loc: Fernandina Beach, FL, USA

I didn't want to seem like I was coming down on DXO but you are right.

There is no way that the Z7 raw file can record a value of 16383 for the straight line version of the raw values (BL=1008) since that needs to be at least 1008 less than the BL=0 value. But even that is too high. You still need to lower that some more, maybe 65 or 130 units at base ISO to make room for one or two standard deviations (SD) that would otherwise get lost in BL=0. That means that the largest value in the BL=1008 line might be no more than about 15245 which translates to a maximum DR of about 13.9 at base ISO if there were no read noise at all. The A7 II and III are not much different with BL=512, still around 13.9.

But my pet peeve about both sites is the use of ISO for the horizontal axis. It suggests that the DR is changing because of a change in ISO. But if the exposure is reduced without changing the ISO, the read noise does not increase. You just get less signal and lower raw values. The proper way to show this is with EV or LV numbers on the horizontal axis.

And if you lower the ISO and the exposure by the same amount you end up with the same 18% gray or whatever you were aiming for. Lowering the exposure actually reduces the SD in the signal. But now the increase in gain from the higher ISO is applied to the base ISO read noise resulting in a net loss of SNR. That's what the DXO SNR plot does not make clear.

Horizontal axis is LV, not ISO

selmslie Loc: Fernandina Beach, FL, USA

You are right. Lab work and math are probably not the way to approach this.

Insufficient exposure is the major culprit in noisy images. Anyone forced to use high ISO and low exposure because of subject matter is simply going to have to try and repair the image in post processing.

But the first option is whether you really need to recover shadows in order to make an image complete. The simplest approach is to raise the black point during post processing and cover up rather than recover the shadows. A lot of clean black may be a better compositional element than muddy or noisy deep shadows.

TriX Loc: Raleigh, NC

Since I do a lot of high ISO/low light shooting, I find the data in the tabulations below the graph on P2P useful when choosing a camera body. Also there are ~20 papers available on the site, and they are very useful for those that want to know how your camera sensor/digitizer system really works under various conditions (and why).

selmslie Loc: Fernandina Beach, FL, USA

I seldom shoot in low light but I have only three camera choices. They all work pretty well up to ISO 6400.

But the three comparisons I have seen don't really tell me much I don't already know.

For good lighting and lower ISO, DR and SNR are not an issue.

DXO DR - This is the display that's really worthless.


DXO SNR


P2P DR

(Download)

selmslie Loc: Fernandina Beach, FL, USA

If we measure DR from the maximum recordable value at each ISO and subtract the read noise at that ISO, we still have a problem.

Since read values can go below 1 (and their logarithm's become negative) the DR at base ISO could go over 14 as reported by DXO.

The maximum recordable value drops by a full stop each time you double the ISO. At the same time the read noise is increasing. Eventually it reaches the same level as the raw value and the DR, as calculated, drops to 0.

DXO shows DR for the A7 III 14.61 and 8.42 at base ISO and ISO 25600 respectively. P2P reports 11.6 and 4.49 for the A7 III. The results are similar for the Z7.

But DXO and P2P do not disclose how they calculate DR.

A7 III


Z7

TriX Loc: Raleigh, NC

Here’s the index of all interactive charts, papers and investigations on the Photons to Photos site - it’s quite extensive and represents a very large body of work. I’ve only read a few of the papers, but I noticed at least one - “DR collaboration” that tells how he suggests DR is to be measured and submitted by various persons/organizations. Lacking Bill on this thread to tell us, perhaps you’ll find the information somewhere in this list: https://www.photonstophotos.net/

selmslie Loc: Fernandina Beach, FL, USA

I looked at Dynamic Range Collaboration and it talks about how to collect raw data to send him for analysis.

It does not explain what we would like to know - how he defines DR and how it relates to read noise, which he has already collected.

MJPerini

You are also right about the High ISO not being an 'automatic cure" for low light exposures and BlackPoint being under used. If you look at the work of experienced photographers you'll find countless examples of good pictures made in poor conditions by waiting for a subject to turn towards the light, or moving their own point of view, or adding light. One of my favorite quotes about Available Light Photography *as it used to be called is by Gene Smith, a true master of the genre. When asked about it (being and available light photographer) he said Of course I am, I use any damn light that is available....... he understood the you need photons on the subject. Raising the iso doesn't really do that.

TriX Loc: Raleigh, NC

But it does help utililize the entire DR of the A/D, and on dual gain sensors, it configures the sensor chain for max performance at low light.

selmslie Loc: Fernandina Beach, FL, USA

I was able to replicate DXO's version of DR with a couple of simple constants.

First, I subtracted the read noise SD (it was more consistent than the actual raw values) from the signal (in this case 13.9) that drops by exactly 1 for each doubling of the ISO (see image below). That gave me a starting value 0.24 units lower than the one at DXO. It's possible that their read noise test of the A7 III produced a different SD level at ISO 100 so that was not much of a stretch. In any case, I simply added 0.24 to my initial result.

Next, I spread the difference between the resulting end point and DXO's DR at ISO 51200 over the range from ISO 100 through 51200. This changed the ISO 51200 DR to DSO's 7.54.

The result is an almost perfect replication of the DXO plot.

The problem is that there is no legitimate explanation why this works. It's a kludge. It leaves me to wonder if DXO's ending DR is just made up.

DXO DR for the A7 III - It's fudged.

MJPerini

I’ll accept that, I’m sure you are correct , I have not yet owned a camera with a dual gain sensor.
Thanks