spencehg1

First off, I have done extensive internet search and contacted XRite - no useful feedback yet.
The problem - I am doing a technical archaeology project. My ultimate goal is to have multiple photos taken on different dates of the same subject closely match in color and exposure. I am looking for change in the subjects color over time. I am taking both ambient light and strobe photographs of archaeological features over period of months. Same camera, same lens, same ISO. XRite Passport in every photo. Creating profiles using Xrite software and applying to DNG files as required. After applying profile for specific light source during each session, applying White Balance to each DNG file. Have tried using 1st, 2nd, 3rd, 4th gray patch on lower row and also central patch on landscape row.
I have confirmed that only image files with no clipping in highlights or shadows are present in areas of interest.
After all of the above, the following issues in play.
1 - White Balance in gray color patch used for WB has equal RGB values as expected. Usually all the other gray patches have RGB values that although have magnitude change as expected do not match between colors and sometimes very substantially. Other people have evidently reported this. I.E. swatch used for WB (150, 150, 150), adjacent patch (190, 200, 205)
????? Does anyone have a reasonable explanation for this. ?????
2 - After profiles are applied and whether WB applied or not, the colors on the chart are highly variable and usually do not come close to matching between photos on different dates or between light sources on the same day. I am a bit concerned that changing angle of the Godox TT660 flash may vary and therefore reflections might also vary, the ambient light photos are always in shade and are closely matched in distance, angle, ISO. Only slight variations in aperture and shutter speed.
Acquisition and processing sequence as follows
Nikon D5200, Nikon 40mm micro, similar distance and angle with Passport in photo
DNG selected based on good exposure and no clipping in highlights or shadows
Profile created and applied to DNG file
WB applied usually via 3rd or 4th swatch on 4th row but others have been tried.
????? Does anybody have any information is there is a better or different way to process or acquire the photos?????
I do have a means of selecting small areas and histogram matching different photos. When applied, the differences in color values on the Passport (and therefore the subject) are obvious and sometimes very significant. An example of the differences is that in one set of photos with the profile applied the Red patch on the 3rd row will have an RGB value (###,0,0) and another will have (###,23,18). It does not seem to matter if I apply WB prior to running profile or applying the profile.
This has nothing to do with the monitor calibration. I am working directly on the DNG files and it is the same monitor anyway. Have 3 different D5200 bodies. Problem is the same regardless of camera or lens.

Any help or explanations would be greatly appreciated.

speters Loc: Grangeville/Idaho

I just got to read through the first two or three sentences of your post and noticed right away some flaws in your undertaking. For a projsect like that it would be wisely to eliminate all ambient light and shoot with stobes or speedlights to have consistent color temps, especially if you're shooting for longer periods!

spencehg1

Thanks,
I am using both ambient light but also separately the strobe at settings where the electronic light is seriously overpowering the ambient light. Theoretically the color profiles should take care of this. I am specifically using the Xrite passport every time rather than just for general lighting conditions.

jdubu Loc: San Jose, CA

You are introducing too many lighting variables into your succession of shots. As speters has said, allowing ambient lighting can seriously alter your color results. Even shade can vary in K temperature depending on the time of day. If you cannot totally exclude ambient, at least control it as much as possible and as consistently as possible. Setting up blackout fabrics, etc. and using the same lighting and positions reduces variable effects on your project.

Once you have a profile generated for your shot, why are you messing around with white balance, especially the cooling and warming grays? Each time you generate a color profile, that brings the color chart into standard color... You are needlessly changing that profile by altering white balance. The warming and cooling chart is for artistic license to change the visual mood of a photo, the white balance neutral gray is to bring you to an acceptable WB for most purposes, but not when you are delving into minute color differences and change over time. The color profile is the more precise and correct.

If you are doing this as a true time color test, you shouldn't cavalierly be changing the parameters each step of the way.

spencehg1

Thank you,
Really good points. I am purposely doing a sequence of every sample with strobe and the subject shaded as much as possible as back up for the ambient light shots. Theoretically (at least per XRite and many reviewers) the applied profile should sort out the color differences between cameras and light sources. I do not expect perfection but our results are not reasonably close.
I agree that WB should not be necessary but virtually everyone including XRite indicate that it should be applied after the color profile is applied. They indicate that they are independent processes. This makes no sense to me but input from virtually are sources indicates it should be done. I have checked and not applying WB does not improve the situation. I never use the warming or cooling grays for WB. I have tried with no WB, 2nd, 3rd and 4th gray swatch on the bottom row (as recommended by Xrite and others). The mere fact that WB on any of the gray patches recommended results in inconsistent RGB values on the others. Sometimes minor, sometimes quite significant. This suggests to me that after color profiles applied with or without WB, there is a color cast.
There are multiple sources on this subject indicating that for product photography where "accurate" colors are required, this is the method to use. I am wrong in believing that "accurate" should also mean consistent and repeatable.
I am doing everything possible to keep equipment, settings, etc. consistent between sessions. The samples are features and pictographs on cliff walls. They are 100% unmovable. Camera always on tripod with distance to subject usually consistent within 5% or less.
I very much appreciate your comments. I do not have an option for most of the subjects to shoot at night with only strobe light source. I am open to any other suggestions. This project is being done by self funded volunteers. We can support some additions and changes but can not do many other tests with other equipment.

Thanks again - your input is appreciated. Unfortunately we think alike and for whatever reason, it does not appear to be working.

JD750 Loc: SoCal

I am sorry but it does not sound like you are taking a very scientific approach. As has been mentioned, the lighting is critical. What instrument are you using to quantify color shift? Your post implies you might be using the human eye? There are more scientific ways to quantify color shift. Just saying.

spencehg1

Thank you for commenting - Unfortunately, I do not understand your criticism. This project is an effort to identify degradation of ancient cultural resources in remote locations. Although I accept that you may not have information to help resolve the specific technical issue, your comments are exactly what we need to deal with once we resolve the problem. Any result we ultimately achieve must be seen as reasonable, repeatable and accurate. In other words via a valid scientific approach. The method we are employing is similar to that used by biologists and chemists. Usually they are working on small samples in a studio or laboratory. In our case, the samples are located in remote areas of the forest and desert. The examples can not be moved from their natural location (think scale of major solid rock cliff walls. Color spectrometers and related equipment are far out of our financial support range and normally have similar repeat-ability issues over the time periods in question. Trying to use visual comparisons via the human eye or a computer monitor clearly are unsuitable. The use of calibrated photo images is common in both industry and academic research. We are using XRite passport color cards to create color profiles, then color and histogram matching photos of the same subject taken months apart and reading RGB and luminance values of very small comparative areas (4 - 10 pixels of a 24mp image file). Using standard software such as Photoshop we can easily select small areas of a few pixels and read RGB values on a scale of 0-255. The human eye and most monitors can only differentiate values on a scale of 10s out of 255. This method, if done successfully is repeatable on a scale of 2-3 out of 255. We have a very few successes that meet these criteria and thus far I can not determine why they worked and most don't. Multiple sources indicate that this method is valid. I have to assume that I am doing something wrong but have not been able to determine what that is.
We are open to any suggestions that might move us forward.

jamesl Loc: Pennsylvania

---------
Shooting in RAW, try setting your exposure using a gray card for accuracy. Check that you have a good exposure with no clipping in the highlights or in the shadows. This has to be correct with no adjustments made to the RAW file before creating your csmera/lens profile. After the profile is created, open you images and apply the created profile to it. Use the middle square in the landscape row or else use the 2nd white square from the left on the gray scale at the bottom of the color panel to set your white balance. I would use the 2nd white square myself though in theory it shouldn't matter. The Landscape row is normally used to set a statting point then to either warm or cool the color temp to your preference. If you want accuracy you can't make creative adjustments. Once the profile has been applied and the white balance set you should have a good and repeatable accurate result. If you play around with adjustments changing the results from the profile you have defeated your purpose of being able to compare images to see changes.

spencehg1

Thank you for your comments. I have done EXACTLY what you describe. Essentially the only thing that is changing on photos taken weeks and months apart is the light. I do both ambient light and electronic flash. No modifications are being made to the DNG files prior to applying the generated profile. Any photos with clipping or uneven lighting are excluded. I normally use the 3rd gray patch on the bottom row but I have tried with the 2nd and 4th. No difference. I have several different D5200s, no difference. We have tried some other 24+mp cameras but realistically with the photos taken over the past year, we need to use the D5200 photo sets.
It is noteable that photos taken on a single day with the same light source and different exposures including some clipping usually match quite well as expected but ambient vs strobe sets each with it's own profile frequently have significant differences.
I accept that I must be doing something wrong because other people indicate that this should work but I can not determine thus far what that error(s) might be.

dbfrancy

Check out Joe Brady's you tube video on using Color checker Passport. Includes indoor studio flash portraits and outdoor landscapes. Gee, all along I thought creating profiles was all about different light conditions.

Strodav Loc: Houston, Tx

So I wanted to see if my results are similar to yours, so I grabbed my D850 with a Nikon 60mm micro lens and Passport Classic and went out to the patio and took a dozen shots in the shade then loaded them into LR Classic. Also grabbed a spectrophotometer and read the 6 white to black (gray) patches in the Lab color space (white is 1 and black is 6). Remember L is luminescence and is independent of color, a* is the green (-) red (+) axis and b is the blue (-) yellow (+) axis. A pure gray would have a L value and the a*, b* values would be close to zero.

1: 95.95, -0.55, +2.51; 2: 81.26, -0.28, -0.14; 3: 67.04, -0.48, -0.17; 4: 50.49, 0.02, 0.05; 5: 35.37, -0.19, -0.73; 6: 19.89, 0.24, -0.50.

So the white patch has a very slight yellow bias and the rest of the patches are basically neutral gray.

The uncorrected LR rgb values with temp = 5600, tint=0 as shot.
1: 94.3, 95.0, 97.2; 2: 88.3, 89.5, 93.6; 3: 79.2, 78.1, 83.5; 4: 58.3, 59.9, 65.6; 5: 36.0, 37.2, 41.8; 6: 11.9, 12.5, 15.8

So there is a definite blue tint in the uncorrected file. That's expected as the shots were taken in shade.

I used the eye dropper on the 4th gray patch and the temp went to 7200 with tint at +1 with rgb readings
1: 95.7, 95.6, 95.4; 2: 90.4, 90.4, 90.8; 3: 81.4, 80.2, 80.5; 4: 60.0, 59.7, 60.1; 5: 38.4, 38.7 38.9; 6: 12.5, 12.2, 12.5

So the LR white balance (eyedropper) made the gray patches virtually pure gray as one would hope.

My conclusion is that the colorchecker passport was well made with fairly neutral grays. The LR white balance (eyedropper) function works very well. If your passport, camera, lens, and PP software work similar to mine, then my best guess is you have a lighting problem, which is probably your biggest variable.

spencehg1

WOW, thanks for this effort. With the exception of using a spectrophotometer, I have done what you have done. I am interpreting that the values you quote are after creating and applying a profile for the camera and light. When I do the same thing, I typically get similar results with significantly improved values for the gray patches. In general pre WB I can have greater variation on some of gray patches, especially the lighter ones. I attribute this to virtually all of our samples being in direct association and surrounded by the red rocks of Sedona. Things tend to be very red when we are in shade.
Question 1 - with the tests you have done, after calibration and before or after WB do you consistently get "pure values for Red, Green and Blue. In our case, sometimes yes, frequently no. When no, sometimes they are way off. Unless I completely do not understand how the application of profiles works, the image should have consistent values for all or most of the patches and certainly for the prime colors.
Question 2 - If you do the same test with a flash unit or in direct sun, do you get similar results and most importantly, if the luminescence for a patch is the same the RGB values should be the same in comparative photos with different light sources. We are working on the basis of the color profiling being used to correct for varying light during different photo sessions. We are doing everything we reasonably can to keep all other factors identical or very similar. In a majority of our cases, the colors do not match from photos on different dates.
Comment 1 - We have 3 Xrite passports. They are all in good shape and checked. I had another version of a Macbeth card (Chinese) it had at least two patches that were way out of line. It was tossed out. I get very consistent results under controlled conditions with the 3 cards.

I am waiting for someone to tell me the XRite claims of color calibrating both equipment and light sources are not valid. Thus far, all feedback I have received is that their system does work.
Our very specific goal is to have different photos taken over periods of months or longer that once adjusted for luminescence on a specific point will have almost identical RGB values. In a majority of our samples, this is not the case. If I can not demonstrate that the values on the color checker match there is no hope on our samples.
Thanks again for you efforts and input. I am still not confidant that i am somehow doing something wrong or if the system cannot cope with the varying light conditions. My hope is that an error I have introduced is identified.

Strodav Loc: Houston, Tx

Just reading the first few sentences of your post, No, I do not use camera profiles. I calibrate my monitor, printer, and scanner, but I do not profile my cameras, which are set to Adobe RGB. Here's where my knowledge is weak. I shoot raw, so the file getting to LR contains the original sensor data, but the camera also puts instructions on how the camera thinks you should process the raw data in the file. I think they call it metadata. I do not know if LR uses the metadata to "process" the data before putting it on the screen. I believe that if you use a camera profile, you are color correcting the data with an unknown algorithm, but having a patent on a color corrector years ago, I can tell you its a non-linear operation and may be the reason only one of your gray scale patches is gray.

I'll read the rest of your post when I get a minute...

Strodav Loc: Houston, Tx

Question 1: About Red, Green, and Blue. I am assuming you mean the blue, green, red, yellow, magenta, cyan row on the colorchecker. I know they look like pure colors, but they are not according to the spectrophotometer. I get a reasonably close visual match between the colorchecker and my monitor, but it can't quite match the saturated reds and greens on the colorchecker. As far as consistency goes, I would need to run tests over weeks so see if I could get repeatable results.

I just read an article that LR uses the Adobe rgb color space in library mode, but uses ProPhoto RGB color space, which supposedly captures most of the color space cameras see, in develop mode. If you click the soft proof box you can select different color spaces and see the difference on the monitor. I have no idea how profiles are used as part of this. So if you change between library and develop, you are using 2 different color spaces. Go Figure! Just being curious I will probably run some tests to see what is really going on.

Question 2: As I mentioned previously, lighting is probably your greatest variable. Unless you can set your exact same lights up to the exact same place at the same angles at the same power and color temperature you will get different results. Light bounces around and modern cameras will catch any small changes. Depending on the area you are photographing, you might be able to use ring light, which might give you more consistent results than flash and/or strobe.

I use to work for a large commercial printer and one goal was to match product colors to the printed catalog. It's not easy and it looks like your project has an even more difficult goal to measure subtle changes in color (and probably detail?) over time. There might be another measurement device you can use. Let me think about it. BTW, has anyone else solved this problem? Archeology journals? Maybe call the premiere archeology university in the country?

Strodav Loc: Houston, Tx

If I understand correctly, I'm thinking a non-contact small area spectrophotometer might be a better way to accomplish your goals. It will give you accurate data over time as long as you keep it calibrated.

https://www.hunterlab.com/miniscan-ez-4500l-spectrophotometer.html