Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Metadata Question: Sidecar XMP or Embedded Metadata
Dec 18, 2019 07:50:30   #
jbk224 Loc: Long Island, NY
 
I am using Capture 1 Pro; but I figure this may also apply to PhotoShop/LR users. What is your preference? Pros/Cons? I shoot RAW.
Thanks.

Reply
Dec 18, 2019 08:02:21   #
CHG_CANON Loc: the Windy City
 
Just something more to back-up. For LR, if you feared your catalog could be corrupted, you'd have the edit info in the sidecar file. But, if you think about how you could lose your catalog but not your image files and side car files all at the same time, you see it's just another set of files to back-up.

Reply
Dec 18, 2019 08:04:36   #
jbk224 Loc: Long Island, NY
 
So you are a fan of embedded metadata.

Reply
 
 
Dec 18, 2019 08:32:21   #
CHG_CANON Loc: the Windy City
 
jbk224 wrote:
So you are a fan of embedded metadata.


You might need to define your meaning of "Embedded Metadata" to clarify, because 'edit instructions' in general cannot be embedded into the RAW file. Rather, edit instructions are stored in the tool's proprietary tool / database or a sidecar file (or both). For a RAW file, the only 'embedded metadata' is data that was captured by the camera we call EXIF data. None of the 3rd party editors write more / new data into the RAW file. I believe the only exception is Canon's DPP that is specific to Canon RAW. And even then, Canon's RAW edits are readable only to DPP. Other software opening a Canon RAW see only the image data and EXIF and none of DPP's instructions. The RAW files are proprietary and universally not editable. Only the EXIF data can be modified with something like EXIFTool to update / change the EXIF data, that's literally just text in the file 'header', not the image data in the 'body'. DPP's edits are written into that 'header' area, just in a format understood only by DPP.

The side cars contain descriptors of how to modify the image. I have FastRawViewer, for example, where I can rotate an image and add star / color ratings. These XMP files are recognized when I import the image into LR. If I set LR to continue to maintain the XMP files and I then want to 'share' those edits, I have a few options:

a) I can send the file and XMP and another LR user could import these, as well as other XMP-reading software, with probably limits of what those other tools can read / understand of Adobe's XMP-embedded data.

b) I can export the edited image from LR into one of a few of transferable formats, including DNG that transfers the image and LR edits as a single importable file, again where the usefulness depends on the target software's ability to use the DNG or another format.

What I can't do is write edit-instruction data into the RAW file and transfer only the RAW file around. I'm concerned your use of "Embedded Metadata" in this discussion is misleading and inaccurate as XMP and "Embedded Metadata" are not a technical choice relative to options of sharing / maintaining the edit instructions of a RAW file.

Reply
Dec 18, 2019 08:39:42   #
jbk224 Loc: Long Island, NY
 
I think you are right about my use of terms. I need to wrap my head around this better.
Thanks.

Reply
Dec 18, 2019 08:45:06   #
bleirer
 
CHG_CANON wrote:
You might need to define your meaning of "Embedded Metadata" to clarify, because 'edit instructions' in general cannot be embedded into the RAW file. Rather, edit instructions are stored in the tool's proprietary tool / database or a sidecar file (or both). For a RAW file, the only 'embedded metadata' is data that was captured by the camera we call EXIF data. None of the 3rd party editors write more / new data into the RAW file. I believe the only exception is Canon's DPP that is specific to Canon RAW. And even then, Canon's RAW edits are readable only to DPP. Other software opening a Canon RAW see only the image data and EXIF and none of DPP's instructions. The RAW files are proprietary and universally not editable. Only the EXIF data can be modified with something like EXIFTool to update / change the EXIF data, that's literally just text in the file 'header', not the image data in the 'body'. DPP's edits are written into that 'header' area, just in a format understood only by DPP.

The side cars contain descriptors of how to modify the image. I have FastRawViewer, for example, where I can rotate an image and add star / color ratings. These XMP files are recognized when I import the image into LR. If I set LR to continue to maintain the XMP files and I then want to 'share' those edits, I have a few options:

a) I can send the file and XMP and another LR user could import these, as well as other XMP-reading software, with probably limits of what those other tools can read / understand of Adobe's XMP-embedded data.

b) I can export the edited image from LR into one of a few of transferable formats, including DNG that transfers the image and LR edits as a single importable file, again where the usefulness depends on the target software's ability to use the DNG or another format.

What I can't do is write edit-instruction data into the RAW file and transfer only the RAW file around. I'm concerned your use of "Embedded Metadata" in this discussion is misleading and inaccurate as XMP and "Embedded Metadata" are not a technical choice relative to options of sharing / maintaining the edit instructions of a RAW file.
You might need to define your meaning of "Emb... (show quote)


I saw this article, seems to argue one pro of DNG is embedded vs. Sidecar. I know little about it nor have a position other than folks here said I should stay raw so I did.

https://photographylife.com/dng-vs-raw

Reply
Dec 18, 2019 08:56:07   #
CHG_CANON Loc: the Windy City
 
bleirer wrote:
I saw this article, seems to argue one pro of DNG is embedded vs. Sidecar. I know little about it nor have a position other than folks here said I should stay raw so I did.

https://photographylife.com/dng-vs-raw


This is a different discussion with some overlap, but no direct relation to the original question about XMP files. The various tools / vendors and configuration options then intersect with various file formats; they all combine in a host of unique ways and decisions trees.

The way to cut through the forest of options and confusion is to consider, as the individual photographer: what do I want to accomplish regarding a) reuse / sharing of my edits (if at all) and b) efficient creation of back-up data and c) efficient and complete recovery from my back-up data should a disaster hit my computer harddrive, image files and edit instructions. Do have I have a complete back-up and do I know how to use it to recover? Far too many people realize, too late, there are too many "no's" in these related / overlapping questions.

Reply
 
 
Dec 18, 2019 09:00:48   #
MichaelH Loc: NorCal via Lansing, MI
 
jbk224 wrote:
I am using Capture 1 Pro; but I figure this may also apply to PhotoShop/LR users. What is your preference? Pros/Cons? I shoot RAW.
Thanks.


I do not use Capture 1 Pro so I cannot comment on that program. But in Lightroom when you hand a RAW file to Photoshop you have two choices - "Edit in Photoshop" or "Edit original". When you "Edit in Photoshop" you give Photoshop the RAW file and the Lightroom edits in a sidecar file - I do not think Photoshop can edit the Lightroom catalog. Photoshop will edit the sidecar file and hand it back to Lightroom. I use sidecar files so I do not know if Lightroom automatically creates a sidecar file if one is not there for the hand off to Photoshop but I assume that it does. Other editing programs can also use the sidecar file but none that I know of will edit the Lightroom catalog. "Edit Original" hands the unedited RAW file to Photoshop and I assume that Photoshop creates a sidecar file with its Adobe Camera Raw edits in it.

Since you are backing up your data anyway I do not see the downside to having sidecar files from a backup standpoint.

A view in favor: https://photo.stackexchange.com/questions/79552/are-there-any-downsides-to-automatically-writing-changes-to-xmp-files-in-lightro

A view against: https://fstoppers.com/post-production/most-important-setting-lightroom-set-default-8366
Note, I have no issues with Lightroom performance due to sidecar files.

A good discussion on sidecar files: https://www.lightroomqueen.com/community/threads/lightroom-catalogue-backup-vs-xmp-files.21431/

My Google search with more viewpoints: https://www.google.com/search?client=firefox-b-1-d&ei=kSr6XYSxKcGv-gT3i5WwCQ&q=lightroom+sidecar+vs+metadata+in+catalog&oq=lightroom+sidecar+vs+metadata+in+catalog&gs_l=psy-ab.3...98344.98888..99495...0.2..0.62.122.2......0....1..gws-wiz.......0i71.8n6-eqxvF70&ved=0ahUKEwjEvbXgp7_mAhXBl54KHfdFBZYQ4dUDCAo&uact=5

I use sidecar files.

And as to converting your RAW files to DNG files - if you do not keep the original RAW file in addition to the converted DNG file you loose the ability to see how other programs interpret the RAW file. Each RAW editing program has its own way of interpreting the RAW file and that is lost when you let Adobe interpret the RAW file and bake it into a DNG file.

Reply
Dec 18, 2019 09:09:15   #
SpikeW Loc: Butler PA
 
Lost me. too early in the morning, I will try to follow this again later.

Reply
Dec 18, 2019 15:41:13   #
bleirer
 
MichaelH wrote:
I do not use Capture 1 Pro so I cannot comment on that program. But in Lightroom when you hand a RAW file to Photoshop you have two choices - "Edit in Photoshop" or "Edit original". When you "Edit in Photoshop" you give Photoshop the RAW file and the Lightroom edits in a sidecar file - I do not think Photoshop can edit the Lightroom catalog. Photoshop will edit the sidecar file and hand it back to Lightroom. I use sidecar files so I do not know if Lightroom automatically creates a sidecar file if one is not there for the hand off to Photoshop but I assume that it does. Other editing programs can also use the sidecar file but none that I know of will edit the Lightroom catalog. "Edit Original" hands the unedited RAW file to Photoshop and I assume that Photoshop creates a sidecar file with its Adobe Camera Raw edits in it.

Since you are backing up your data anyway I do not see the downside to having sidecar files from a backup standpoint.

A view in favor: https://photo.stackexchange.com/questions/79552/are-there-any-downsides-to-automatically-writing-changes-to-xmp-files-in-lightro

A view against: https://fstoppers.com/post-production/most-important-setting-lightroom-set-default-8366
Note, I have no issues with Lightroom performance due to sidecar files.

A good discussion on sidecar files: https://www.lightroomqueen.com/community/threads/lightroom-catalogue-backup-vs-xmp-files.21431/

My Google search with more viewpoints: https://www.google.com/search?client=firefox-b-1-d&ei=kSr6XYSxKcGv-gT3i5WwCQ&q=lightroom+sidecar+vs+metadata+in+catalog&oq=lightroom+sidecar+vs+metadata+in+catalog&gs_l=psy-ab.3...98344.98888..99495...0.2..0.62.122.2......0....1..gws-wiz.......0i71.8n6-eqxvF70&ved=0ahUKEwjEvbXgp7_mAhXBl54KHfdFBZYQ4dUDCAo&uact=5

I use sidecar files.

And as to converting your RAW files to DNG files - if you do not keep the original RAW file in addition to the converted DNG file you loose the ability to see how other programs interpret the RAW file. Each RAW editing program has its own way of interpreting the RAW file and that is lost when you let Adobe interpret the RAW file and bake it into a DNG file.
I do not use Capture 1 Pro so I cannot comment on ... (show quote)


I thought if you edit it tiff in Photoshop, everything is written into the tiff file and there is no need for a sidecar? Admittedly out of my depth.

Reply
Dec 18, 2019 16:02:05   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
CHG_CANON wrote:
... it's just another set of files to back-up.


I keep the xmp files in the same folder as the raw files so I back up the folder, not individual files. XMP files are included automatically. I don't have to think about it.

Reply
 
 
Dec 18, 2019 16:08:58   #
CHG_CANON Loc: the Windy City
 
DirtFarmer wrote:
I keep the xmp files in the same folder as the raw files so I back up the folder, not individual files. XMP files are included automatically. I don't have to think about it.


Agreed, a rather immaterial issue in size and location, causing one to wonder why it comes up regularly for consideration.

Reply
Dec 19, 2019 08:55:21   #
TGanner Loc: Haines, Alaska
 
catalog backups will go quicker if you use the XMP file system, as LR is just needing to update text files. I prefer having all my eggs in the one basket: DNG.

Reply
Dec 19, 2019 09:00:00   #
DirtFarmer Loc: Escaped from the NYC area, back to MA
 
TGanner wrote:
catalog backups will go quicker if you use the XMP file system, as LR is just needing to update text files. I prefer having all my eggs in the one basket: DNG.


If you choose to use dng you should back up your dng files after every editing session.
If you choose to use xmp you should back up your xmp files after every editing session.

Reply
Dec 19, 2019 11:14:38   #
ronaldwrightdallas
 
there seems to be a lot of people that don't really understand this process. notice 2 completely different descriptions above. I think I am also one who doesn't know what he is talking about also. but...

there are a few fields that are only stored in the raw file, a few fields that are only stored in the catalog file. some of those fields can be stored by Lightroom in a xmp file instead of in the raw or catalog file. a few years ago it was clearly stated by adobe and others that the xmp file choice was more secure and quicker. don't have the quotes however. so I went that way.

now I am not so sure and plan on researching this again myself. y'all keep telling me how it works please.

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.