Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
The case of the missing NEF files
Jan 26, 2018 23:40:17   #
jdmarks64
 
Friday, 26 January. My partner (Windows 7 computer) and I were working the day shift out of the residence. The following events happened just as described. Just the facts.

Here's the story: I shoot NEF (Nikon RAW) with my D500. My usual procedure is to download my pictures from the XQD card directly into a dedicated computer folder using Adobe Bridge. Later, I visit the file and do post-processing on selected shots using Photoshop CC. All processed photos are saved as .jpeg files for sharing, publishing, etc. The original NEF shots remain in the folder, as well as those that haven't been processed. (I re-format the XQD card in the camera to "wipe" it clean for the next session.)

Lately, alarming facts have come to light. When I visited some of my previous folders (large), I found a number of NEF files missing! All the PP'd .jpeg files are there, as well a good number of the NEF files, BUT NOT ALL!

I back up my files once a year on two hard drives and a set of flash drives. Going back and checking all the backed-up folders, the same NEF files are missing from every one! Now, the missing files are from the last shots of the session. One theory: When the original folder was loaded, its capacity somehow got fixed to that size. Then when I did my post processing, the newly created .jpeg files pushed NEF files out to make room. Far fetched? Yes, but that's all I got.

Anyone out there ever had a similar experience? or a theory (half-baked or not)?

Thanks for any and all replies. John (Marks)

Reply
Jan 27, 2018 00:21:01   #
Toolking Loc: Pacific Northwest
 
Something you might try. Do a *.nef search in file explorer on each of your drives. If you set the results to details (view) you might be able to locate them if they went somewhere they shouldn't have.

Also you could look in your recycle bin depending when you last emptied it.

Good luck

Reply
Jan 27, 2018 00:31:50   #
sloscheider Loc: Minnesota
 
Assuming the drive is formatted with NTFS file system the file limit in a folder is 4,294,967,295 - not likely you'll get that many shots in a lifetime. How long ago was the backup made? Do you find the phenomena happening with newer folders/files as well as old?

I cringe at the thought of a once a year backup plan, buy two more drives and make a backup every 3 months using a different drive each time. Better yet, use 2 of the drive for weekly or monthly backups and the other two for 6 month backups - rotate each pair of drives so you can go back a year on the 6 month drives and a couple weeks or months with the shorter term drives.

Reply
 
 
Jan 27, 2018 02:11:27   #
TheShoe Loc: Lacey, WA
 
I would be paranoid enough to back up the files more frequently and keep the backups longer. It sounds like you are overwriting the backup folders without regard to contents. Pay attention to sloscheider, he appears to have spent some time figuring this out for himself.

Reply
Jan 27, 2018 04:32:35   #
Rongnongno Loc: FL
 
jdmarks64 wrote:
Friday, 26 January. My partner (Windows 7 computer) and I were working the day shift out of the residence. The following events happened just as described. Just the facts.

Here's the story: I shoot NEF (Nikon RAW) with my D500. My usual procedure is to download my pictures from the XQD card directly into a dedicated computer folder using Adobe Bridge. Later, I visit the file and do post-processing on selected shots using Photoshop CC. All processed photos are saved as .jpeg files for sharing, publishing, etc. The original NEF shots remain in the folder, as well as those that haven't been processed. (I re-format the XQD card in the camera to "wipe" it clean for the next session.)

Lately, alarming facts have come to light. When I visited some of my previous folders (large), I found a number of NEF files missing! All the PP'd .jpeg files are there, as well a good number of the NEF files, BUT NOT ALL!

I back up my files once a year on two hard drives and a set of flash drives. Going back and checking all the backed-up folders, the same NEF files are missing from every one! Now, the missing files are from the last shots of the session. One theory: When the original folder was loaded, its capacity somehow got fixed to that size. Then when I did my post processing, the newly created .jpeg files pushed NEF files out to make room. Far fetched? Yes, but that's all I got.

Anyone out there ever had a similar experience? or a theory (half-baked or not)?

Thanks for any and all replies. John (Marks)
Friday, 26 January. My partner (Windows 7 compute... (show quote)
Never mind PS CC and Bridge. (this was asking a stupid question).

Reply
Jan 27, 2018 06:29:09   #
Gene51 Loc: Yonkers, NY, now in LSD (LowerSlowerDelaware)
 
jdmarks64 wrote:
Friday, 26 January. My partner (Windows 7 computer) and I were working the day shift out of the residence. The following events happened just as described. Just the facts.

Here's the story: I shoot NEF (Nikon RAW) with my D500. My usual procedure is to download my pictures from the XQD card directly into a dedicated computer folder using Adobe Bridge. Later, I visit the file and do post-processing on selected shots using Photoshop CC. All processed photos are saved as .jpeg files for sharing, publishing, etc. The original NEF shots remain in the folder, as well as those that haven't been processed. (I re-format the XQD card in the camera to "wipe" it clean for the next session.)

Lately, alarming facts have come to light. When I visited some of my previous folders (large), I found a number of NEF files missing! All the PP'd .jpeg files are there, as well a good number of the NEF files, BUT NOT ALL!

I back up my files once a year on two hard drives and a set of flash drives. Going back and checking all the backed-up folders, the same NEF files are missing from every one! Now, the missing files are from the last shots of the session. One theory: When the original folder was loaded, its capacity somehow got fixed to that size. Then when I did my post processing, the newly created .jpeg files pushed NEF files out to make room. Far fetched? Yes, but that's all I got.

Anyone out there ever had a similar experience? or a theory (half-baked or not)?

Thanks for any and all replies. John (Marks)
Friday, 26 January. My partner (Windows 7 compute... (show quote)


I don't have an answer with regard to your missing files. However, I do have some suggestions.

1. Backup your files daily.
2. Do not rely on flash drives for long term storage.
3. Use Lightroom to manage your images. More reliable. If a file goes missing from it's catalog, the folder indicates this with a question mark, as does the individual file preview.
4. Rename files on import - eliminates the possibility of one file being overwritten by another. I use naming that pulls information from the metadata, including the date, camera serial number, event/job name and a unique to the import file counter that starts at 0001.

As a rule, I do not save jpegs. I consider them output files, generally not suitable for editing anyway. I only save original raw files and the PSD files that I use to work up an image. In LR I have a dozen or so export presets that give me a variety of jpegs for various destinations - each has unique size and quality settings. The file I send to a print lab is bigger (in pixels wide by pixels tall) and higher quality, than the file I load up to Facebook, or a photo competition, etc.

I've been using PS since 1998 and LR since 2012 - this is a new one on me.

Reply
Jan 27, 2018 09:30:50   #
Haenzel Loc: South Holland, The Netherlands
 
I mentioned in another post. Never overwrite a backup. Make sure the file system of your backup device is protective and self-healing, such as a ZFS file system.

Can't you restore your pc to a previous version? (Right-click on C-drive)

Reply
 
 
Jan 27, 2018 10:32:14   #
skibumpkin Loc: Lakes Region, Maine
 
jdmarks64 wrote:
Friday, 26 January. My partner (Windows 7 computer) and I were working the day shift out of the residence. The following events happened just as described. Just the facts.

Here's the story: I shoot NEF (Nikon RAW) with my D500. My usual procedure is to download my pictures from the XQD card directly into a dedicated computer folder using Adobe Bridge. Later, I visit the file and do post-processing on selected shots using Photoshop CC. All processed photos are saved as .jpeg files for sharing, publishing, etc. The original NEF shots remain in the folder, as well as those that haven't been processed. (I re-format the XQD card in the camera to "wipe" it clean for the next session.)

Lately, alarming facts have come to light. When I visited some of my previous folders (large), I found a number of NEF files missing! All the PP'd .jpeg files are there, as well a good number of the NEF files, BUT NOT ALL!

I back up my files once a year on two hard drives and a set of flash drives. Going back and checking all the backed-up folders, the same NEF files are missing from every one! Now, the missing files are from the last shots of the session. One theory: When the original folder was loaded, its capacity somehow got fixed to that size. Then when I did my post processing, the newly created .jpeg files pushed NEF files out to make room. Far fetched? Yes, but that's all I got.

Anyone out there ever had a similar experience? or a theory (half-baked or not)?

Thanks for any and all replies. John (Marks)
Friday, 26 January. My partner (Windows 7 compute... (show quote)


My guess is that the files are the result of performing a ‘save’ in Photoshop versus a ‘save as’. Your backup strategy is fine as long as the computer fails the day after the backup. If you are not sure that will be the case, you may want to revisit the plan as others have suggested.

Reply
Jan 27, 2018 10:46:27   #
Nikon1201
 
In LR it happened to me when I rename a file , sometimes LR cant find it . In LR the Library drop down ( find all missing photos ), try this

Reply
Jan 27, 2018 11:08:22   #
gvarner Loc: Central Oregon Coast
 
I don't use it but it sounds like you may have to tweak something in Bridge. The PC won't by itself kick files out of a folder. As a suggestion, I use this workflow: 1. Copy files from my SD card to an external drive into a folder named with the following naming convention - yyyy/mo/descriptive name, such as "2016 12 Christmas lights"; 2. Cull out the bad ones; 3. Copy this folder and it's files to my hard drive; 4. Import the files into my software program.

I use Photoshop Elements 14 with its Catalogue to keep track of and retrieve photos as needed.

Reply
Jan 27, 2018 13:19:13   #
JD750 Loc: SoCal
 
You download NEF to a folder on the computer, then you remove the card, open the NEF in the folder and and process & save-as .jpeg. Correct? But then the NEF is gone afterword? That is very odd. I could suggest several things but first I have a question(s).

First do the missing .nefs also have no corresponding .jpegs?

OR are there cases where you find the .jpg but no corresponding .nef file?

And I'm gonna add my 2 cents about backing up once a year. Doing that means if your hard drive fails on day 363 since the last backup, you will lose a year of work.

Reply
 
 
Jan 27, 2018 14:34:36   #
rehess Loc: South Bend, Indiana, USA
 
JD750 wrote:
And I'm gonna add my 2 cents about backing up once a year. Doing that means if your hard drive fails on day 363 since the last backup, you will lose a year of work.

Yes!! I'm a low volume shooter, so I've been backing up periodically, depending on how many pictures I've taken, and normally I don't delete from my SD card, so typically I have three copies of each picture - in my camera, in my desktop computer, and in my backup. After our vacation at the end of July I did a backup. Then, in late August there was a question here about formatting memory cards, so I made two complete copies of the SD card on the computer before experimenting on my card. Beginning of December, my desktop computer failed. One of these days I need to take it's disk out, but I don't have any way to read it right now. If you followed my tale, I had at least three copies of pictures taken in August ... but all of them were on that disk - none on the backup and none on the SD card.

Reply
Jan 27, 2018 16:52:58   #
1grumpybear
 
Your computer is an accident waiting to happen. At least mirror your hard drive and backup weekly if not more often. HDs are cheap enough now that I would recommend setting up a raid system. RAID 5 or even a RAID 6, with a 5 you can lose a drive and keep on working, with a 6 you can lose two and keep on working. Replace the drives and they will rebuild themselves. I use two raid 6s and over the last 10 years have lost drives but not data.

Reply
Jan 27, 2018 17:10:03   #
jdmarks64
 
Rongnongno wrote:
Never mind PS CC and Bridge. (this was asking a stupid question).


Say what?

Reply
Jan 27, 2018 17:52:56   #
sloscheider Loc: Minnesota
 
jdmarks64 wrote:
Say what?

I suspect he asked what software and then saw it was answered already?

Reply
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.