Duplicate photos: Most of the discussion has been about removing duplicate photo's. I have just the opposite need. How do I verify all the photo's that I have on my local hard drives...

Greg from Romeoville illinois Loc: Romeoville illinois

Most of the discussion has been about removing duplicate photo's. I have just the opposite need. How do I verify all the photo's that I have on my local hard drives are on both my local backup NAS and to the cloud server? All of the programs that I see out there appear to worry about removing files. I need to make sure all the files are backed up. The main problem is that on my NAS, I have them set to be able to find files based on family and on the local based on what took the picture and date.

Sidwalkastronomy Loc: New Jersey Shore

I needed a new hard drive on my desk top as it was filled so I backed it up on 2 seperate 6T hard drives and got different file sizes every time. Not sure what's being dropped or left out

CHG_CANON Loc: the Windy City

Could counting the files be a valid method? Could comparing the storage used be a valid method?

therwol Loc: USA

Sidwalkastronomy wrote:

I needed a new hard drive on my desk top as it was filled so I backed it up on 2 seperate 6T hard drives and got different file sizes every time. Not sure what's being dropped or left out

I've noticed that if I back up one external drive to another, the size of the overall backup may not exactly match that of the original. Someone can correct me if I'm wrong, but I believe that this has to do with the data being laid down differently in the sectors on the drive. The files themselves always appear to be identical.

CaltechNerd Loc: Whittier, CA, USA

I use file counts combined with a little spot checking. So I'm never 100% confident, but I am reasonably confident that I've got everything. Overall size definitely varies. I suspect therwol is right, how the files are allocated to disk sectors varies, leading to different amounts of wasted space.

CHG_CANON Loc: the Windy City

CaltechNerd wrote:

I use file counts combined with a little spot checking. So I'm never 100% confident, but I am reasonably confident that I've got everything. Overall size definitely varies. I suspect therwol is right, how the files are allocated to disk sectors varies, leading to different amounts of wasted space.

Another 'spot check' might be a count of specific file types.

If the total count matches, I'm not sure what else needs to be investigated, but you might double-check the individual counts of *.JPG and *.TIF, maybe a third if applicable.

Quixdraw Loc: x

A good many years ago I worked with very large masses of data. Though I haven't tried it, it should be possible to export detailed lists of images to Excel. Once there, the functions available will let you parse the data any way you want to. Count, compare, merge, etc., etc.

CHG_CANON Loc: the Windy City

Quixdraw wrote:

A good many years ago I worked with very large masses of data. Though I haven't tried it, it should be possible to export detailed lists of images to Excel. Once there, the functions available will let you parse the data any way you want to. Count, compare, merge, etc., etc.

Another good, relatively easy method to perform. I'd just be concerned about the total row count limit of a spreadsheet. But then again, if the rows matched through what ever the limit (and you had more files than the current 2022 limit), that is a another 'support' of the copies being in-sync, along with the other measures and successful confirmations.

Quixdraw Loc: x

You can actually directly compare very large tables of data - vlookup from one table to another, countif to get numbers by type, etc. etc. I'm a bit rusty but can think of all sorts of ways to use and check one list vs. another and total by all sorts of data points. Older versions of Excel, like mine, will do over 65 thousand rows - since 2007 over a Million given a capable computer.

CaltechNerd Loc: Whittier, CA, USA

Quixdraw wrote:

A good many years ago I worked with very large masses of data. Though I haven't tried it, it should be possible to export detailed lists of images to Excel. Once there, the functions available will let you parse the data any way you want to. Count, compare, merge, etc., etc.

Yes, it's doable, but you have to google how to get there. It's easier if you have shareware that lets you run Unix tools in windows (you're not running or simulating Unix, but you have similar tools). Google for several sources. They let you do the things that Windows should do but doesn't do. Like listing a catalog and outputting it to a file. Which you can then load into Excel. Then get DiffEngineX, a great piece of shareware that compares two Excel files or sheets and lists the differences. It was a godsend when I worked in IT, seldom use it in my blessed retirement :-) Ideal for this task.

Quixdraw Loc: x

CaltechNerd wrote:

Yes, it's doable, but you have to google how to ge... (show quote)

Thanks! Same game, sounds like you had the right tools. We had Excel operating well beyond what it should have had to do, even building fairly complex "applets" for want of a better word. I don't miss the work either!

CHG_CANON Loc: the Windy City

CaltechNerd wrote:

Yes, it's doable, but you have to google how to ge... (show quote)

Everything you describe does, in fact, exist in Windows. You can merge the two directory lists into a single Excel spreadsheet in different (or the same) tabs and let any of several Excel @functions compare all the row in one vs two.

Greg from Romeoville illinois Loc: Romeoville illinois

CHG_CANON wrote:

Everything you describe does, in fact, exist in Windows. You can merge the two directory lists into a single Excel spreadsheet in different (or the same) tabs and let any of several Excel @functions compare all the row in one vs two.

Thank you. I figured it out by going into Ms-Dos, copying all the files in my picture folders including sub-directories by first going to the main picture directory and then issuing the following command dir/b/a-do:n/s > files.xls

This gives a list of all files. Now I have to do the same to my NAS and then just do a compare function between two xls files.

LLC Loc: Ontario, Canada

The simplest way is to run SyncBack between drives. If there is difference in files - files will be copied. You can also run SyncBack in simulation mode without actual copying files. You will get summary with totals for "Files only on Source", "Files only on Destination", "Total files" as a web page. You can expand each line to see actual file names...
Zero effort....

mtino312 Loc: Pennsylvania

I use this too, set to Mirror the source - it does all the work.