Superb Duplicate File Finder, Czkawka (hiccup, Polish): I ran across this must-have utility called czkawka. It is the best duplicate file finder I've found. It searches by...

awesome14 Loc: UK

I ran across this must-have utility called czkawka. It is the best duplicate file finder I've found. It searches by cryptographic hash, so the files must be actual duplicates, not just same file name. It will also find 'similar' images and videos! It is extremely good, and fast!

On one machine with (2) 1TB NVMe drives, it found 95,000+ dupes in about 5 minutes. Random sampling revealed a statistical 100% accuracy! With similar images and videos it picked up 1 false alarm, but the images were essentially identical. I remembered the shoot, and that I shot that frame twice, because I feared camera shake the first shot.

So, it is possible to fool it. And I doubt it works with RAW formats for finding similar images. But for exact duplicates it's flawless! But don't ever trust any utility all the way. It has an option to select in several ways, such as 'all but newest'. Then you can move the dupes into image Purgatory, where they'll wait until you finally send them to deletion.

Or you can just delete based on several select criteria. And, No, 95,000 is not a typo. That machine is kind of a catchall for everything I don't feel like sorting through, but about 100GBs lighter now. The link for czkawka for Mac, Windows and Linux is: here .

Longshadow Loc: Audubon, PA, United States

Wow, I might have a dozen dupes, if that.
What am I doing wrong?

TriX Loc: Raleigh, NC

How on earth does anyone accumulate 95,000 dupes UNLESS they’re music loaded by ITunes (which is notorious for creating dupes). If they’re not MP3s or Apple Music files, I’d certainly store them somewhere rather than permanently delete them just yet.

george19

TriX wrote:

How on earth does anyone accumulate 95,000 dupes UNLESS they’re music loaded by ITunes (which is notorious for creating dupes). If they’re not MP3s or Apple Music files, I’d certainly store them somewhere rather than permanently delete them just yet.

So…my iPhone shares pictures with my iPad, and both automatically get backed up to separate folders on my server…

But…those are not my primary photographic endeavors, and don’t come close to 95k duplicates.

And at 6 Tb, I don’t really care.

Longshadow Loc: Audubon, PA, United States

george19 wrote:

So…my iPhone shares pictures with my iPad, and both automatically get backed up to separate folders on my server…

But…those are not my primary photographic endeavors, and don’t come close to 95k duplicates.

And at 6 Tb, I don’t really care.

Just because one has a bazigabyte drive is no reason to let a crapload of dupes exist.
How do they get created????

goldstar46 Loc: Tampa, Fl

awesome14 wrote:

I ran across this must-have utility called czkawka... (show quote)

goldstar46 Loc: Tampa, Fl

awesome14 wrote:

I ran across this must-have utility called czkawka... (show quote)

=======================================================

Awsome........ and others

Followup Note:
.......... Use Caution with this program

The software will find duplicates of "Everything" including maybe system files or CAB biles, or ZIP files...... ect

This software is NOT just exclusive to photographs ~~ Caution

As for me only: I will forgo the use of it for the time being.

Cheers
George Veazey
####

abc1234 Loc: Elk Grove Village, Illinois

Well, that was a waste of time. I thought I had to create a GrubHub account. When I finally found the download, it would not install (Win 11).

kvanhook Loc: Oriental, NC

I followed the link and based on what I saw I don't have the skills needed to figure out how to load and use the software. I really didn't see any instructions or explanations of whet it does or how to use it. I do have a lot of duplicates. But, I may let them live where they want to. They don't worry me.

abc1234 Loc: Elk Grove Village, Illinois

I will stick to WinMerge. Been using it for years.

TriX Loc: Raleigh, NC

In the enterprise/commercial world, where many users are accessing a server, de duplication can save a decent amount of storage and is built into many storage systems. If you REALLY need dedup (careful it isn’t “deduping” your backup or DR files), I like OpenZFS running on a Linux box. A great, mature FS that incorporates everything from RAID to Dedup to virtualization, and the open version is free.

jerryc41 Loc: Catskill Mts of NY

Good! It's always nice to find a program that is just what you want. Duplicate File Finder works for me.

jerryc41 Loc: Catskill Mts of NY

Longshadow wrote:

Wow, I might have a dozen dupes, if that.
What am I doing wrong?

Wow, back atcha. I have 201 in "My Pictures" alone. Actually, I was surprised the count was so low. I intentionally have the same picture in different folders for different reasons, so I keep those duplicates. Every now and then, I go through them and eliminate some duplicates.

Longshadow Loc: Audubon, PA, United States

jerryc41 wrote:

Wow, back atcha. I have 201 in "My Pictures" alone. Actually, I was surprised the count was so low. I intentionally have the same picture in different folders for different reasons, so I keep those duplicates. Every now and then, I go through them and eliminate some duplicates.

I don't do dupes at all.
If I change one I have to know where the other(s) is(are) located to update that one(those) also.

If I have a shot of three kids, I could put one in each of their directories, plus one for the location.

Then, how does one differentiate between intended dupes and "accidental" dupes.

Too much trouble.

DBQ49er Loc: Dubuque, IA

abc1234 wrote:

Well, that was a waste of time. I thought I had to create a GrubHub account. When I finally found the download, it would not install (Win 11).

My Windows Defender would NOT let me install. Win 10