Ugly Hedgehog - Photography Forum
Home Active Topics Newest Pictures Search Login Register
Main Photography Discussion
Duplicate File Finder
Page <<first <prev 3 of 3
Jan 6, 2022 13:25:44   #
Longshadow Loc: Audubon, PA, United States
 
lonedog wrote:
Read it again. He said "Awesome duplicate finder is still listed as one of the ten best after over a decade of being out. Drawback: It does look at raw files. The program is free."


Maybe.
Does he need to look at RAW files? I don't, so it wouldn't be a drawback for me.

Reply
Jan 6, 2022 13:38:41   #
BigOldArt Loc: Seminole, FL
 
I thought I replied earlier, but do not see my reply in this thread.
in the duplicate finder in ccleaner you can include or exclude different filetypes.

What filetypes are your raw files? Filetypes were called "extensions" in 1970.

Dsn1234.jpg has a jpg filetype. jpg is a very common way to store an image from a camera, an x-ray, etc.
memo 123.txt has a txt file type. txt files have text intended to read by people.
Sample.for has an for filetype. This kind of file has FORTRAN code.
Payroll.cbl has a cbl filetype. This kind of file has COBOL code.

Whatever you use you might start in ccleaner with <tools> <Disk Analyser>



Reply
Jan 6, 2022 14:00:59   #
ChrisKet Loc: Orange, CA
 
lonedog wrote:
That sounds like a good system and would save me a lot of grief.

Have a Happy New Year and good luck with those 21 years of images!


Your process sounds very similar to mine. I started my filing system many years ago when I became the family historian. I was using Mac’s photos (or whatever it was named back then) and learned how to create folders in there. I do folders for decades, then years within the decades, then the events - named: YYYY-Event. Because I was doing this for my extended family, I also diligently face-tagged each photo and changed the dates of photos that I’d scanned, the oldest being mid-1800’s.

Fast forward to retirement, adopting the hobby of photography, taking a class to learn the camera, and being introduced to LR. My previous folder system, keyword tags, including faces, all ported nicely to LR. But, I continued to use both apps, now all new photos were going to LR, and any edited photos into Photos. Two libraries! YIKES!!!

At the beginning of the lockdown, I tackled the job and eliminated 30k duplicates. I also took additional LR courses and liked the functionality it offered. But here are a couple of things I learned along the way:

1) until it becomes ingrained and second nature to you…write down your process and refer to it frequently; this includes your definitions of the Star ratings, color codes, and the most critical key words (a post-it on my monitor worked well for me)

2) the sub-folders became more confusing, than helpful for me (I’d frequently forget to check and uncheck the show photos in sub-folders box), so I eliminated using them. Now if I need to separate .jpgs from Raw, etc. I just select file type as an attribute on the folder and limit what’s showing in my window.

Good luck on this journey…it’s a lot of work; but, very satisfying when complete! Happy New Year!!!

Reply
 
 
Jan 6, 2022 14:06:30   #
spaceytracey Loc: East Glacier Park, MT
 
Why is it a drawback to have it look @ RAW files?

Reply
Jan 6, 2022 14:12:34   #
Longshadow Loc: Audubon, PA, United States
 
spaceytracey wrote:
Why is it a drawback to have it look @ RAW files?


To whom are you directing your question?
If you click "Quote Reply", we'll know.

Reply
Jan 6, 2022 14:32:58   #
frankraney Loc: Clovis, Ca.
 
lonedog wrote:
I am going nuts here with trying to clean up my dups.

I did a search here and the latest discussion is from 2015 so a lot of software has changed since then so I am looking for any more current recommendations. I tried one that i downloaded and the dup images they showed were so small I would have had to use a magnifying glass to use it.

I want to do my LR cat as well as photos on my hard drive. I have a secondary cat in LR6 but main photos are in the subscription LR. I STILL can't figure out the naming convention of all the LR variations. It is totally confusing to me.

Any suggestions would be much appreciated and yes, my workflow has improved over the years in an attempt to reduce this problem.

Thanks, Kathy
I am going nuts here with trying to clean up my du... (show quote)


Someone already mentioned awesome duplicate finder. This is a stand alone, and could cause problems of you are using light room. Instead use a plug in for lightroom.

Here's is one.... I'm not at home and do more remember the one I use. But you can do a search for duplicate finder plugin for lightroom classic and get more.

https://www.mindgems.com/article/lightroom-duplicate-finder-delete-duplicate-photos/

Reply
Jan 6, 2022 18:06:57   #
KenY Loc: Glenside, Pa
 
Rongnongno wrote:
Note:
A Simple duplicate file search already exists on Windows, if you use MS.


Can you elaborate on how to find dupes with apps already in Windows?

Thanks.

Reply
 
 
Jan 6, 2022 18:09:29   #
PhotogHobbyist Loc: Bradford, PA
 
BigOldArt wrote:
CCleaner is free and easy to use. You may want to do this is phases
1) "perfectly identical" not need to eyeball differences
Then when you might want to eyeball pairs in the next steps.
2) same name and size in bytes
3) same in name
start it
<tools>
<duplicate finder>
Then check Name, Size, and Content. IIRC Content does something like a Checksum so if 1 bit in 1 byte is changed there will not be a match.

Then go down the page and specify paths and filetypes to include and exclude.
CCleaner is free and easy to use. You may want to... (show quote)


Does CCleaner search for and remove duplicate photos? I've only used the free version and understand it simply removes unnecessary or unused repetitive web site cookies and other things that slow down the computer. I use it about once a month for that purpose.

Reply
Jan 6, 2022 19:01:27   #
frankraney Loc: Clovis, Ca.
 
Here is the link to the lightroom duplicate finder

https://www.mindgems.com/article/lightroom-duplicate-finder-delete-duplicate-photos/#:~:text=%20Steps%20to%20Find%20Duplicate%20Photos%20in%20Lightroom,operation%20and%20press%20the%20%E2%80%9CPerform%E2%80%9D%20button%20More%20

Reply
Jan 6, 2022 20:05:37   #
rkaminer Loc: New York, NY
 
Rongnongno wrote:
Awesome duplicate finder is still listed as one of the ten best after over a decade of being out. Drawback: It does look at raw files. The program is free.

https://www.ashisoft.com/ is powerful but does not offer the flexibility of awesome file finder. It does search raw files, thought.


That program is only for Widows; any suggestions for a Mac?

Reply
Jan 7, 2022 10:28:55   #
BigOldArt Loc: Seminole, FL
 
PhotogHobbyist wrote:
Does CCleaner search for and remove duplicate photos? I've only used the free version and understand it simply removes unnecessary or unused repetitive web site cookies and other things that slow down the computer. I use it about once a month for that purpose.


The free version has a duplicate file finder under<tools>. It can be limited to look for different filetypes.
I do have a lot of photos (jpg). But I also have a lot of txt files, for example the Federalist Papers for stylometry analysis. I have a lot of statistical data files (sav), for example US Census. I usually unduplicate the different filetypes separately. Often it is useful to exclude my AppData, Documents, P R O J E C T S, U T I L I T I E S, or other major top level folders.

I guess you click <Health Check>. look down the left side for <tools>



Reply
 
 
Jan 7, 2022 14:36:47   #
lonedog Loc: DC area
 
ChrisKet wrote:
Your process sounds very similar to mine. I started my filing system many years ago when I became the family historian. I was using Mac’s photos (or whatever it was named back then) and learned how to create folders in there. I do folders for decades, then years within the decades, then the events - named: YYYY-Event. Because I was doing this for my extended family, I also diligently face-tagged each photo and changed the dates of photos that I’d scanned, the oldest being mid-1800’s.

Fast forward to retirement, adopting the hobby of photography, taking a class to learn the camera, and being introduced to LR. My previous folder system, keyword tags, including faces, all ported nicely to LR. But, I continued to use both apps, now all new photos were going to LR, and any edited photos into Photos. Two libraries! YIKES!!!

At the beginning of the lockdown, I tackled the job and eliminated 30k duplicates. I also took additional LR courses and liked the functionality it offered. But here are a couple of things I learned along the way:

1) until it becomes ingrained and second nature to you…write down your process and refer to it frequently; this includes your definitions of the Star ratings, color codes, and the most critical key words (a post-it on my monitor worked well for me)

2) the sub-folders became more confusing, than helpful for me (I’d frequently forget to check and uncheck the show photos in sub-folders box), so I eliminated using them. Now if I need to separate .jpgs from Raw, etc. I just select file type as an attribute on the folder and limit what’s showing in my window.

Good luck on this journey…it’s a lot of work; but, very satisfying when complete! Happy New Year!!!
Your process sounds very similar to mine. I start... (show quote)



Thanks for your comment. I like your thinking. I have assigned the ratings meaning and did put them on a sticky on my computer. I did it for a while but became overwhelmed, sloppy, and lazy!!! After I thin the herd I’m going to have some serious key wording and ratings work to do!!! Discipline is not one of my strong points 😂

Reply
Jan 7, 2022 14:38:00   #
lonedog Loc: DC area
 
Rongnongno wrote:
Awesome duplicate finder is still listed as one of the ten best after over a decade of being out. Drawback: It does NOT look at raw files. The program is free.

https://www.ashisoft.com/ is powerful but does not offer the flexibility of awesome file finder. It does search raw files, thought.

It should have said NOT!!!

Sorry about the confusion.
 


Thanks for the clarification.

Reply
Jan 7, 2022 14:39:55   #
lonedog Loc: DC area
 
[quote=Bogin Bob]
lonedog wrote:
I am going nuts here with trying to clean up my dups.

I have used and routinely use Teekesselchen. It takes a wee bit of use to get comfortable with it. The duplicate search creates a Collection called Duplicates marking dupes with the same keyword ... Then it's just a matter of you manually vetting the marked images and unchecking 'keepers'.

Recommendation: create a test folder and copy 10-15 images (purposely including duplicate images) from your library into it. Then highlight that folder and run Teekesselchen to see how it works (suggested so you understand dupes are not automatically deleted)

http://www.bungenstock.de/teekesselchen/
I am going nuts here with trying to clean up my du... (show quote)


Thanks. I will give this a try. It is for LR, correct?

Reply
Jan 7, 2022 15:01:49   #
lonedog Loc: DC area
 
BigOldArt wrote:
What filetype (aka extension) are your raw files?

MyPhoto001.jpg is filetype jpg
MySpreaSheet999.xlsx is filetype xlsx.


I’m not sure who you are directing your question to but having gone through several cameras over the years I have NEF(Nikon),
CR2 (canon)
ORF (Olympus)
TIFF (from topaz)
PSD (from photoshop)
DNG
HEIC (iPhone. Only 38)

You never can have too many file types 😂

Reply
Page <<first <prev 3 of 3
If you want to reply, then register here. Registration is free and your account is created instantly, so you can post right away.
Main Photography Discussion
UglyHedgehog.com - Forum
Copyright 2011-2024 Ugly Hedgehog, Inc.