Better alternatives to Gemini II for duplicate finding?

Howdy all.

Experimented with Gemini II via Setapp. … It helped me find and move some duplicate photos & files (not deleting at this moment, because I am uncertain about the files portion…).

One failure: It did not tag photos where I have one version as a .jpg and one IDENTICAL version as an .heic file.

This is the mixed and (a little bit) “messed up” Dropbox folder labeled “Camera Uploads.”

One snapshot, I have a photo as a .jpg at 6.7 MB and the same photo as an .heic at 3.3 MB. I’d like to delete the .jpg.

It’s not as easy as “sorting by type and deleting a chunk of JPGs” because there are some JPGs in there without a corresponding .heic file.

I spent some time physically highlighting the .jpgs with duplicates and (CTL+selecting the others) grabbing just the intermittent JPGs and deleting them. … BUT there has to be a way to automate this.

Anyone have any tips?

Is there a more modern utility than Gemini II? I don’t think it can properly “see” the .heic files (I may be wrong, but this newer file format might not be in its coded ability to “see”).

Thanks!

–Tim

Have you tried Duplicate Annhilator?
https://brattoo.com/propaganda/

1 Like

I’ve been disappointed with Gemini II as well for finding duplicate files in general. I used it to try to prune a database folder of PDFs in a Papers3 library before I migrated it to Bookends. I found that, after running Gemini II, I still had duplicate citation attachments listed in the Papers3 library. Some of those duplicates were split across sub-folders in the top level. But then, Gemini II found other duplicates that were split the same way. So I was not sure why it failed in some cases and it succeeded in others.

In summary and by analogy, I concluded that Gemini II was only as useful as a spray down of your car to remove the top layer of dirt after having taken a long trip on the back roads. When you really want to get everything including the wheels clean, you need to roll up your sleeves and do the work manually.


JJW

2 Likes

Yeah, Gemini no longer washes my car very well either :slightly_smiling_face:
I’m not sure what their comparison criteria are or if they work correctly, but don’t have much confidence in it. I wind up triple checking everything anyway.

2 Likes

FWIW - 2-3 years ago (I think) I tried almost all of the duplicate-finding apps that I could find, I would up satisfied with none of them, but Gemini 2 was the best of the lot.

Which was not, unfortunately, saying a lot.

This seems like something computers should be able to do better and faster than people, but so far I haven’t found the app that proves that theory.

3 Likes

Sorry for reviving an old thread, but since I’ve been looking for a better alternative to Gemini II lately and have partially succeeded I figured I’d finally join the forum (I’ve been lurking for a while) to share my findings…

The bad news is that I’ve still not found a better general-purpose deduplication tool than Gemini. That sucks because it’s pretty poor at finding all duplicates, especially when comparing large folders. I’ve found that if I compare the relevant subfolders again afterward it finds more duplicates, but that’s quite labor-intensive, and the duplicates may not be located in a similar folder structure anyway. If anyone has found a better tool by now, please share!

The good news is that specifically for photo deduplication I’ve found PhotoSweeper to be way better than anything else I’ve come across. Not sure how well known it is among the community, so sharing just in case it could help someone out as it did for me.

It’s just so much more customizable, plus it can search your entire Apple Photos library in a much better way than Gemini (works with photos not stored on your Mac as long as you’re comparing photos that look like each other, not exactly the same file—but this of course also includes exact replicas), and much more.

One challenge for me was to keep the versions of photos and videos that had the correct creation timestamps, and it made it much easier than any other tool that I’ve tried (it surfaces much more metadata about the photos when reviewing the duplicates). My brief description of it definitely doesn’t do it justice, so if you’re trying to deduplicate photos I’d recommend checking it out—it’s paid, but they have a quite limited trial that will allow you to see if it’ll work for your use case.

PhotoSweeper doesn’t only work with photos, btw, but also videos and PDFs. Unfortunately, it doesn’t work with any type of file…

1 Like

Welcome! I think it’s fair to revive this since duplicate finders aren’t a fast-moving lane with many threads. And I’ll vouch for PhotoSweeper too, as well as that dev’s other app, PhotoMill (batch image processor). Both are tiny, super polished, and powerful.

2 Likes

I would love to see the development in this thread. I too have tried many many duplication finder app for photos. I have used Gemini II and Duplicator finder (or similar name) but none has impressed me. I am going to check out Photosweeper and PhotoMill too.

Keep the discussion and suggestion coming !!

Welcome!

For comparing files and folders, Beyond Compare is beyond compare.
It has viewers, one of which is for images, that allow you to compare, but doesn’t do automated comparison of images (beyond comparing files by bytes).

3 Likes

I downloaded Photosweeper. I found that it would not delete photos on my Synology NAS (but Gemini II can). Not sure there is a setting that I have missed

Thanks for the suggestion of Beyond Compare, John! I’ll give that a try.

1 Like

Gemini II scares me – letting it make changes seems final and troublesome to undo.

Anyway, I don’t use it, because when I found Beyond Compare a few years ago I was sold. It’s like a Leatherman: lots of well made tools bundled in a solid case. It’s paid for itself many times over.

Trigger warning: people who write here about “ugly” software will hate BC – it proves the point that function over form is the right perspective

And while Beyond Compare is expensive by today’s App Store standards, it is considerably cheaper than Kaleidoscope 3.

2 Likes

Same. I feel like I have to be hyper-vigilant when sifting through its choices.

1 Like

If we’re talking about file and folder diff tools, not necessarily for keeping duplicates off your drive, I have to plug Araxis Merge. It is a very expensive app. But it is native, fast, supports 3-way comparisons, and is simply the best diff tool I have used on Mac.

In terms of diff, Kaleidoscope is pretty but simply not standout powerful. Beyond Compare is powerful and affordable but slow and lacks 3-way comparison. DeltaWalker, which I also have, is extra slow due to being Java. Unless you only need diff occasionally and get DeltaWalker on sale, or really need 3-way comparison without breaking the bank, I would actually advise against this one. Araxis Merge just hits everything out of the park. At a very steep price.

1 Like

BC has three-way merge, if that’s the same thing as 3-way comparison.
https://www.scootersoftware.com/features.php

Thank you, I was talking about actually comparing 3 files though, not merging 2 files into a third. I can’t pretend Beyond Compare isn’t great especially for the price. But it does have a few Achilles’ heels such as this one. Just putting it out there that if someone has a use case for a super powerful diff tool that would justify a high price, there’s an option out there even above Beyond Compare.

1 Like

DeltaWalker’s website is also a mess on current versions of Safari. Perhaps that means no one is at home to support the product?

Looks fine to me, Big Sur and Safari 15.1 (and Vivaldi).

I’m a long time EagleFiler user. I mainly use it these days as an archive for thousands of old files that I want to keep. But, on occasion, I will create a library just to de-duplicate PDFs and photos, etc.

It accepts all kinds of files and separates duplicates on import. Once import is complete I remove the folder containing my sorted files and discard the rest of the library (which includes the folder of duplicates it rejected).

As a test, on a couple of occasions I used Gemini II then EF on the same batch of files. EF caught duplicates G2 had missed. Since then I have only used EF.

3 Likes