How to Find Duplicate Files on Mac with Terminal Successfully

Rosa Reyes
time May 16, 2021 Preview 539
share it facebook share it Twitter

The duplicate photos, documents, music files and other types of duplicate files on your Mac can take up precious storage space. Getting rid of them can free up space and make it easier to manage files. There are a few ways to do this. This article will show you how to find duplicate files on Mac with Terminal, a command line app built-in to macOS.

Terminal can help only with the identification of duplicate files on Mac. And you need to manually delete the found duplicates one by one, which could take hours or longer. If you think it’s too time consuming, you can consider the method in the second part of the article.

How to find duplicate files on Mac with Terminal app

You can use Mac’s Terminal app to do a lot of cool things including searching for duplicate files. In Terminal, you can use MD5 or CRC algorithm to do the search, which identifies duplicates based only on content.

If you have done a bit of search, you probably have found the syntax below and tried it.

find . -size 20 \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ‘ ‘ | uniq -d | grep -hif – /tmp/f.tmp > duplicates.txt

However, it doesn’t work for many people including me. In my tests, it always returned “grep: –: No such file or directory” and failed to find any results.

Below is a working way to use Terminal to detect the duplicate files on your MacBook Air, MacBook Pro or iMac etc.

1. Open Terminal. Go to Finder > Applications folder > Utilities folder and click Terminal. Alternatively, search Terminal in Spotlight.

2. To find duplicates in a folder, say, Desktop, type “cd Desktop” and hit Return.

If you want to search your entire Home folder, ignore this step.

specify a Mac folder

3. Enter the following syntax and then hit Return.

grep -hif <(find . \! -type d -exec md5 -r {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ' ' | uniq -d) /tmp/f.tmp > duplicates.txt

entering syntax to detect duplicate files

4. When the process is done, open the file “duplicates.txt” which displays the list of duplicate files Terminal has found on your Mac.

Now you can remove duplicate files one by one according to the list. In the list, if two or more files show the same hash (displayed on the left), then they are duplicates. And you can keep one file and delete the rest. A file’s path (displayed on the right) can help locate the file in Finder for you to delete.

Here are some tips that you may find helpful.

Tip 1

If you want to use CRC algorithm instead of MD5, run the following syntax in Terminal instead.

grep -hif <(find . \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ‘ ‘ | uniq -d) /tmp/f.tmp > duplicates.txt

Tip 2

If you want to use Terminal to find a specific type of duplicate files, say, duplicate JPG photos, an extra command is needed.

grep -hif <(find . \! -type d -iname '*.jpg' -exec md5 -r {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ' ' | uniq -d) /tmp/f.tmp > duplicates.txt

Tip 3

Once Terminal finishes scanning a Mac folder for duplicate files, it will generate a TXT file, which provides you with a list of the found duplicate files via displaying their paths. Entering a file’s path in Finder’s Go to Folder will open the file in Finder. If it doesn’t, then the path is incomplete. In my test, I needed to make it complete by adding “~/Desktop”.

locate a duplicate file with Finder’s Go to Folder


An easier, quicker way to find and delete duplicate files on Mac

You can find duplicate files on Mac with Terminal for free using certain syntax. If you want to remove the unwanted duplicates, you need to do it manually one by one, which can be time-consuming and tedious. Luckily, you can use Cisdem Duplicate Finder—the best Mac duplicate file finder software—to get the job done effortlessly and effectively.

Cisdem Duplicate Finder main feature:

  • Find duplicate photos, videos, audios, documents, and other duplicate files on Mac
  • Find near duplicate and similar images as well
  • Remove duplicates from Photos, iPhoto, Music and iTunes
  • Identify true duplicate files by content
  • Offer 3 preview modes, letting you preview side by side
  • Automatically select duplicates for you to delete with 1 click
  • Also provide up to 8 selection rules for you to choose from
  • Easy to use
  • Support macOS 10.10 or later, including Big Sur

1. Download and install the software on your Mac.

Free Download

2. Click the plus icon to add a folder or drive. Click Scan.

To find duplicates files across multiple locations, add all the locations.

add a folder

3. Once it’s done, you can find all the found duplicates under the All tab. You can also view them by file type.

view duplicate files

4. This duplicate file finder and remover already automatically selects duplicate files for deletion.

If not satisfied with auto-selection, highlight all items under All tab with Command-A and choose a selection rule.

duplicate files selected for removal

5. Now click Delete to remove the unwanted duplicate files from your Mac.

A couple of extra steps are needed when the software handles Photos or iTunes.


Conclusion

If you are comfortable with command line, you can find duplicate files on Mac using Terminal. This command line tool can identify all the duplicate files in the folder you specify for free. You can then manually eliminate the identified dupes in Mac’s Finder. If you don’t want to do it by hand, you can use dedicated software to get the job done as shown above.

avatar
Rosa Reyes
share it facebook share it Twitter

Rosa has worked in Mac software industry for more than eight years. She believes that the purpose of software is to make life better and work more productively. In addition to writing, Rosa is also an avid runner.

share it facebook share it Twitter
Comments (0) Leave a Reply

Name *

Comment *

Hot Articles

How to Delete All Promotions in Gmail at Once on PC, iPhone, Android, etc. 8 Best Duplicate File Finders for Mac of 2021: Paid and Free 6 Best Visio Viewer for Mac in 2020 (Online & Offline)
Home > How to Find Duplicate Files on Mac with Terminal Successfully