ubuntu:file:delete_duplicate_files
This is an old revision of the document!
Ubuntu - File - Delete duplicate files
find . -regex '.* ([0-9]).*' -delete
find "$@" -type f -print0 | xargs -0 -n1 md5sum | sort --key=1,32 | uniq -w 32 -d --all-repeated=separate | sed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/ls \1/'
Using rdfind
Rdfind stands for redundant data find; and is a free and open source utility to find duplicate files across and/or within directories and sub-directories.
It compares files based on their content, not on their file names.
Rdfind uses ranking algorithm to classify original and duplicate files. If you have two or more equal files, Rdfind is smart enough to find which is original file, and consider the rest of the files as duplicates.
Once it found the duplicates, it will report them to you. You can decide to either delete them or replace them with hard links or symbolic (soft) links.
rdfind -deleteduplicates true ~/.
ubuntu/file/delete_duplicate_files.1580567594.txt.gz ยท Last modified: 2020/07/15 09:30 (external edit)