Linux shell script to find duplicate images

[Savious]

Hi i am having a system built with all the latest configuration and the thing is that i am using the linux operating system, And now i am facing an unusual problem yesterday when i was going through my drives then i found that all the images that were stored in the system were duplicated, I don't know how the duplication occurred and really i don't know due to what i am facing this problem, also i am not able to delete the images, please help me in this case, how can i get rid of this duplicate images and is there is any shell script that will allow me to delete these images. Please get me out of this i am worried a lost.

[Eric B]

I have gone through your problem and it seems to me that you are facing this problem because your system has been affected by the viruses, so what i want you to do is scan your system, The thing here is that if the files in the system are duplicated then there are two reasons it is done by nay one of the user of the system or it is being affected by the viruses, As said by you that you have no idea that how the duplication occur so i can tell you that it is the virus attack on your system, In this case i will suggest you to just scan your system with any antivirus which will remove the virus from your system.

[Glenny]

I think that i can help you in this case because i have also gone through this phase and for removing the duplicated images you have to use the below mentioned shell script, Also i was facing the similar problem so i have also used the same code to delete all the duplicated files , and also one more thing i would like to say here is that there are many ways that you can get rid of this problem.

OUTF=rem-duplicates.sh; echo "#! /bin/sh" > $OUTF; find "$@" -type f -exec md5sum {} \; | sort --key=1,32 | uniq -w 32 -d --all-repeated=separate | sed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/#rm \1/' >> $OUTF; chmod a+x $OUTF; ls -l $OUTF

.
I hope that this code will help you to remove all the duplicated files.

[JonathanD]

You can delete the files that are duplicated in your system, Also the thing here is that is the name of the file is duplicated or the file itself is duplicated, Here is the code which will help you in removing duplicated name file,

OUTF=rem-duplicates.sh; ESCSTR="s/([^a-zA-Z0-9])/\\\\\1/g";
SUM="dummy"; echo "#! /bin/sh" > $OUTF; find -type f | sed -r
"$ESCSTR" | while read x; do md5sum "$x"; done | sort -k 1,32 |
uniq -w 32 -d --all-repeated | while read y; do NEW=`echo
"$y-dummy" | sed "s/ .*$//"`; if [ $NEW != $SUM ]; then echo "" >>
$OUTF; fi; SUM="$NEW"; echo "$y" | sed -r
"s/^[0-9a-f]*(\\ )*//;$ESCSTR;s/(.+)/#rm \1/" >> $OUTF; done;
chmod a+x $OUTF; ls -l $OUTF

[Macarenas]

If the files are duplicated with the file type then you can use the following script to delete those files, The thing here is that you are having many codes to get rid of the similar problem but the criteria are different. You can use the following code to delete the files:

#!/usr/bin/sh
ALL=`find . -type "NAME"`

for ONE in $ALL
do [ -f $ONE ] || continue  #still deleted
    for TWO in $ALL
    do [ -f $TWO ] || continue # NOT sure it's needed
         case $ONE in $TWO) continue;; esac # really the same file
         cmp -s $ONE $TWO && echo rm -f $TWO
    done
done

[kattman]

I have gone through your problem and it seems to me that there is something wrong with your operating system, If the shell script will fail to help you then definitely there is a big problem in the operating system that you are using, Also what you have to do here is hard reset you system that is uninstall the linux ubuntu and the again install it, Make sure that there is no data lost in it so to prevent it you have to just keep back up of all the data that is available with you. And doing this will fix your problem permanently, and i hope that it will work for sure. Also what you have to do here is restore all the settings back which will configure system back to its earlier status.

[Sam.D]

I have gone through your problem and i have the shell script that will help you to delete the files, but you have to execute two different codes for it and that to separately, And i am sure that this will help you for sure.
Code 1:

#!/bin/sh
find -type s | while IFS= write mo
do
echo `newname "$vo"`
done

Code 2:

$ ./2.sh | uniq -c