Yasoob Khalid | Python 101: Writing a cleanup script

So hi there guys! I hope you are fine. First of all I would like to make a public apology that my English is not good so you may find some grammer mistakes. You are requested to email them to me so that I can improve my English. So what is in this post? Today we will be writing a cleanup script. The idea for this post came from Mike Driscol who recently wrote a very useful post about writing a cleanup script in python. So how is my post different from his post. So in my post I will be using path.py. When I used path.py for the first time I just fell in love with it.

Installing path.py:

So there are several ways for installing path.py. Path.py may be installed using setuptools or distribute or pip: easy_install path.py The latest release is always updated to the Python Package Index. The source code is hosted on Github.

Finding the number of files in a directory:

So our first task is to find the number of files present in a directory. In this example we will not iterate over subdirectories instead we will just count the number of files present in the top level directory. This one is simple. Here is my solution:

from path import path
num_files = 0
d = path(DIRECTORY) 
#Replace DIRECTORY with your required directory
for i in d.files():
    num_files += 1

print num_files

In this script we first of all imported the path module. Then we set the num_file variable to 0. This variable is going to keep count for the number of files in our directory. Then we call the path function with a directory name. Firthermore we iterate over the files present in the root of our directory and increment the num_files variable. Finally we print the value of num_files variable. Here is a litle bit modified version of this script which outputs the number of subdirectories present in the root of our directory.

from path import path
num_dirs = 0
d = path(DIRECTORY) 
#Replace DIRECTORY with your required directory
for i in d.dirs():
    num_dirs += 1

print num_dirs

Finding the number of files recursively in a directory:

That was easy! Wasn't it? So now our work is to find the number of files recursively in a directory. In order to acomplish this task we are given the walk() method by path.py. This is the same is os.walk(). So lets write a simple script for recursively listing all files in a directory and its subdirectories in Python.

from path import path
file_count = 0
dir_count = 0
total = 0
d = path(DIRECTORY)
#Replace DIRECTORY with your required directory
for i in d.walk():
    if i.isfile():
        file_count += 1
    elif i.isdir():
        dir_count += 1
    else:
        pass
    total += 1

print "Total number of files == {0}".format(file_count)
print "Total number of directories == {0}".format(dir_count)

That was again very easy. Now what if we want to pretty print the directory names? I know there are some terminal one-liners but here we are talking about Python only. Lets see how we can achieve that.

from path import path
d = path(DIRECTORY)
files_loc = {}
file_count = 0
dir_count = 0
total = 0
for i in d.walk():
    if i.isfile():
        if i.dirname().basename() in files_loc:
            files_loc[i.dirname().basename()].append(i.basename())
        else:
            files_loc[i.dirname().basename()] = []
            files_loc[i.dirname().basename()].append(i.basename())
        file_count += 1
    elif i.isdir():
        dir_count += 1
    else:
        pass
    total += 1

for i in files_loc:
    print "|---"+i
    for i in files_loc[i]:
        print "|   |"
        print "|   `---"+i
    print "|"

There is nothing fancy here. In this script we are just pretty printing a directory and the files it contains. Now lets continue.

Deleting a specific file from a directory:

So lets suppose we have a file called this_file_sucks.py. Now how do we delete it. Lets make this seranio more real by saying that we do not know in which directory it is placed. Its simple to solve this problem as well. Just go to the top level directory and execute this script:

from path import path
d = path(DIRECTORY)
#replace directory with your desired directory
for i in d.walk():
    if i.isfile():
        if i.name == 'php.py':
            i.remove()

In the above script I did not implement any logging and error handling. That is left as an exercise for the reader.

Deleting files based on their size:

So another interesting scenario. What if we want to delete those files which exceed 5Mb size? NOTE: There is a difference between Mb and MB. I will be covering Mb here.
Is it possible with path.py? Yes it is! So here is a script which does this work:

from path import path
d = path('./')
del_size = 
for i in d.walk():
    if i.isfile():
        if i.size > 4522420:
        #4522420 is approximately equal to 4.1Mb
        #Change it to your desired size
            i.remove()

So we saw how we can remove files based on their size but wait! It is really difficult to do conversion to bytes like this. Here is a handy function I found lying on the internet which can do these conversions for you.

def bytesto(bytes, to, bsize=1024):
    """convert bytes to megabytes, etc.
       sample code:
           print('mb= ' + str(bytesto(314575262000000, 'm')))

       sample output: 
           mb= 300002347.946
    """

    a = {'k' : 1, 'm': 2, 'g' : 3, 't' : 4, 'p' : 5, 'e' : 6 }
    r = float(bytes)
    for i in range(a[to]):
        r = r / bsize

    return(r)

Now if we want to remove files larger than 4 Mb we would do:

d = path('./')
del_size = 
for i in d.walk():
    if i.isfile():
        if i.size > 4522420:
        #4522420 is approximately equal to 4.1Mb
            i.remove()