Python nb-no:Problemløsing

Vi har sett på forskjellige deler av programmeringsspråket Python, og nå skal vi se på hvordan disse delene passer sammen ved å designe og skrive et program som gjør noe nyttig. Tanken er at du skal lære hvordan du skriver et eget Python-skript.

[edit] Problemet

Problemet et "Jeg vil ha et program som lager en sikkerhetskopi av alle de viktige filene mine".

Selv om dette er et enkelt problem er det ikke nok informasjon her til at vi kan starte med å løse det. Vi trenger litt mer analyse. For eksempel, hvordan skal vi bestemme hvilke filer som skal tas sikkerhetskopi av? Hvordan skal de lagres? Hvor skal de lagres?

Etter å ha analysert problemet skikkelig, utformer vi programmet vårt. Vi lager en liste med punkter om hvordan programmet vårt skal virke. I dette tilfellet har jeg laget den følgende listen om hvordan jeg vil at det skal virke. Hvis du designer selv kommer du kanskje ikke på den samme typen analyse, siden alle har sin egen måte å gjøre ting på - og det er helt OK.

1. Filene og mappene som skal tas sikkerhetskopi av skal være skrevet i en liste.
2. Sikkerhetskopien skal være lagret i et mappe for sikkerhetskopier.
3. Filene blir sikkerhetskopiert til en .zip-fil.
4. Navnet på zip-arkivet er datoen og tiden filen blir laget.
5. Vi bruker zip-kommandoen som er tilgjengelig i alle Linux/Unix-distribusjoner. Windows-brukere kan installere fra GnuWin32-prosjektsiden og legge til C:\Program Files\GnuWin32\bin til PATH-miljøvariablen, som vi gjorde i kapitlet om installasjon for å få kommandolinjen til å kjenne igjen python-kommandoen. Legg merke til at du kan bruke alle kommandoer for å arkivere så lenge det har et kommandolinjegrensesnitt - slik at vi kan gi argumenter til det gjennom skriptet vårt.

[edit] Løsningen

Siden vi har et ganske stabilt design av programmet kan vi skrive koden, som er en implementering av løsningen vår.

#!/usr/bin/python
# Filename: backup_ver1.py
 
import os
import time
 
# 1. The files and directories to be backed up are specified in a list.
source = ['"C:\\My Documents"', 'C:\\Code']
# Notice we had to use double quotes inside the string for names with spaces in it.
 
# 2. The backup must be stored in a main backup directory
target_dir = 'E:\\Backup' # Remember to change this to what you will be using
 
# 3. The files are backed up into a zip file.
# 4. The name of the zip archive is the current date and time
target = target_dir + os.sep + time.strftime('%Y%m%d%H%M%S') + '.zip'
 
# 5. We use the zip command to put the files in a zip archive
zip_command = "zip -qr {0} {1}".format(target, ' '.join(source))
 
# Run the backup
if os.system(zip_command) == 0:
    print('Successful backup to', target)
else:
    print('Backup FAILED')

Output:

   $ python backup_ver1.py
   Successful backup to E:\Backup\20080702185040.zip

Nå er vi i testingsfasen, der vi tester om programmet vårt virker som vi ønsker. Hvis det ikke oppfører seg som vi ventet, må vi debugge' programmet; det vil si at vi fjerner bugs (feil) fra programmet.

Hvis programmet ovenfor ikke virker, legg inn en print(zip_command) rett før os.system-påkallingen og kjør programmet. Kopier og lim inn zip_command til kommandolinjen og se om den kjører ordentlig av seg selv. Hvis denne kommandoen ikke virker, sjekk zip command-manualen for å se hva som kan være feil. Hvis denne kommandoen virker, sjekk programmet ditt for å se om det passer det jeg har skrevet ovenfor.

Hvordan det virker:

You will notice how we have converted our design into code in a step-by-step manner.

We make use of the os and time modules by first importing them. Then, we specify the files and directories to be backed up in the source list. The target directory is where store all the backup files and this is specified in the target_dir variable. The name of the zip archive that we are going to create is the current date and time which we find out using the time.strftime() function. It will also have the .zip extension and will be stored in the target_dir directory.

Notice the use of os.sep variable - this gives the directory separator according to your operating system i.e. it will be '/' in Linux, Unix, it will be '\\' in Windows and ':' in Mac OS. Using os.sep instead of these characters directly will make our program portable and work across these systems.

The time.strftime() function takes a specification such as the one we have used in the above program. The %Y specification will be replaced by the year without the century. The %m specification will be replaced by the month as a decimal number between 01 and 12 and so on. The complete list of such specifications can be found in the Python Reference Manual.

We create the name of the target zip file using the addition operator which concatenates the strings i.e. it joins the two strings together and returns a new one. Then, we create a string zip_command which contains the command that we are going to execute. You can check if this command works by running it on the shell (Linux terminal or DOS prompt).

The zip command that we are using has some options and parameters passed. The -q option is used to indicate that the zip command should work quietly. The -r option specifies that the zip command should work recursively for directories i.e. it should include all the subdirectories and files. The two options are combined and specified in a shortcut as -qr. The options are followed by the name of the zip archive to create followed by the list of files and directories to backup. We convert the source list into a string using the join method of strings which we have already seen how to use.

Then, we finally run the command using the os.system function which runs the command as if it was run from the system i.e. in the shell - it returns 0 if the command was successfully, else it returns an error number.

Depending on the outcome of the command, we print the appropriate message that the backup has failed or succeeded.

That's it, we have created a script to take a backup of our important files!

Note to Windows Users

Instead of double backslash escape sequences, you can also use raw strings. For example, use 'C:\\Documents' or r'C:\Documents'. However, do not use 'C:\Documents' since you end up using an unknown escape sequence \D.

Now that we have a working backup script, we can use it whenever we want to take a backup of the files. Linux/Unix users are advised to use the executable method as discussed earlier so that they can run the backup script anytime anywhere. This is called the operation phase or the deployment phase of the software.

The above program works properly, but (usually) first programs do not work exactly as you expect. For example, there might be problems if you have not designed the program properly or if you have made a mistake in typing the code, etc. Appropriately, you will have to go back to the design phase or you will have to debug your program.

[edit] Second Version

The first version of our script works. However, we can make some refinements to it so that it can work better on a daily basis. This is called the maintenance phase of the software.

One of the refinements I felt was useful is a better file-naming mechanism - using the time as the name of the file within a directory with the current date as a directory within the main backup directory. First advantage is that your backups are stored in a hierarchical manner and therefore it is much easier to manage. Second advantage is that the length of the filenames are much shorter. Third advantage is that separate directories will help you to easily check if you have taken a backup for each day since the directory would be created only if you have taken a backup for that day.

#!/usr/bin/python
# Filename: backup_ver2.py
 
import os
import time
 
# 1. The files and directories to be backed up are specified in a list.
source = ['"C:\\My Documents"', 'C:\\Code']
# Notice we had to use double quotes inside the string for names with spaces in it.
 
# 2. The backup must be stored in a main backup directory
target_dir = 'E:\\Backup' # Remember to change this to what you will be using
 
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + os.sep + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
 
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
    os.mkdir(today) # make directory
    print('Successfully created directory', today)
 
# The name of the zip file
target = today + os.sep + now + '.zip'
 
# 5. We use the zip command to put the files in a zip archive
zip_command = "zip -qr {0} {1}".format(target, ' '.join(source))
 
# Run the backup
if os.system(zip_command) == 0:
    print('Successful backup to', target)
else:
    print('Backup FAILED')

Output:

   $ python backup_ver2.py
   Successfully created directory E:\Backup\20080702
   Successful backup to E:\Backup\20080702\202311.zip
   
   $ python backup_ver2.py
   Successful backup to E:\Backup\20080702\202325.zip

How It Works:

Most of the program remains the same. The changes is that we check if there is a directory with the current day as name inside the main backup directory using the os.path.exists function. If it doesn't exist, we create it using the os.mkdir function.

[edit] Third Version

The second version works fine when I do many backups, but when there are lots of backups, I am finding it hard to differentiate what the backups were for! For example, I might have made some major changes to a program or presentation, then I want to associate what those changes are with the name of the zip archive. This can be easily achieved by attaching a user-supplied comment to the name of the zip archive.

Note

The following program does not work, so do not be alarmed, please follow along because there's a lesson in here.

#!/usr/bin/python
# Filename: backup_ver3.py
 
import os
import time
 
# 1. The files and directories to be backed up are specified in a list.
source = ['"C:\\My Documents"', 'C:\\Code']
# Notice we had to use double quotes inside the string for names with spaces in it.
 
# 2. The backup must be stored in a main backup directory
target_dir = 'E:\\Backup' # Remember to change this to what you will be using
 
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + os.sep + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
 
# Take a comment from the user to create the name of the zip file
comment = input('Enter a comment --> ')
if len(comment) == 0: # check if a comment was entered
    target = today + os.sep + now + '.zip'
else:
    target = today + os.sep + now + '_' +
        comment.replace(' ', '_') + '.zip'
 
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
    os.mkdir(today) # make directory
    print('Successfully created directory', today)
 
# 5. We use the zip command to put the files in a zip archive
zip_command = "zip -qr {0} {1}".format(target, ' '.join(source))
 
# Run the backup
if os.system(zip_command) == 0:
    print('Successful backup to', target)
else:
    print('Backup FAILED')

Output:

   $ python backup_ver3.py
     File "backup_ver3.py", line 25
       target = today + os.sep + now + '_' +
                                           ^
   SyntaxError: invalid syntax

How This (does not) Work:

This program does not work! Python says there is a syntax error which means that the script does not satisfy the structure that Python expects to see. When we observe the error given by Python, it also tells us the place where it detected the error as well. So we start debugging our program from that line.

On careful observation, we see that the single logical line has been split into two physical lines but we have not specified that these two physical lines belong together. Basically, Python has found the addition operator (+) without any operand in that logical line and hence it doesn't know how to continue. Remember that we can specify that the logical line continues in the next physical line by the use of a backslash at the end of the physical line. So, we make this correction to our program. This correction of the program when we find errors is called bug fixing.

[edit] Fourth Version

#!/usr/bin/python
# Filename: backup_ver4.py
 
import os
import time
 
# 1. The files and directories to be backed up are specified in a list.
source = ['"C:\\My Documents"', 'C:\\Code']
# Notice we had to use double quotes inside the string for names with spaces in it.
 
# 2. The backup must be stored in a main backup directory
target_dir = 'E:\\Backup' # Remember to change this to what you will be using
 
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + os.sep + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
 
# Take a comment from the user to create the name of the zip file
comment = input('Enter a comment --> ')
if len(comment) == 0: # check if a comment was entered
    target = today + os.sep + now + '.zip'
else:
    target = today + os.sep + now + '_' + \
        comment.replace(' ', '_') + '.zip'
 
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
    os.mkdir(today) # make directory
    print('Successfully created directory', today)
 
# 5. We use the zip command to put the files in a zip archive
zip_command = "zip -qr {0} {1}".format(target, ' '.join(source))
 
# Run the backup
if os.system(zip_command) == 0:
    print('Successful backup to', target)
else:
    print('Backup FAILED')

Output:

   $ python backup_ver4.py
   Enter a comment --> added new examples
   Successful backup to E:\Backup\20080702\202836_added_new_examples.zip
   
   $ python backup_ver4.py
   Enter a comment -->
   Successful backup to E:\Backup\20080702\202839.zip

How It Works:

This program now works! Let us go through the actual enhancements that we had made in version 3. We take in the user's comments using the input function and then check if the user actually entered something by finding out the length of the input using the len function. If the user has just pressed enter without entering anything (maybe it was just a routine backup or no special changes were made), then we proceed as we have done before.

However, if a comment was supplied, then this is attached to the name of the zip archive just before the .zip extension. Notice that we are replacing spaces in the comment with underscores - this is because managing filenames without spaces are much easier.

[edit] More Refinements

The fourth version is a satisfactorily working script for most users, but there is always room for improvement. For example, you can include a verbosity level for the program where you can specify a -v option to make your program become more talkative.

Another possible enhancement would be to allow extra files and directories to be passed to the script at the command line. We can get these names from the sys.argv list and we can add them to our source list using the extend method provided by the list class.

The most important refinement would be to not use the os.system way of creating archives and instead using the zipfile or tarfile built-in module to create these archives. They are part of the standard library and available already for you to use without external dependencies on the zip program to be available on your computer.

However, I have been using the os.system way of creating a backup in the above examples purely for pedagogical purposes, so that the example is simple enough to be understood by everybody but real enough to be useful.

Can you try writing the fifth version that uses the zipfile module instead of the os.system call?

[edit] The Software Development Process

We have now gone through the various phases in the process of writing a software. These phases can be summarised as follows:

1. What (Analysis)
2. How (Design)
3. Do It (Implementation)
4. Test (Testing and Debugging)
5. Use (Operation or Deployment)
6. Maintain (Refinement)

A recommended way of writing programs is the procedure we have followed in creating the backup script: Do the analysis and design. Start implementing with a simple version. Test and debug it. Use it to ensure that it works as expected. Now, add any features that you want and continue to repeat the Do It-Test-Use cycle as many times as required. Remember, Software is grown, not built.

[edit] Summary

We have seen how to create our own Python programs/scripts and the various stages involved in writing such programs. You may find it useful to create your own program just like we did in this chapter so that you become comfortable with Python as well as problem-solving.

Next, we will discuss object-oriented programming.