CSSE1001/7030 Notes

What’s all this then, Amen!

– Monty Python

Introduction to Software Engineering and Programming

What is Software Engineering?

Software Engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software (IEEE std 610.12-1990). A software engineer must have a good understanding of tools and techniques for requirements gathering, specification, design, implementation, testing and maintenance.

These days, software systems are often very large and many contain key safety or mission-critical components. The complexity of modern software systems require the application of good software engineering principles. Indeed, many agencies mandate the use of particular tools and techniques to achieve very high quality and reliable software.

This course introduces some of the key principles of software engineering and gives you the chance to put them into practice by using the programming language Python. While you’ll learn lots of Python in this course, it is equally important that you learn some common programming principles and techniques so that learning the next language is not as daunting.

What is Python?

Python is a powerful, yet simple to use high-level programming language. Python was designed and written by Guido van Rossum and named in honour of Monty Python’s Flying Circus. The idea being that programming in Python should be fun! The quotes in these notes all come from Monty Python and it has become somewhat of a tradition to reference Monty Python when writing Python tutorials.

What is Software?

Software (a computer program) is a set of instructions that tell the computer (hardware) what to do. All computer programs are written in some kind of programming language, which allows us to precisely tell the computer what we want to do with it.

Features of a Programming Language

There are many programming languages but they all have some things in common.

Syntax and Semantics

Firstly, each program language has a well defined syntax. The syntax of a language describes the well-formed ‘sentences’ of the language. Syntax tells us how to write the instructions: what is the order of the words, how our program must be structured, etc. It’s a bit like grammar and sentence rules when we write in English.

When we write a sentence in English, we may mess up the grammar and still be understood by the reader (or, the reader may ask us what we mean). On the other hand, if we mess up the syntax when we write a computer program, the computer is not as forgiving - it will usually respond by throwing up an error message and exit. So it is fundamentally important that we understand the syntax of a programming language.

Secondly, each self-contained piece of valid syntax has well defined semantics (meaning). For programming languages, it is vitally important that there are no ambiguities - i.e. a valid piece of syntax has only one meaning.

High- and Low- Level Languages

Programming languages are typically divided into high-level and low-level languages.

Low-level languages (like assembler) are ‘close’ to the hardware in the sense that little translation is required to execute programs on the hardware. Programs written in such languages are very verbose; making it difficult for humans to write and equally importantly, difficult to read and understand!

On the other hand, high-level languages are much more human-friendly. The code is more compact and easier to understand. The downside is that more translation is required in order to execute the program.

Compiled and Interpreted Languages

Before we can run our program, we need to type out the list of instructions (called our source code or just code) that outlines what our program does when we want to run it later. So how does our source code then turn into a running program? This depends on whether the programming language we have written the source code in is either a Compiled or Interpreted language.

Compiled languages come with a special program called a compiler which takes source code written by a user and translates it into object code. Object code is a sequence of very low-level instructions that are executed when the program runs. Once a user’s program has been compiled, it can be run repeatedly, simply by executing the object code. Examples of compiled languages include Java, Visual Basic and C.

Interpreted languages come with a special program called an interpreter which takes source code written by the user, determines the semantics of the source code (interprets it) and executes the semantics. This is typically done in a step-by-step fashion, repeatedly taking the next semantic unit and executing it. Examples of interpreted languages include Python, Ruby and Lisp.

Both compilers and interpreters need to understand the semantics of the language - the compiler uses the semantics for the translation to object code; the interpreter uses the semantics for direct execution. Consequently, a program in a compiled language executes faster than the equivalent program in an interpreted language but has the overhead of the compilation phase (i.e. we need to compile our program first before we can run it, which can take a while if the program is very big). If we make a change in our code with a compiled language, we need to recompile the entire program before we can test out our new changes.

One advantage for an interpreted language is the relatively quick turn-around time for program development. Individual components of a large program can be written, tested and debugged without the overhead of compiling a complete program. Interpreters also encourage experimentation, particularly when starting out - just enter an expression into the interpreter and see what happens.

Python is an interpreted language - it has an interpreter. The python interpreter is a typical read-eval-print loop interpreter. In other words, it repeatedly reads expressions input by the user, evaluates the expressions and prints out the result.

Data Types

Another important issue when considering programming languages is the way the language deals with types. Types are used by programming languages (and users) to distinguish between “apples and oranges”. At the lowest level all data stored in the computer are simply sequences of 1’s and 0’s. What is the meaning of a given sequence of 1’s and 0’s? Does it represent an “apple” or an “orange”? In order to determine the intended meaning of such a sequence, it has a type associated it. The type is used to determine the meaning of the 1’s and 0’s and to determine what operations are valid for that data. Programming languages come with built-in types such as integers (whole numbers) and strings (sequences of characters). They also allow users to define their own types.

Programming languages implement type checking in order to ensure the consistency of types in our code. This stops us from doing silly things like trying to add a number to a word, or trying to store an “apple” in a memory location which should only contain an “orange”.

Programming languages are typically either statically typed or dynamically typed languages. When using a statically typed language, checks for the consistency of types are done ‘up-front’, typically by the compiler and any inconsistencies are reported to the user as type errors at compile time (i.e. while the compiler is compiling the program). When using a dynamically typed language, checks for type errors are carried out at run time (i.e. while the user is running the program).

There is a connection between whether the language is compiled or interpreted and whether the language is statically or dynamically typed; many statically typed languages are compiled, and many dynamically typed languages are interpreted. Statically typed languages are usually preferred for large-scale development because there is better control over one source of ‘bugs’: type errors. (But remember, just because the program has no type errors doesn’t make it correct! This is just one kind of bug that our program must not have to run properly.) On the other hand, dynamically typed languages tend to provide a gentler introduction to types and programs tend to be simpler. Python is dynamically typed.

Notes Formatting

In the remainder of the notes we will use different boxes to indicate different parts of the content. These may appear in the readings or separately on Blackboard. Below are examples.

Our galaxy itself contains a hundred million stars;

It’s a hundred thousand light-years side to side;

It bulges in the middle sixteen thousand light-years thick,

But out by us it’s just three thousand light-years wide.

Arithmetic, Basic Types and Variables

Python Arithmetic and Integers

Python understands arithmetical expressions and so the interpreter can be used as a calculator. For example:

>>> 2 + 3 * 4
14
>>> (2 + 3) * 4
20
>>> 10 - 4 - 3
3
>>> 10 - (4 - 3)
9
>>> -3 * 4
-12
>>> 2 ** 3
8
>>> 2 * 3 ** 2
18
>>> (2 * 3) ** 2
36
>>> 7 // 3
2

(Notice how ** is the Python operator for exponentiation.)

There are a few things worth pointing out about these examples. Firstly, the Python interpreter reads user input as strings – i.e. sequences of characters. So for example, the first user input is read in as the string “2 + 3 * 4”. This is treated by the interpreter as a piece of syntax – in this case valid syntax that represents an arithmetic expression. The interpreter then uses the semantics to convert from a string representing an arithmetic expression into the actual expression. So the substring “2” is converted to its semantics (meaning) – i.e. the number 2. The substring “+” is converted to its meaning – i.e. the addition operator. Once the interpreter converts from the syntax to the semantics, the interpreter then evaluates the expression, displaying the result.

This process of converting from syntax to semantics is called parsing.

Secondly, the relationship between the arithmetical operators is what humans normally expect – for example, multiplication takes precedence over addition. When looking at expressions involving operators, we need to consider both precedence and associativity.

Precedence of operators describes how tightly the operators bind to the arguments. For example, exponentiation binds more tightly than multiplication or division, which in turn bind more tightly than addition or subtraction. Precedence dictates the order of operator application (i.e. exponentiation is done first, etc.)

Associativity describes the order of application when the same operator appears in a sequence – for example 10 - 4 - 3. All the arithmetical operators are left-associative – i.e. the evaluation is from left to right. Just like in mathematics, we can use brackets when the desired expression requires the operators to be evaluated in a different order.

Finally, dividing one integer by another, using the // divide operator (known as the div operator), produces an integer – the fractional part of the answer is discarded.

These examples are about integer arithmetic – integers form a built-in type in Python.

>>> type(3)
<class 'int'>

Note how the abbreviation for integers in Python is int. Also, note that type is a built-in Python function that takes an object (in this case the number 3) and returns the type of the object. We will see plenty of examples of Python functions in following sections.

Floats

There is another number type that most programming languages support – floats. Unlike integers, which can only store whole numbers, floats can store fractional parts as well. Below are some examples involving floats in Python.

>>> 7/4
1.75
>>> 7//4
1
>>> -7//4
-2
>>> 7.9//3
2.0
>>> 2.0**60
1.152921504606847e+18
>>> 0.5**60
8.6736173798840355e-19
>>> 2e3
2000.0
>>> type(2e3)
<class 'float'>
>>> int(2.3)
2
>>> float(3)
3.0
>>> 5/6
0.8333333333333334
>>> -5/6
-0.8333333333333334

The syntax for floats is a sequence of characters representing digits, optionally containing the decimal point character and also optionally a trailing ‘e’ and another sequence of digit characters. The character sequence after the ‘e’ represents the power of 10 by which to multiply the first part of the number. This is called scientific notation.

The first example shows that dividing two integers with the / operator gives a float result of the division. The single / division is known as float division and will always result in a float. If an integer result is desired, the // operator must be used. This is the integer division operator and will divide the two numbers as if they are integers (performs the mathematical operation div). Note that this will always round down towards negative infinity, as shown in the third example. If one of the numbers is a float, it turns both numbers into their integer form and then performs the integer division but returns a float answer still, as seen in the third example.

The last two examples highlights one very important aspect of floats – they are approximations to numbers. Not all decimal numbers can be accurately represented by the computer in its internal representation. This is because floats occupy a fixed chunk of memory. The amount of memory used to store a float constrains the precision of the numbers that it can represent.

Variables and Assignments

Most calculators allow results to be stored away in memory and later retrieved in order to carry out complex calculations. This can be done in Python by using variables. As we will soon see, variables are not just for numerical calculations but can be used to store any information for later use.

The valid syntax for variables is any string starting with an alphabetic or underscore character (‘a’-‘z’ or ‘A’- ‘Z’ or ‘’) followed by a sequence of alphanumeric characters (alphabetic + ‘0’-‘9’) and ‘’. Python uses special keywords itself and these cannot be used for any other purpose (for example, as variable names). The Python keywords are listed in the table below. A way of knowing that a word is a keyword is that it will appear in a different colour in IDLE.

    False      class      finally    is         return
    None       continue   for        lambda     try
    True       def        from       nonlocal   while
    and        del        global     not        with
    as         elif       if         or         yield
    assert     else       import     pass
    break      except     in         raise

In later sections, we will see that it is important to choose a good name for variables that accurately shows what the value is meant to represent. There is a convention as well in Python of using lowercase letters in variable names and underscores to separate words, which is adhered to in these notes. There is not anything stopping us ignoring the points in this paragraph, but it becomes an unnecessary annoyance for other people reading our code. Below are some examples of the use of variables.

>>> num1 = 2
>>> num1
2
>>> 12**num1
144
>>> num2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'num2' is not defined
>>> num3 = num1**2
>>> num3
4

The first example above shows an assignment – the variable num1 is assigned the value 2. The semantics of the assignment statement is: evaluate (i.e. work out the value of) the right hand expression and associate the variable on the left hand side of the statement with that value. The left hand side of an assignment statement must be a variable (at least for now – later we will see other possibilities). If the variable already has a value, that value is overwritten by the value on the right hand side of the statement. Note that the assignment statement (when evaluated) has no value – the interpreter does not display a value. Assignments are used for their side-effect – the association of a variable with a value.

The second example shows what happens when we ask the interpreter to evaluate num1. The result is the value associated with the variable. The third example extends this by putting num1 inside an arithmetical expression. The fourth example shows what happens when we try to get the value of the variable num2, to which we have not yet given a value. This is known as an exception and will appear whenever something is typed that Python cannot understand or evaluate. The final example shows how variables can appear on both sides of an assignment (in this case, a variable called num3 is assigned the value worked out by evaluating num1**2; the value of num1 is 2, so num1**2 is 4 and this is the value given to num3.

Strings

To finish this section we introduce the string type, which is used to represent text, and we give some examples of type conversion and printing.

>>> s = "Spam "
>>> 3 * s
'Spam Spam Spam '
>>> 2 + s
Traceback (most recent call last):
  File "<stdin>", line 1, in 
TypeError: unsupported operand type(s) for +: 'int' and 'str'
>>> type('')
<class 'str'>
>>> int("42")
42
>>> str(42)
'42'
>>> print(s + 42)
Traceback (most recent call last):
  File "<stdin>", line 1, in 
TypeError: cannot concatenate 'str' and 'int' objects
>>> print(s + str(42))
Spam 42

The syntax for strings is a double or single quote, followed by a sequence of characters, followed by a matching double or single quote. Notice how the space character is part of the string assigned in the variable s, as it is contained within the double quotes. The string s, therefore, is five characters in length. Notice carefully how this affects the statements executed after it.

As the second example shows, the multiplication operator is overloaded and generates a string that is the number copies of the original string.

Type conversion is used to change between a string representing a number and the number itself.

The built-in print function takes a string argument and displays the result to the user. Note, that in this example, str converts 42 into a string and then the addition operator concatenates (i.e. joins) the two strings together and the result is displayed to the user – another example of overloading.

Uhh, you do realise, uh, he has to be, uh,… well, dead,… by the terms of the card, uh, before he donates his liver.

— Monty Python’s The Meaning of Life

Python Programming

Writing Software

Software is often written by using a program called an Integrated Development Environment, or IDE. An IDE typically consists of an editor (often with syntax highlighting and layout support), a compiler or interpreter, class or library browsers and other tools to aid in software development and maintenance.

In this course we will be using IDLE, which is an IDE for Python.

When IDLE is started it will look something like this.

The window shown contains the running interpreter. The >>> is the interpreter prompt – the interpreter is waiting for user input. Note: This image is taken from Windows 10 and will appear different depending on the Operating System you are using.

Below is a simple example of an interaction with the interpreter – note the syntax highlighting used by IDLE. The user inputs an expression at the prompt (followed by pressing Enter) and the interpreter responds by printing the result and another prompt.

>>> 2+3
5
>>> 6*7
42
>>> print("I don't like spam!")
I don't like spam!

Using IDLE to write code

Once we get beyond simple examples in the interpreter, we typically want to be able to save our code so we can use it again later. We can do this by writing our code in a file. IDLE has an editor window that enables us to write our code and save it into a file.

In the File menu choose New Window.

This will open a new window similar to the one below in which to enter the code.

In this window we will write our code. Let’s start with the simplest program used in programming, the “Hello World!” program.

Type the following code into the new window that you just opened.

print('Hello World!')

When you have finished choose Save As from the File menu and save the file in a folder in which you want to do Python development. Name the file hello.py. It is important to use the .py extension for Python files. Apart from making it easier for us to spot the Python files, various operating systems and some tools also recognise the extension. You will notice that if the .py extension is missing the colours of the code that you write and have written will no longer be present in the IDLE editor window. If this happens re-save the file with the .py extension. Now under the Run menu choose Run Module (or alternatively press F5) to run the program that we have just written. The output will appear in IDLE similar to the following image.

As can be seen the print function of Python displays the data that is the input to the print function (data inside the brackets) in the interpreter. In this case, the string 'Hello World!' is displayed.

Code Layout – Good Programming Practices

Most editors will automatically lay out the code with the use of whitespace (a combination of spaces and tabs). In most programming languages, however, whitespace is (mostly) unimportant, as it does not affect the code.

So why bother with layout then?

It is for human readability. Software engineers typically work in teams and so they need to share code. Consistency is important as it makes it easier for one human to understand what another human has written (we are not machines). It also helps for maintenance and modification of the code at a later date – if we were looking at code we wrote 6 months ago, it’s unlikely we would remember it, so we would want it to be easy to read.

Python takes layout one step further:

Whitespaces affect the semantics!

When writing a block of code (the body of a function definition for example), some way of determining the start and end of code blocks is required. In many languages, keywords or some form of brackets are used to mark the start and end of a block.

Python uses indentation.

When a new block is started (indicated by a :, the level of indentation is increased (typically one tab stop or 4 spaces). After the last statement in the block the indentation level is decremented.

It is also regarded as good coding practice to keep the width of any code written to within 80 characters wide. There are multiple reasons for this including:

• It is easy to read – 80 characters is an easy to read line width especially for something that we are already straining our brains to read, such as code.
• Screen sizes are different – If we write code on a wide screen and do not care about line width and later read it on a standard width monitor then it may not necessarily fit (even with the window enlarged), producing unexpected line wrapping.
• An A4 page, in portrait layout, with normal margins and font size is approximately 80 characters wide – if we keep within this then our code should print without any unwanted line-wrap
• Windows that are used for writing code have a default width of just over the 80 characters wide. If the 80 characters width is changed then the window needs to be resized to fit.

In this section we will look at some issues relating to the design and implementation of software via a simple example. As we go we will introduce more of Python’s syntax and semantics. Before doing so, we look at the software lifecycle. One description of the software lifecycle can be found in Wikipedia at https://en.wikipedia.org/wiki/Software_development_process.

Basically, the software lifecycle describes all the processes required in order to follow good software engineering practices – the aim being to produce high quality software. The basic components of the software lifecycle are as follows:

• Requirements gathering
  • What does the client want?
  • What operating system will the system run on?
  • What commercial off-the-shelf (COTS) software/hardware will be available?
  • Will a hazard/risk analysis be required? (This is often needed when the software will be part of a safety or security critical system)
  • What other systems will this system interface to?
• Design
  • Top-level specification of the system. This may be informal - e.g. a structured English specification, or formal – using logic and mathematics (e.g. for safety/security critical systems)
  • Problem decomposition
  • Module design
  • Interface design
• Coding
  • Implementation of modules
  • System integration
• Testing
  • Bottom-up (unit) testing
  • Integration testing (gluing modules together)
  • Systems testing (does the overall system perform as required?)
• Maintenance
  • Fix problems
  • Add features
  • Respond to changed client requirements

Earlier phases often need to be revisited as problems are uncovered in later phases. In this course we will concentrate mostly on the design, coding and testing phases, typically by working through examples.

In follow-on courses, we will broaden the scope and move from programming as individuals to software development by teams of software engineers where no individual can reasonably be expected to have a deep understanding of all the software relating to a given project.

Other Man: Well I’m very sorry but you didn’t pay!

Man: Ah hah! Well if I didn’t pay, why are you arguing?

Ah HAAAAAAHHH! Gotcha!

Other Man: No you haven’t!

Man: Yes I have! If you’re arguing, I must have paid.

Other Man: Not necessarily. I could be arguing in my spare time.

Man: I’ve had enough of this!

Other Man: No you haven’t.

Man: Oh shut up!

Introduction to Software Design and Implementation

Asking a Question

There comes a time when input from the user is required to be able to collect data to process. This can be done using the input function. This function takes a string prompt as an argument. When the code is run the user sees the prompt and types in the data. input takes this data in as a string, i.e. a piece of text. An example of using input follows.

name = input("What is your name? ")
print("Hello", name + "! Have a nice day.")

Saving as input.py and running the code, the output is similar to

>>> 
What is your name? Arthur, King of the Britons
Hello Arthur, King of the Britons! Have a nice day.

Notice that the input prompt has a space at the end after the question mark. This space is included to separate the question from the user’s response in the interaction. Without it, the interaction would look less appealing to the user: What is your name?Arthur, King of the Britons

As the example shows, strings can be joined together to form one string for use in a print function. This is very useful in situations such as this where we want to print a combination of messages and values (as in our example). The examples show both methods of joining strings together for printing. The , is the first one used, it joins any items together with spaces automatically placed in between. The second is the + symbol, it joins the items together by adding one string to the next to form a single string.

input Syntax 
variable_name = input(string_prompt)
input Semantics
variable_name is given the string value of what the user types in after being prompted.
print Syntax 
print(item1, item2, ..., itemn)
print Semantics 
Each item is displayed in the interpreter, with a space separating each item. print also takes two optional arguments. The sep argument changes what the separator between the items. If not given, the items are separated with a space otherwise, the items will be separated by the given string. For example:
	print(item1, item2, ..., itemn, sep="separator")
Results in the items being separated by the "separator" string instead of spaces. The end argument changes what is printed after all the items. Multiple print function calls will display output on separate lines, unless the end argument is changed as the default is to end prints with a newline character. For example: 
	print(item1, item2, ..., itemn, end="ending")
Will end the print with the "ending" string after all the items are printed. 

True or False

In programing there is always a time when a test is required. This can be used, for example, to see if a number has a relationship with another or if two objects are the same.

There are several character combinations that allow for testing.

== is equal to
!=  not equal to
<   Less than
>   Greater than
<=  Less than or equal to
>=  Greater than or equal to

>>> 1 == 1
True
>>> 2 != 1
True
>>> 2 < 1
False
>>> "Tim" < "Tom"
True
>>> "Apple" > "Banana"
False
>>> "A" < "a"
True
>>> type(True)
<class 'bool'>

As can be seen these statements result in either True or False. True or False are the two possible values of the type bool (short for boolean). Also note that upper and lower case letters in strings are not equal and that an upper case letter is less than the corresponding lower case letter. The reason for this is that computers can only understand numbers and not any characters. Therefore, there is a convention set up to map every character to a number. This convention has become the ASCII scheme. ASCII makes the upper case letters be the numbers 65 through to 90 and the lower case letters the numbers 97 to 122.

Making Decisions

The ability to do a test has no use if it can not be used in a program. It is possible to test and execute a body of code if the test evaluates to True.

This is done using the if statement.

Let’s start with a simple example. The following code is an example of an if statement that will display a hello message if the name input by the user is "Tim".

name = input("What is your name? ")

if name == "Tim" :
    print("Greetings, Tim the Enchanter")

Saving this code as if.py and running, the output from this code looks like:

>>> 
What is your name? Tim
Greetings, Tim the Enchanter

>>> 
What is your name? Arthur, King of the Britons

As can be seen if name does not equal "Tim" then nothing is output from the code.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">test</span><span class="p">:</span>
<span class="n">body</span> </code></pre></div>    </div>

What if we want to run a different block of code if the test is False? This requires an if, else statement Our example can be modified to print a different message if name is not "Tim".

name = input("What is your name? ")

if name == "Tim" :
    print("Greetings, Tim the Enchanter")
else :
    print("Hello", name)

Saving this code as if_else.py , the output from this code looks like:

>>> 
What is your name? Tim
Greetings, Tim the Enchanter

>>> 
What is your name? Arthur, King of the Britons
Hello Arthur, King of the Britons

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">if</span> <span class="n">test</span><span class="p">:</span>
<span class="n">body1</span>  <span class="k">else</span><span class="p">:</span>
<span class="n">body2</span> </code></pre></div>    </div>

It is also possible to carry out multiple tests within the same if statement and execute different blocks of code depending on which test evaluates to True. We can do this simply by using an if, elif, else statement. Our example can be modified further to look like the following

name = input("What is your name? ")

if name == "Tim" :
    print("Greetings, Tim the Enchanter")
elif name == "Brian" :
    print("Bad luck, Brian")
else :
    print("Hello", name)

Saving this code as if_elif_else.py , our example now has the following output:

>>> 
What is your name? Tim
Greetings, Tim the Enchanter

>>> 
What is your name? Brian
Bad luck, Brian

>>> 
What is your name? Arthur, King of the Britons
Hello Arthur, King of the Britons

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">test1</span><span class="p">:</span>
<span class="n">body1</span> <span class="k">elif</span> <span class="n">test2</span><span class="p">:</span>
<span class="n">body2</span> <span class="o">.</span> <span class="o">.</span> <span class="o">.</span> <span class="k">elif</span> <span class="n">testn</span><span class="p">:</span>
<span class="n">bodyn</span> <span class="k">else</span><span class="p">:</span>
<span class="n">bodyn1</span> </code></pre></div>    </div>

Being Repetitive

We are off to a good start, but the interaction is not very long. We are not doing much before we abruptly end the conversation. For our next addition to the program, we would like to be able to talk to the user for as long as we can. Let’s accomplish this by asking the user for a topic, talking about that topic, then asking for another topic. Here is an example of what we might want:

What is your name? Tim
Greetings, Tim the Enchanter
What do you want to talk about? Python
Do you like Python? yes
Why do you think that? it's easy to use
I also think that it's easy to use
What do you want to talk about? coconuts
Do you like coconuts? no
Why do you think that? they cannot migrate
I also think that they cannot migrate
What do you want to talk about? CSSE1001
Do you like CSSE1001? very much
Why do you think that? the course notes are very useful
I also think that the course notes are very useful
What do you want to talk about? nothing
Okay. Goodbye, Tim!

To do this, we will need to have a way to repeat the discussion until the conversation is over. We can use a construct called a while loop. We need to consider what code should be repeated (in this case, the discussion of a topic) and when it should keep going (in this case, when the topic is not “nothing”). Let’s update our code to include the repetition. This code can be downloaded as interaction_while.py

name = input("What is your name? ")
if name == "Tim" :
    print("Greetings, Tim the Enchanter")
elif name == "Brian" :
    print("Bad luck, Brian")
else :
    print("Hello", name)

topic = input("What do you want to talk about? ")
while topic != "nothing" :
    like = input("Do you like " + topic + "? ")
    response = input("Why do you think that? ")
    print("I also think that", response)
    topic = input("What do you want to talk about? ")

print("Okay. Goodbye, " + name + "!")

The topic = input(...) line above the loop asks for the first topic. The while topic != "nothing" line checks if the given condition is True, and if it is, then the loop body repeatedly performs the actions until the condition is False. Notice that the last line of the body asks for a new topic, which is used as the topic for the next repetition of the loop. If that new topic is nothing, then the loop test topic != "nothing" becomes False, so the while loop will stop running and the code will continue after the loop body (where the indentation stops). Notice that we are asking if the user likes the topic, but we are not using the response that the user gives. Ignoring the input from the user is very unusual behaviour, but we have done it here to simplify the example. Run this code to experiment with it. What happens when the first topic is “nothing”? Why does that happen? One thing that might seem a bit odd about our code is that the topic = input(...) line occurs in two places, once before the while loop, and once at the end of the loop body. This has to happen this way because the topic needs to be entered in before the topic != "nothing" test happens. We can avoid this by exiting the loop from inside the body. This is done using the break keyword. When break is executed inside a loop, the program will immediately exit the loop and continue after the body. We can use this with an if statement to specify how the loop should finish. This code is available at interaction_break.py

name = input("What is your name? ")
if name == "Tim" :
    print("Greetings, Tim the Enchanter")
elif name == "Brian" :
    print("Bad luck, Brian")
else :
    print("Hello", name)

while True :
    topic = input("What do you want to talk about? ")
    if topic == "nothing" :
        break
    like = input("Do you like " + topic + "? ")
    response = input("Why do you think that? ")
    print("I also think that", response)

print("Okay. Goodbye, " + name + "!")

The first change that we have made is the condition test of the while loop. Since the loop keeps going as long as the condition is True, that means that “while True” will keep going until the break is reached (or it will go on forever if there is no break; loops that go forever are called infinite loops). Using break in this way means that the topic = input(...) line only has to appear once in our code. In many situations, it is considered bad programming practice to have logic that exits a loop from within the middle of the loop body. This is because it complicates understanding the loop’s behaviour. The reader needs to understand the loop’s logical condition plus find and understand the break logic in the middle of the loop body. In the example above, the loop body is short and simple enough that finding the break logic is not difficult. However, once the logic becomes more complex and the code longer it would be harder to read. Try to avoid writing code that exits a loop from the middle of the loop’s body. But, be aware that it is possible and you may need to identify this type of logic in someone else’s code.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="n">test</span><span class="p">:</span>
<span class="n">body</span> </code></pre></div>    </div>

Then you must cut down the mightiest tree in the forest… with… a herring!

Loop Invariants

It is not always easy to tell if a loop will create the correct result in every circumstance. Therefore, we use a loop invariant that states if this condition is True then the result should be correct. For this to work completely the invariant needs to be True at the start, at the end of the loop body and if the loop condition is False the invariant must still be True.

Here is an example that finds the prime factors of a number and puts them into a list. This function will use the next_prime function found in prime.py.

from prime import *

def factorise(n) :
    """Returns a list of the prime factors of 'n'.

    Parameters:
        n (int): Number for which factors are to be found.

    Preconditions:
        n >= 2

    Return:
        list<int>: Prime factors of 'n'.
    """
    prime = 2
    factors = []
    while n > 1 :
        # Loop Invariant: product(factors) * n == initial n
        while n % prime == 0 :
            factors.append(prime)
            n /= prime
        prime = next_prime(prime)
    return factors

This function starts with the current prime number set to 2, as 2 is the first prime number, and an empty list to store the primes in. We then start a while loop that keeps iterating as long as n > 1. We do not want to go less than 2 as there are no primes less than 2. It is here, in a comment, that we introduce our loop invariant. In this case it is that the product of all the primes found so far multiplied by the current number that we are operating on equals the initial number. We then start another while loop that keeps appending the current prime number to the factors list while the current number is divisible by that prime, also dividing the current number by that prime to move onto the next number. Back in the first while loop we use the next_prime function to move onto the next prime number. The last thing the function does is return the list of factors.

Saving as factorise.py we can run a few tests to show this function working.

>>> factorise(10)
[2, 5]
>>> factorise(340)
[2, 2, 5, 17]
>>> 2 * 2 * 5 * 17
340

All right … all right … but apart from better sanitation and

medicine and education and irrigation and public health and roads and a

freshwater system and baths and public order … what have the Romans

ever done for us?

— Monty Python’s Life of Brian

Functional Decomposition

Reusing the Code

In the previous readings we used control structures to implement a simple conversation between a user and the computer. It is not very much to ask the user for all the topics to discuss. Let’s add in some code that starts off with a query about university, and make the computer’s reply “That’s very interesting”. In principle, this is easy, because we already have code that does a discussion, so perhaps we can reuse it by just copying it and changing certain parts.

name = input("What is your name? ")
if name == "Tim" :
    print("Greetings, Tim the Enchanter")
elif name == "Brian" :
    print("Bad luck, Brian")
else :
    print("Hello " + name + ".")

like = input("Do you like university? ")
response = input("Why do you think that? ")
print("That's very interesting.")

while True :
    topic = input("What do you want to talk about? ")
    if topic == "nothing" :
        break
    like = input("Do you like " + topic + "? ")
    response = input("Why do you think that? ")
    print("I also think that", response)

print("Okay. Goodbye, " + name + "!")

This works, but there are a couple of things that might go wrong with copying code like this. First, it makes the code look more complicated than it is, and it is harder to read when the code is in two places instead of one. The second issue is to do with updating the program: if we want to modify how a discussion works, then we would need to update the code in two separate places. Worse, if we forget one of those two places, then the two discussions become different.

What we want to be able to do is to pull out, or abstract the idea of a “discussion” into a separate part of the code, and then be able to easily use that code in as many places as possible. Ideally, we want to end up with code that behaves like this (with code instead of the comments):

# have a discussion on "university"
print("That's very interesting.")
while True :
    topic = input("What do you want to talk about? ")
    if topic == "nothing" :
        break
    # have a discussion on "topic"
    print("I also think that")# , user's response

And then somewhere else in the code, we can define how a discussion works. This abstraction is called a function. Here is what the definition of a “discuss” function looks like in Python:

def discuss(topic) :
    like = input("Do you like " + topic + "? ")
    response = input("Why do you think that? ")
    return response

We use the def keyword to say that we are defining the function called discuss. Inside parentheses, we state the formal parameters, also called arguments, that the function takes, which are the bits of information that the function needs to know to complete its task. In this case, we cannot discuss something without knowing what topic to discuss, so we specify that the discuss function takes a single parameter, which we will give the name topic. After the first line is an indented body of code, most of which we are already familiar with. In this case, the discussion needs to supply or ‘give back’ a result at the end, which is the response that the user entered. We do this using the return keyword, which is shown in the last line above. When a return statement is reached, the function will end immediately; if there is a value after the return keyword, it is the result that is ‘given back’ (we say the value is returned). Every time that we use the discuss function, this body of code is what actually gets executed. The indentation behaves in the same way we’ve seen before – when we stop indenting, then we have finished the function definition. Now that we have this abstraction, we can use the discuss function instead of using the actual discussion code directly:

name = input("What is your name? ")
if name == "Tim" :
    print("Greetings, Tim the Enchanter")
elif name == "Brian" :
    print("Bad luck, Brian")
else :
    print("Hello " + name + ".")

discuss("university")
print("That's very interesting.")

while True :
    topic = input("What do you want to talk about? ")
    if topic == "nothing" :
        break
    response = discuss(topic)
    print("I also think that", response)

print("Okay. Goodbye, " + name + "!")

The full program that we have written is in interaction.py. This file also includes comments, which are discussed in the next section.

Notice the use of the return value in this line: response = discuss(topic). This will execute the discuss function above, and the value that is returned is assigned to the variable response. The line discuss("university") shows a situation where our function is used for its side-effect of interacting with the user. The value is still returned by the function, but it is discarded by the main program.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">function_name</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="p">:</span>
<span class="n">body</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">function_name</span><span class="p">(</span><span class="n">value1</span><span class="p">,</span> <span class="n">value2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">m</span><span class="p">)</span> <span class="p">:</span>
<span class="k">return</span> <span class="n">n</span> <span class="o">+</span> <span class="n">m</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="n">add</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="mi">5</span> <span class="o">&gt;&gt;&gt;</span> <span class="mi">3</span> <span class="o">*</span> <span class="n">add</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span> <span class="mi">15</span> </code></pre></div>    </div>

Decomposing Problems

There is more to software engineering than knowing how to write code. Part of the design process discussed above is problem decomposition. If we are given a description of a problem or task, how can we write a program that solves the problem?

The problem we will work on in this section is “write a program to find the nth prime number”. The first prime is 2, the second is 3, the third is 5 and so on.

The first step (after we understand the problem) is to decompose the problem into simpler and simpler subproblems until we reach a point where the subproblems are easy to implement.

Given we need to find the nth prime number, we could start at some prime (like 2 or 3) and find the next prime number. If we repeat this process, we will get to the prime we want. So an interesting subproblem is “given a number n, find the next prime number after n”.

To do this we can repeatedly look at the next number and ask if that number is a prime, if so we have finished, if not we keep looking. So the next interesting subproblem is “given a number n determine if n is a prime”.

Recall that a prime is a number that is divisible by only itself and one. We can therefore test for a number being a prime if we can test for divisibility of every number from 2 up to n. So, the last subproblem is “given numbers n and m, determine if m divides n (exactly)”.

Divisible or not Divisible

There is a useful mathematical operation that can be used for testing for divisibility. The operator % is known as mod or modulo. It returns the remainder of division, making it useful to test for divisibility.

>>> 7 % 2
1
>>> 7 % 4
3
>>> 9 % 3
0
>>> 10 % 5 == 0
True
>>> 7 % 4 == 0
False

The first three results are because 7 // 2 is 3 with remainder 1, 7 // 4 is 1 with remainder 3, and 9 // 3 is 3 with remainder 0. Testing for divisibility is the same as testing if the remainder is equal to 0. So, 10 is divisible by 5, and 7 is not divisible by 4.

Now we have reached a level of detail that we know how to write it all in Python code, so we can now start writing the code, building up to larger subproblems.

Is a Number Prime?

We will write a function called is_prime to test a number and return whether it is a prime number or not, using the idea above of testing numbers from 2 to n. This code can be downloaded as the file is_prime1.py.

def is_prime(num) :
    """Returns True iff 'num' is prime.

    Parameters:
        num (int): Integer value to be tested to see if it is prime.

    Return:
        bool: True if 'num' is prime. False otherwise.

    Preconditions:
        num > 1
    """
    i = 2
    while i < num :
        if num % i == 0 :
            return False
        i = i + 1
    return True

The input to the function is the number num, this is the number that we wish to test if it is prime. Notice in the comment there is a precondition. As we discussed earlier, preconditions form a “contract” with the user; the function will work only if certain conditions are met. So, for example, if someone uses is_prime with a number less than 2, then the result could be anything - in this case, an incorrect value will be returned, in other cases the function could cause an error - but that is the caller’s problem because this falls outside the contract.

The first line of the function sets i to 2, i is the counting variable that is being used to keep track of the current value to test. It is common coding practice to use i (and j and k as counting variables in loops). The next line is a while, this while tests if i < num, this will mean that we can test the divisibility of all the numbers less than num. The first line of the body of code in the while is an if statement. Inside the while loop, all we need to do is check if i divides num using the % operator. If i does divide num then num is not prime, therefore the body of the if statement is to simply return False. Notice that this takes advantage of the way return works: a return statement will end the function immediately; at this stage, we already know that num is not prime, so we can return immediately and ignore the rest of the function. The last line of the while body is to increment i by 1, this moves onto the next number to check. The last line of the function is to return True. This again uses the “stop immediately” property of return statements: if the function has not returned False by now, then the if num % i == 0: test never became True during the while loop, so we know that num must be prime, so we return True.

Here are a couple of examples of is_prime. Try out some more to test your understanding of the function.

>>> is_prime(4)
False
>>> is_prime(101)
True

This code works fine but we can do better! Firstly note that if 2 does not divide num then there is no point testing if other even numbers divide num (as all even numbers are divisible by 2). We only need to consider odd i. Secondly if i does divide num then there is a j such that num == i*j. Therefore, if num has a factor then one will be less than or equal to the square root of num. Summarising, we only need to test if 2 divides num and if any odd i less than or equal to the square root of num divide num.

The function below implements these ideas. Update the definition to match the following code, or download is_prime2.py.

import math

def is_prime(num) :
    """Returns True iff 'num' is prime.

    Parameters:
        num (int): Integer value to be tested to see if it is prime.

    Return:
        bool: True if 'num' is prime. False otherwise.

    Preconditions:
        num > 1
    """
    if num == 2 :
        return True
    elif n % 2 == 0 :
        return False

    sqrt_num = math.sqrt(num)
    i = 3
    while i <= sqrt_num :
        if num % i == 0 :
            return False
        i = i + 2
    return True

The first line introduces an import statement. This is used to load a module of other functions and data that could be useful. In this case the “math” module is used. This module contains many mathematical functions and numbers (such as pi) not present in the default Python libraries, in our case we are using the square root function (sqrt). To see more of what is in the math module, try >>> help(math) after importing the math module.

The first if statement in the code checks if num is 2 and if so, it is obviously prime so True is returned. The elif statement deals with divisibility by 2, if num is divisible by 2 then it is not prime, therefore False is returned. sqrt_num is set to the square toot of num. To do this, we call the square root function (sqrt) of the math library using the syntax math.sqrt(num). i is then started at 3 as we are now checking the odd numbers only. The while loop will keep going while i <= sqrt_num and will terminate when this test becomes false (i.e. when i > sqrt_num). The if statement checks divisibility of num with i as before and returns False if that is the case. i is then incremented by 2, moving to the next odd number. As before, if the function has not returned by the end of the while loop, then the last line will be executed returning True (i.e. num is prime).

Now that we have updated the function, we can test it again.

>>> is_prime(2)
True
>>> is_prime(9)
False
>>> is_prime(19)
True

Thinking more about how to test for primality enabled us to write more efficient code. The second version of the is_prime function is more efficient as there are fewer iterations through the while loop. This means the code will return an answer faster (especially for larger numbers). Try comparing the two functions with really large numbers and see if there is a difference. Although efficiency is very important in software engineering, we leave more detailed and formal discussions of efficiency to later courses.

The Next Prime

The next function to define is next_prime which takes an num and returns the next prime after num. We will use the same idea as in the previous function - i.e. increment by twos. So we will do slightly different things depending on whether num is odd or even.

def next_prime(num) :
    """Returns the next prime number after 'num'.

    Parameters:
        num (int): Starting point for the search for the next prime number.

    Return:
        int: The next prime number that can be found after 'num'.

    Preconditions:
        num > 1
    """
    if num % 2 == 0 :
        next_number = num + 1
    else :
        next_number = num + 2

    # next_number is the next odd number after num
    while not is_prime(next_number) :
        next_number = next_number + 2
    return next_number

Looking at the code in detail – we start with an if statement that tests if num is even. We use the variable next_number to be the next candidate for being a prime and we initialise it to be either one or two more than num, depending on whether num is odd or even. We have added a comment to remind us about this. The while loop simply increments next_number by 2 until it becomes prime. Note that the test in the while loop is the logical negation of the is_prime test. In other words, we continue looping while next_number is not a prime.

Here are the results of testing.

>>> next_prime(3)
5
>>> next_prime(13)
17
>>> next_prime(101)
103
>>> next_prime(2)
3
>>> next_prime(20)
23

The n^th Prime

Now we bring it all together by writing the top-level function nth_prime that returns the n^th prime number.

def nth_prime(n) :
    """Returns the n'th prime number.

    Parameters:
        n (int): The number of prime numbers to find.

    Return:
        int: The n'th prime.

    Preconditions:
        n > 0
    """
    next_prime_number = 2
    i = 1
    while i < n :
        # loop invariant: next_prime_number is the i'th prime
        i += 1
        next_prime_number = next_prime(next_prime_number)
    return next_prime_number

In this example we introduce the idea of a loop invariant. A loop invariant is a property that, if it is true at the beginning of the body, then it is also true at the end of the loop body. So if the code in the body satisfies this property and it is also true initially (i.e. when we enter the while loop) then it must be true when we exit the loop. A loop invariant can be useful in helping decide on how variables should be initialised, how the variable should change in the body, and when to exit the loop. It is also helps document what the loop does. So think of the loop invariant and then write the code!

In our example the loop invariant is true initially (because 2 is the first prime). Assuming it is true at the start of the loop, then the code advances next_prime_number to the next prime and increments i and so the loop invariant is true again at the end of the loop. Therefore, it is also true when we terminate. In which case, not only is the invariant true but also i == n and so next_prime_number is indeed the n^th prime.

By the way, i += 1 is a shorthand for i = i + 1. The file prime.py contains all the code above plus a top-level comment (using the triple quotes style). In fact, what we have just done is write our own module! In the interpreter try import prime and help(prime), and see how all the trouble we went to writing comments pays off!

King Arthur: ‘What does it say?’

Maynard: ‘It reads, ‘Here may be found the last words of Joseph of

Arimathea. He who is valiant and pure of spirit may find the Holy Grail

in the Castle of aaarrrrggh’.’

King Arthur: ‘What?’

Maynard: ‘…The Castle of aaarrrrggh.’

Bedevere: ‘What is that?’

Maynard: ‘He must have died while carving it.’

Programming Style

Code should be written in a style that makes it easier to comprehend. How you structure and layout your code is called programming style and it is a seemingly trivial but important part of ensuring your code is easy to read. (This is called making the code ‘readable’.) In this course we will following the Google Python Style Guide when writing code. Please ensure that you read and follow these rules. In the style guide the first section, Python Language Rules, relates to how to use particular language features. The second section, Python Style Rules, relates to how to structure your code. The style rules are most immediately applicable to the code you write in the early stages of this course. Some of the language rules will become more applicable later in the course.

Commenting

Writing Comments

Documentation is a software engineering concept that provides the “explanation” of the code. Comments should describe what the code is supposed to do, how to use it properly, and anything else that would be useful to know. A comment is a piece of syntax in a programming language which lets us describe the program code. We can add comments without affecting what our code actually does. Programming languages do this to give us a way of easily documenting our code.

Documenting code is very important — it aids communication between developers and is a great help when it comes to maintenance! Just as an agreed upon layout style is critical, the same holds for commenting style. In Python there are two types of comments, each serving a different purpose.

The first style of comment uses the # character followed by the comment until the end of the line. These comments are useful for describing complicated lines of code, or as a reminder for what a particular line of code is used for. These lines are ignored by the Python interpreter and are mainly notes to the writer of the code.

The second, more important style of comments, are called triple-quote comments or docstrings. These are written using triple quotes (three quote marks: """). Docstrings are meant to explain what use a function serves, without going into the details of “how it does what it does”. In Python, docstrings, are composed of two important parts. The first is a brief explanation of what the function does. This is followed by a detailed explanation of how to use the function. This detailed explanation should include a description of the parameters; any preconditions, which are any requirements that need to be met before the function is called in order for the function to perform correctly; a description of what the function returns; and possibly some examples of usage. We can think of these comments as a kind of contract between the writer and user of the function. The writer is promising that, if the user calls the function with arguments of the correct type that satisfy the precondition (if there is one), the function will behave as described in the comment. If the user calls the function with arguments of the wrong type, or that do not satisfy the precondition, the writer is not responsible for the function’s behaviour.

Function Docstrings

Below is our discuss function updated with some helpful comments, which is in the interaction.py file.

def discuss(topic) :
    """Discuss a topic with the user and return their response.

    Ask if the user likes the topic and why.

    Parameters:
        topic (str): The topic under discussion.

    Return:
        str: Response to the question of why they like the topic.
    """
    like = input("Do you like " + topic + "? ")
    response = input("Why do you think that? ")
    return response

In this course, we will use the following triple quote commenting style.

The opening triple quote should be the first line of the body, indented as part of the body. The first line should give a short summary of the function’s behaviour. If necessary, this line is followed by a blank line, and then by a more detailed explanation of the function’s behaviour. This is followed by a blank line followed by a description of each parameter. The parameter description should indicate the type the function expects for that parameter, the type is in parenthesis after the function name, and then should provide a description of what the function expects to be passed to that parameter. If the function does not have any parameters this section is omitted. Following the parameter description is another blank line and then a description of what the function returns. The return description should indicate the type of value being returned and then describe what is being returned. In this case, the discuss function has one parameter topic of type str (short for “string”) that is the topic used for the discussion in the function’s body. The function returns a str that is the response to the question of why the user likes the topic or not. We can also add a precondition to the comments. Preconditions state what must be done (be logically true) before the function is called. Often this is a constraint on the values of some parameters, over and above the type constraint. (i.e. The code calling the function must ensure that the value of the parameters meets the precondition constraints.) The precondition may also be something that needs to be done before the function is called (e.g. some other function must be called before this function is called). The prime numbers example below shows a function with preconditions. Examples of usage demonstrate what the caller can expect when the use the function. These examples serve two purposes. Firstly, they demonstrate what result will be returned with particular parameters. This is useful for functions with complex logic. Secondly, the example of usage provides test cases, as the function can be executed and the results checked against the results indicated in the example of usage. The examples of usage should be formatted to look like an interactive Python session in the interpreter. This allows the Python doctest tool to automatically test the function to ensure it produces the expected results. (Note: Initially do not worry about understanding how to use doctest or getting the examples of usage perfectly formatted. It is more important to get the idea of writing comments that help other programmers, than it is to worry about understanding automatic testing. Automated testing concepts will be explored in detail in later courses.) At the end is a line containing the terminating triple quotes.

def is_prime(num) :
    """Returns True iff 'num' is prime.

    Parameters:
        num (int): Integer value to be tested to see if it is prime.

    Return:
        bool: True if 'num' is prime. False otherwise.

    Preconditions:
        num > 1

    Examples:
        >>> is_prime(2)
        True
        >>> is_prime(3)
        True
        >>> is_prime(4)
        False
        >>> is_prime(5)
        True
        >>> is_prime(9)
        False
    """

The # style comments are completely ignored by the interpreter. On the other hand the triple quotes comments become part of the function. If you load interaction.py into the interpreter and then start typing a function call, you will see that the first line of the comment appears in the little pop-up window, as shown below.

Further, some python tools, like pydoc, extract this documentation to, for example, generate documentation. Also, below is an example using the help function which displays the docstring comments of the function.

>>> help(discuss)
Help on function discuss in module __main__:

discuss(topic)
    Discuss a topic with the user and return their response.
    
    Ask if the user likes the topic and why.
    
    Parameters:
        topic (str): The topic under discussion.

    Return:
        str: Response to the question of why they like the topic.

>>> 

Classes and Methods Docstrings

Commenting classes and their methods is slightly different to commenting functions. The class itself requires a comment describing what the class does. Again not explaining how it does it. The methods are commented similarly to functions but with a slight difference to the type declaration. The other difference is that the __init__ method of the class has a Constructor in place of the type declaration. Below is the Point class from the Class Design notes.

class Point(object) :
    """A 2D point ADT using Cartesian coordinates."""

    def __init__(self, x, y) :
        """Construct a point object based on (x, y) coordinates.
        
        Parameters:
            x (float): x coordinate in a 2D cartesian grid.
            y (float): y coordinate in a 2D cartesian grid.
        """
        self._x = x
        self._y = y

    def x(self) :
        """(float) Return the x coordinate of the point."""
        return self._x

    def y(self) :
        """(float) Return the y coordinate of the point."""
        return self._y

    def move(self, dx, dy) :
        """Move the point by (dx, dy).

        Parameters:
            dx (float): Amount to move in the x direction.
            dy (float): Amount to move in the y direction.
        """
        self._x += dx
        self._y += dy

Note that the comment for the __init__ method does not have a Return: comment. This is due to a class constructor not returning a value, it creates an object of the class type instead of returning a value.

Also notice that the self parameter is never described in the Parameters: comment. Every method that operates on an object must have a self parameter, and it refers to the object on which the method operates. Consequently its type and value are always known. When calling a method on an object you do not pass the object as a parameter, it is implicitly passed as part of the method call. The move method of Point, would be called like: point_object.move(1.0, 2.5). In this case the self parameter refers to the point_object.

Useful stuff, string, no trouble there

Data Structures

Sequences

A sequence can be thought of as an ordered collection of data – i.e. it has a 0^th element, a first element, a second element and so on.

Note that Python, and many other programming languages, start counting from 0 rather than 1.

A typical operation on any kind of a sequence is to be able to access the i^th element. Another operation is to determine how many elements there are in the sequence. This also leads to the ability to walk through the sequence from start to end. This is known as iterating through the sequence. All sequence-based objects that have this capability are known as iterables. A string can be considered as a specialisation of a sequence as it represents a series of ordered characters. We will use strings as our first example of using ADTs.

Strings

As we have seen before a string can be made using the quotation marks. Python interprets anything held within a pair of “” to be a string. This can be thought of as the constructor of a string.

Because strings are specialisations of sequences, then we expect to be able to get the ith character of a string and find out how long a string is. To be able to access the characters in a string it is possible to do what is called indexing. This is done using [ ] notation – this is the accessor. To be able to find the length of a string we have the len function.

Note that there are other operations we want to perform on strings that may not make sense for other specialisations of sequences.

Here are some examples of indexing and finding the of length strings in action:

>>> s = "spam"
>>> len(s)
4
>>> s[0]
's'
>>> s[3]
'm'
>>> s[4]
Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    s[4]
IndexError: string index out of range
>>> s[-1]
'm'
>>> s[-2]
'a'
>>> i = 0
>>> while i < len(s) :
        print(s[i])
        i += 1

s
p
a
m

Note the 'string index out of range' error in the fourth example – there is no fourth element as Python starts counting at 0. Also notice that negative numbers can be used in indexing. The use of negative numbers starts the indexing at the end of the string. So -1 refers to the last element in the string, -2 refers to the second last, and so on. In the last example we use a while loop to iterate over and print every character in the string. Remember i += 1 is the same as i = i + 1. In Python, there is no separate ‘character type’. Characters in Python are represented as strings of length 1. We will use the term ‘character’ to mean a string of length 1. Strings are immutable sequences – i.e. it is not possible to change the data in a string. We will cover mutable and immutable objects and sequences in the following weeks’ notes.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">len</span><span class="p">(</span><span class="n">sequence</span><span class="p">)</span> </code></pre></div>    </div>

For

In the example above, a while loop was used to iterate over the characters of a string. Iterating over the elements of a sequence is such a common thing to do that Python has a special construct for this purpose – the for loop. Instead of using the while loop as we did earlier we could use a for loop instead, as a string is an iterable object.

>>> s = "spam"
>>> for i in s :
        print(i)

s
p
a
m

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="n">var</span> <span class="ow">in</span> <span class="n">sequence</span> <span class="p">:</span>
<span class="n">body</span> </code></pre></div>    </div>

As a simple example of the use of a for loop we now write a function to determine if a given character is in a string. Here is the definition:

def is_in(char, string) :
    """Return True iff 'char' is in 'string'.

    Parameters:
        char (string): The character being searched for.
        string (string): The string being searched.

    Return:
        bool: True if 'char' is in 'string'. False otherwise.
    """
    for c in string :
        if c == char :
            return True
    return False

Here is an example of is_in:

>>> spam = "spam"
>>> is_in("s", spam)
True
>>> is_in("d", spam)
False

The for loop iterates over the characters in the string, comparing each in turn with the supplied character. If a match is found, True is returned. If we iterate through all the elements without finding a match, then the for loop terminates and the following command is executed – returning False. Determining if a particular item is an element of a sequence is a common operation, and so Python has a built-in operator in that does the job for us – so we don’t need the previous function. Actually, for strings, in does more than the previous function as we can see in the following examples.

>>> 's' in 'spam'
True
>>> 'x' in 'spam'
False
>>> 'sp' in 'spam'
True
>>> 'sa' in 'spam'
False
>>> 'pa' in 'spam'
True

Note how the in keyword is used in for loops as well as boolean tests. It’s important to be aware of both uses of in and how to use it in both cases.

Slice It Up

Earlier we saw how we can access the i_th element using square brackets. We can do more! We can also use the square brackets to do slicing – i.e. extracting subsequences. Here are some examples.

>>> s = 'spam'
>>> s[1:2]
'p'
>>> s[1:3]
'pa'
>>> s[:3]
'spa'
>>> s[1:]
'pam'
>>> s[:-2]
'sp'
>>> s[-3:]
'pam'
>>> s[:]
'spam'

The idea is to supply two indices separated by a colon. So when we write s[n:m], we mean the substring from the n_th index up to, but not including, the m_th index. If the first index is missing we get the slice starting at the beginning of the string or sequence. If the second index is missing we get the slice ending at the last element. In the last case we actually generate a copy of the original string. We can also write s[n:m:k]. This means the substring from n to m in steps of k. Here are some examples.

>>> sp = 'Lovely Spam'
>>> sp[1:10:2]
'oeySa'
>>> sp[0:8:3]
'Le '
>>> sp[-10:2:2]
'o'
>>> sp[-10:10:3]
'olS'
>>> sp[:7:4]
'Ll'
>>> sp[2::3]
'vyp'
>>> sp[::3]
'Le a'
>>> sp[10:1:-2]
'mp lv'
>>> sp[-3:2:-3]
'py'
>>> sp[::-1]
'mapS ylevoL'

The first four examples show starting from one index and ending at another using a step size. The next three examples show ways of starting at the beginning or ending at the end or both using a step size. The last three are examples of using negative step sizes.

To finish off this section we look at two programming problems involving lists. The first problem is, given a character and a string, find the index of the first occurrence of the character in the string.

def find(char, string) :
    """Return the first i such that string[i] == 'char'

    Parameters:
        char (string): The character being searched for.
        string (string): The string being searched.

    Return:
        int: Index position of 'char' in 'string', 
             or -1 if 'char' does not occur in 'string'.
    """
    i = 0
    length = len(string)
    while i < length :
        if char == string[i] :
            return i
        i += 1
    return -1

We use a while loop to iterate over the elements of the string. Either we find an occurrence of the character — in which case we immediately return that index, or we get to the end of the string without finding the character and return -1. Here are some tests of find.

>>> s = "spam"
>>> find('m', s)
3
>>> find('s', s)
0
>>> find('x', s)
-1

Tuples

A tuple is a comma separated list of values. Tuples are useful for storing data that is required to not be modified. As tuples are immutable, they cannot be modified, they are useful for this type of operation.

Here are some examples using tuples.

>>> type((2,3,4))
<class 'tuple'>
>>> point = (2,3)
>>> point
(2, 3)
>>> x, y = point
>>> print(x, y)
2 3
>>> x, y = y, x
>>> print(x, y)
3 2

The second example assigns a tuple to the variable point. This tuple represents an x, y coordinate, one of the more common uses of tuples. (Formally, we say that the name point refers to the tuple (2,3) in memory.) The third is an example of tuple unpacking, also called parallel assignment — it assigns x and y respectively the values of the first and second components of the tuple. The last example uses parallel assignment to swap the values of two variables.

Tuples can also be indexed and sliced in the same way as strings. Here is an example. The tuple in this example represents a person’s details. Using a tuple is an easy way of storing multiple values together to represent the information of a person.

>>> john = ("John", "Cleese", "Ministry of Silly Walks", 5555421, "27/10")
>>> john[2]
'Ministry of Silly Walks'
>>> john[4]
'27/10'
>>> john[:2]
('John', 'Cleese')

Multiple Outputs

Tuples are also useful for using multiple values, for example in for loops and return statements. It is possible to rewrite the find function using a for loop and the enumerate function. Calling enumerate on a sequence will create a sequence of tuples containing a counter and the values in the sequence. By default, the counter starts at 0, so it can be used to generate tuples that contain a value of the sequence and the index of that value. The following is a couple of examples using enumerate.

>>> s = "I like Spam"
>>> for i, c in enumerate(s) :
    print(i, c)

0 I
1  
2 l
3 i
4 k
5 e
6  
7 S
8 p
9 a
10 m
>>> 
>>> 
>>> for i, c in enumerate(s, 3) :
    print('number:', i, 'character:', c)

number: 3 character: I
number: 4 character:  
number: 5 character: l
number: 6 character: i
number: 7 character: k
number: 8 character: e
number: 9 character:  
number: 10 character: S
number: 11 character: p
number: 12 character: a
number: 13 character: m

Notice that the for loop uses tuple unpacking, by assigning the items of the tuple to the variables i and c. The first example simply prints out the index and character as a tuple pair. The second gives a second argument to enumerate, which is the starting value for the counter.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">enumerate</span><span class="p">(</span><span class="n">sequence</span><span class="p">)</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">sequence</span><span class="p">,</span> <span class="n">start</span><span class="p">)</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">enumerate</span><span class="p">(</span><span class="n">seq</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">1</span><span class="p">]),</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">2</span><span class="p">]),</span> <span class="o">...</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">seq</span><span class="p">,</span> <span class="n">start</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">(</span><span class="n">start</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="p">(</span><span class="n">start</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">1</span><span class="p">]),</span> 
                     <span class="p">(</span><span class="n">start</span><span class="o">+</span><span class="mi">2</span><span class="p">,</span> <span class="n">seq</span><span class="p">[</span><span class="mi">2</span><span class="p">]),</span> <span class="o">...</span> </code></pre></div>    </div>

Now we can rewrite find as below:

def find(char, string) :
    """Return the first i such that string[i] == char

    Parameters:
        char (string): The character being searched for.
        string (string): The string being searched.

    Return:
        int: Index position of 'char' in 'string', 
             or -1 if 'char' does not occur in 'string'.
    """
    for i, c in enumerate(string) :
        if c == char :
            return i
    return -1

This function now goes through each character in the string using a for loop with enumerate. The body of the for loop is to check if the current character, c, is the same as char. If it is then the current index i is returned. Otherwise the for loop moves onto the next character in the string. If the for loop ends then -1 is returned. Performing test cases on find.py we can see that it has the same functionality as before.

>>> s = "spam"
>>> find('m', s)
3
>>> find('s', s)
0
>>> find('x', s)
-1

Returning a Tuple

Now we are going to write our own function that returns multiple outputs. Our function is going to take a character and a string and split the string in two, at the first occurrence of the character. In this case we want to write a function that will return three strings - the string up to (but not including) the character; the character; and the string after the character.

Our programming problem can be solved by using find in combination with slicing.

def partition(char, string) :
    """Return 'string' split at 'char'.

    The returned result is a tuple consisting of three strings that
    partition 'string' at 'char' - i.e. the substring before the first occurrence
    of 'char', 'char', and the substring after the first occurrence of 'char'.
    If 'char' does not occur in 'string' then the first component returned
    is the entire 'string' and the last two components are empty strings.

    Parameters:
        char (string): The character used to partition string.
        string (string): The string being partitioned.

    Return:
        tuple<tr, str, str>: sub-string before char, char, sub-string after char;
                             or 'string', "", "".
    """
    index = find(char, string)
    if index == -1 :
        return string, '', ''
    else :
        return string[:index], char, string[index+1:]

Here are some tests of partition.

>>> spam = 'spam'
>>> partition('s', spam)
('', 's', 'pam')
>>> partition('p', spam)
('s', 'p', 'am')
>>> partition('m', spam)
('spa', 'm', '')
>>> partition('x', spam)
('spam', '', '')

In next week’s notes we will see that the find and partition functions, like the is_in function are already part of Python as part of the string ADT interface.

I’m afraid I’m not personally qualified to confuse cats,

but I can recommend an extremely good service.

Mutable and Immutable

As we are about to see lists, are our first example of a mutable sequence and object, now is a good time to discuss mutability. Mutability is the description of whether an object’s data can be modified.

A mutable object is an object whose data is modifiable. The data can be modified either through the methods of the object or through direct access to the data itself. An immutable object is an object where the data cannot be modified in anyway. Strings are examples of immutable objects. It is not possible to modify the data of a string. We are about to meet our first mutable objects: lists.

Lists

Lists are mutable sequences. We can access elements at a particular index as we did with strings and use slicing, but we can also modify a list by changing elements or by adding and removing elements.

Here are some examples of lists in action:

>>> a = [1,2,3,4]
>>> type(a)
<class 'list'>
>>> a[0]
1
>>> a[-1]
4
>>> a[1:3]
[2, 3]
>>> a[::2]
[1, 3]
>>> a[0] = 5
>>> a
[5, 2, 3, 4]
>>> a.append(7)
>>> a
[5, 2, 3, 4, 7]
>>> a.pop(3)
4
>>> a
[5, 2, 3, 7]
>>> a.insert(1,9)
>>> a
[5, 9, 2, 3, 7]
>>> a.sort()
>>> a
[2, 3, 5, 7, 9]
>>>

The first example shows the syntax for lists and the type, the next four examples look up information in the list using indexing and slicing. In the next example, we see indexing used on the left hand side of an assignment statement. The semantics is that the value stored at the supplied index is updated with the value on the right hand side of the assignment. The last four examples use methods of the list class to perform more operations on the list.

The first of these is the append method. The append method adds the item that is the argument to the end of the list. This is followed by the pop method. Pop removes the item at the given index from the list and returns that item. Another way to add items to the list is the insert method. Insert takes an index and an item as arguments and inserts the item at the given index, pushing the other items down the list. The last example shows how we can sort the elements — arrange them in order.

Notice how all the methods and operations performed on the list modify the list stored in the variable being used and most do not return anything. Compare this to strings, where all methods and operations return a new item and do not change the original string in the variable.

Python also includes a function for converting any sequence into a list of the objects in that sequence.

>>> list("spam")
['s', 'p', 'a', 'm']
>>> list((1, 7, 8, 42))
[1, 7, 8, 42]

Because lists are sequences we can iterate over a list using a for loop.

>>> for i in [0, 1, 2, 3, 4] :
    print(i)

0
1
2
3
4

A for loop is not much good if we do not do anything with the items. Here is an example of taking a list of numbers and generating a new list with all the numbers from the original list squared.

>>> x = [2, 5, 8, 14, 18]
>>> y = []
>>> for i in x :
	y.append(i ** 2)

>>> y
[4, 25, 64, 196, 324]

Range

Because iterating over sequences of numbers is very common, Python comes with a function for generating a sequence of numbers. This function is called range. range generates a special object that is iterable, meaning we can loop over the elements in the range using the for loop.

>>> for i in range(10) :
    print(i, end=" ")

0 1 2 3 4 5 6 7 8 9 

In short, range creates an object that can be looped through that contains a sequence of numbers from a start number up to but not including a given end number. The end number is required and is excluded from the sequence as usually we use range to generate indices. The default start number is zero; this is also for generating indices as indexing starts at zero. The step size can also be changed, enabling us to skip numbers instead of going one number at a time. This can be seen in the following examples. Note, to simplify these examples the list function has been used.

>>> list(range(5))
[0, 1, 2, 3, 4]
>>> list(range(1, 10))
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(range(1, 10, 3))
[1, 4, 7]
>>> list(range(10, 1 ,-1))
[10, 9, 8, 7, 6, 5, 4, 3, 2]
>>>

Pass by Reference

The following example is of a function that takes a list and a number and adds all numbers from 0 up to but not including that number to the list.

def add_to(existing_list, num) :
    """Adds numbers 0 to 'num' to the end of 'list'
    Parameters:
        existing_list (list): The list to which numbers will be added.
        num (int): The number up to which will be added to 'existing_list'.
    Examples:
        >>> a_list = []
        >>> add_to(a_list, 4)
        >>> a_list
        [0, 1, 2, 3]
    """
    for i in range(num) :
        existing_list.append(i)

After saving the file as add_to.py and running here is a test.

>>> a_list = [3, 5, 6, 87, 1, 5]
>>> add_to(a_list, 5)
>>> a_list
[3, 5, 6, 87, 1, 5, 0, 1, 2, 3, 4]

Notice how even though the function does not return anything the list is modified simply by passing it to the function. We say that objects in Python are passed by reference, which means that objects passed into functions are passed directly. Consequently, a mutable object, which is changed inside a function body, will still be changed after the function call. In contrast, some programming languages use a pass by value strategy, where a copy of the object is passed into functions, so any modifications will not affect the original object. The add_to function is what is often called a procedure, rather than a function, because it does not return a value. In Python, functions or procedures always return a value, even if there is no return statement. Procedures, like this example, return None if there is no return line. None is an instance of a special type in Python called NoneType which has only one value, None.

>>> type(None)
<class 'NoneType'>
>>> a = None
>>> a
>>> 
>>> help(None)
Help on NoneType object:

class NoneType(object)
 |  Methods defined here:
 |  
 |  __bool__(self, /)
 |      self != 0
 |  
 |  __new__(*args, **kwargs) from builtins.type
 |      Create and return a new object.  See help(type) for accurate signature.
 |  
 |  __repr__(self, /)
 |      Return repr(self).

None is treated specially by the interpreter. The second example shows that anything that evaluates to None is not printed to the interpreter.

Word association football

Dictionaries

A dictionary is similar to a sequence, as it can be indexed. The difference is that, instead of being indexed by 0, 1, 2, etc.; it is indexed by keys. (Dictionaries cannot be sliced.) A dictionary is really a mapping from keys to values. Dictionary keys can be any immutable type — for example strings or numbers, while values can be of any type. Dictionaries are used for storing and retrieving information based on a key. For this reason, there can be no duplicates in the keys, but there can be duplicates in the values.

Here are some examples of dictionaries in action using a phone book as an example.

>>> phone = {'Eric' : 7724, 'John' : 9224, 'Graham' : 8462}
>>> phone
{'John': 9224, 'Graham': 8462, 'Eric': 7724}
>>> type(phone)
<class 'dict'>
>>> phone['John']
9224
>>> phone['Terry']

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    phone['Terry']
KeyError: 'Terry'
>>> phone['Terry'] = 6352
>>> phone['Terry']
6352
>>> phone
{'John': 9224, 'Graham': 8462, 'Eric': 7724, 'Terry': 6352}

The first example shows one way of constructing a dictionary — an open brace (curly bracket) followed by a sequence of key : value pairs followed by a closed brace. Notice how when the dictionary is printed, it is not in the same order that it was created in. This is due to the way dictionaries are stored in memory. They are not necessarily stored in any particular order. The third example shows accessing the dictionary using the square bracket notation. The next example shows that, using the square bracket notation, if a key is not in the dictionary it will raise an error. However, if a key is not in the dictionary it can be added (or if the key is in the dictionary then its value is updated) using the square bracket notation as shown in the last example.

The next couple of examples show other methods of creating a dictionary.

>>> phone = dict(Eric=7724, John=9224, Graham=8462)
>>> phone
{'John': 9224, 'Graham': 8462, 'Eric': 7724}
>>> phone = dict([('Eric', 7724), ('John', 9224), ('Graham', 8462)])
>>> phone
{'John': 9224, 'Graham': 8462, 'Eric': 7724}

The first example shows that a dictionary can be created using the dict function with a sequence of key=value pairs as arguments. The second example shows that the dict function can also be used with a single sequence of (key, value) pairs as an argument.

The next few examples show a few methods of dictionaries as well as a few examples of for loops on dictionaries.

>>> for key in phone :
    print(key, phone[key])

John 9224
Graham 8462
Eric 7724
>>> phone.keys()
dict_keys(['Graham', 'John', 'Eric'])
>>> phone.items()
dict_items([('Graham', 8462), ('John', 9224), ('Eric', 7724)])
>>> for item in phone.items() :
    print(item)

('Graham', 8462)
('John', 9224)
('Eric', 7724)

The first example shows using a for loop directly on a dictionary. The loop variable key becomes each key in the dictionary in turn. This prints out all the key, value pairs of the dictionary. This is the most common method of looping through dictionaries. The next two examples show the keys and items dictionary methods. These two methods return a special class (similar to that of range), but as can be seen they contain a sequence of all the keys or all the key, value pairs that are known as items. These methods (and a similar values are included to provide an efficient way of getting the keys, values, or both and looping through them, as shown in the last example. They are rarely used in any other way.

Using Dictionaries

Let’s start with a simple example of a function that takes a filename and creates a dictionary where the keys are the line numbers and the values are the corresponding lines. Below is the function definition.

def get_lines(filename) :
    """Return a dictionary containing each line in the file as values
    and the corresponding line number as keys.

    Parameters:
        filename (str): Name, including path, of the file to be opened.

    Return:
        dict: Dictionary containing the contents of the file.

    Preconditions:
        'filename' is the name of a file that can be opened for reading.
    """
    lines = {}
    f = open(filename, 'r')
    for i, line in enumerate(f) :
        lines[i] = line.strip()
    f.close()
    return lines

The first line creates an empty dictionary, lines, for us to store our lines in. We then open the filename in universal read mode. Using a for loop, along with the enumerate function seen before, the index and line is easily obtained. We then use the square bracket notation to added the stripped line as the value to the dictionary, with the index (being the line number) as the key. The line was stripped using the strip method of strings, as this is more useful if we were to do anything more with this dictionary. The dictionary is then returned.

Having saved this code as get_lines.py, it can be tested. The following is a test using the text.txt file use previously.

>>> lines = get_lines('text.txt')
>>> lines
{0: 'Python is fun,', 1: 'it lets me play with files.',
2: 'I like playing with files,', 3: 'I can do some really fun stuff.',
4: '', 5: 'I like Python!'}
>>> lines[5]
'I like Python!'

Let’s now look at a slightly more complex example. This example will look at determining the frequency count of characters in a file. We will need to open the file for reading, read the contents of the file and count how many times each character appears. Dictionaries are ideal for this — we can use the characters as the keys and the character counts as the associated values. We will need to make use of the dictionary method get.

>>> d = {}
>>> help(d.get)
Help on built-in function get:

get(...)
    D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.

>>> d = {"one" : 1, "three" : 3 ,"many" : 99999999999}
>>> d["one"]
1
>>> d.get("one")
1
>>> d["two"]

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    d["two"]
KeyError: 'two'
>>> print(d.get("two"))
None
>>> d.get("two", "that number does not exist")
'that number does not exist'

The get method is similar to the square bracket notation that we have used to look up values associated with a key. The biggest difference is that get does not raise an error if the key is not in the dictionary as the square bracket notation does.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">dictionary</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="n">dictionary</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">d</span><span class="p">)</span> </code></pre></div>    </div>

Now we can have a look at our example. The following is the required function definition.

def freq_count(filename) :
    """Return the frequency count of characters occuring in a file.

    Parameters:
        filename (str): Name, including path, of the file to be opened.

    Return:
        dict: Frequency of each character occuring in the file.

    Preconditions:
        'filename' is the name of a file that can be opened for reading.
    """
    freq = {}
    file = open(filename, 'r')
    for line in file :
        for char in line :
            freq[char] = freq.get(char, 0) + 1
    file.close()
    return freq

The function starts with creating an empty dictionary, freq. Then the function opens the file and iterates through the lines of the file with a for loop. Inside the for loop is another (nested) for loop to iterate through the line to get each character. Inside that for loop we start with the square bracket notation for adding a key, value pair to freq using the char as the key. We then use the get method to look up the same key but if char is not in freq then 0 is returned. The value returned by get is incremented by 1, thus incrementing the number of occurrences of char. get is used as it provides the perfect base for this problem. If the char is in freq then the value (the count) is returned and then has 1 added to it and then that new value is assigned to the key. If char is not in freq then get returns 0 which allows us to add 1 to it as this will be the first occurrence of char in the file.

After saving freq.py, it can be applied to a few files that we have used in previous sections.

>>> freq_count("sgame.txt")
{' ': 125, '\n': 9, '1': 5, '3': 3, '2': 3, '5': 3, '4': 3, '7': 2, '6': 3, 
'9': 4, '8': 2}
>>> freq_count("text.txt")
{'\n': 6, '!': 1, ' ': 19, ',': 2, '.': 2, 'I': 3, 'P': 2, 'a': 4, 'c': 1,
'e': 8, 'd': 1, 'g': 1, 'f': 6, 'i': 9, 'h': 4, 'k': 2, 'm': 2, 'l': 9, 'o': 4, 
'n': 6, 'p': 2, 's': 6, 'r': 1, 'u': 3, 't': 7, 'w': 2, 'y': 5}
>>> freq_count("words.txt")
{'\n': 252, '!': 12, ' ': 943, "'": 40, ')': 16, '(': 16, '-': 7, ',': 48, 
'.': 211, '2': 1, '4': 1, '?': 10, 'A': 6, 'C': 7, 'B': 6, 'E': 3, 'D': 12, 
'G': 7, 'F': 30, 'I': 15, 'H': 2, 'K': 1, 'J': 1, 'M': 31, 'L': 4, 'O': 16, 
'N': 10, 'P': 13, 'S': 14, 'R': 5, 'U': 3, 'T': 11, 'W': 8, 'V': 4, 'Y': 10, 
'a': 203, '`': 10, 'c': 51, 'b': 40, 'e': 324, 'd': 114, 'g': 78, 'f': 48, 
'i': 134, 'h': 203, 'k': 21, 'j': 3, 'm': 38, 'l': 99, 'o': 298, 'n': 179, 'q': 5, 'p': 48, 's': 155, 'r': 250, 'u': 98, 't': 294, 'w': 50, 'v': 15, 'y': 72, 'x': 1, 'z': 3}

Note that, instead of reading the file line-by-line, we could have read the entire file into a single string, using read and processed that character-by-character. However, for a very large file this approach would generate a very large (especially in memory) string.

Formatting Strings

The dictionaries, especially when large, do not print out very nicely. Let’s write a function that takes a dictionary and displays it in an easy to read format (also called ‘pretty printing’). To be able to ‘pretty print’ we need to be able to print in a formatted way. We have seen simple examples of this already by simply printing one value after another separated by commas. Python has another approach using the format method of strings. format operates on a format string which has segments where substitutions are made. The items substituted into this format string are the arguments of the format method. Following are a few examples using the format method.

>>> help(str.format)
Help on method_descriptor:

format(...)
    S.format(*args, **kwargs) -> string

    Return a formatted version of S, using substitutions from args and kwargs.
    The substitutions are identified by braces ('{' and '}').

>>> a = 10
>>> b = 22
>>> c = 42
>>> 'a = {0}, b = {1}, c = {2}'.format(a, b, c)
'a = 10, b = 22, c = 42'
>>> 'a = {0}, b = {1}, c = {2}'.format(b, c, a)
'a = 22, b = 42, c = 10'
>>> 'a = {0}, b = {1}, c = b + 2 * a = {1} + 2 * {0} = {2}'.format(a,b,c)
'a = 10, b = 22, c = b + 2 * a = 22 + 2 * 10 = 42'
>>> s = 'hello'
>>> '{0} world, {1}'.format(s, c)
'hello world, 42'
>>> 'a = {0}, b = {1}, c = {2}'.format(b, c)

Traceback (most recent call last):
  File "<pyshell#8>", line 1, in <module>
    'a = {0}, b = {1}, c = {2}'.format(b, c)
IndexError: tuple index out of range
>>> 'a = {0}, b = {1}, c = {2}'.format(a, b, c, s)
'a = 10, b = 22, c = 42'
>>> our_format = 'print {0} everywhere, {1}'
>>> our_format.format(a, s)
'print 10 everywhere, hello'

The first example is of the help description of the format method. The next two examples use the format method to print three numbers out with descriptors as to what they are. Notice how the order of the arguments of format is associated with the numbers in the substitutions. The next example shows that substitutions can be made more than once into the format string. The next example shows that multiple types can be printed. The next example shows that if there are more substitutions to be made than there are arguments in format then an error is raised. However, the next example shows that if there are more arguments in format than substitution areas then there is no error, the extra values are ignored as the example shows that the format string can be assigned to a variable and used later to format print.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="mf">60.0</span><span class="o">/</span><span class="mi">22</span> <span class="mf">2.727272727272727</span> <span class="o">&gt;&gt;&gt;</span> <span class="s">"2 decimal places - {0:.2f}"</span><span class="o">.</span><span class="nb">format</span><span class="p">(</span><span class="mf">60.0</span><span class="o">/</span><span class="mi">22</span><span class="p">)</span> <span class="s">'2 decimal places - 2.73'</span> <span class="o">&gt;&gt;&gt;</span> <span class="s">"5 decimal places - {0:.5f}"</span><span class="o">.</span><span class="nb">format</span><span class="p">(</span><span class="mf">60.0</span><span class="o">/</span><span class="mi">22</span><span class="p">)</span> <span class="s">'5 decimal places - 2.72727'</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="s">"use indexing - {0[3]}, {0[1]}"</span><span class="o">.</span><span class="nb">format</span><span class="p">(</span><span class="s">"hello"</span><span class="p">)</span> <span class="s">'use indexing - l, e'</span> </code></pre></div>    </div>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="s">"give spacing - {0:10}, {1:7}"</span><span class="o">.</span><span class="nb">format</span><span class="p">(</span><span class="s">"hi"</span><span class="p">,</span> <span class="s">"bye"</span><span class="p">)</span> <span class="s">'give spacing - hi        , bye    '</span> <span class="o">&gt;&gt;&gt;</span> <span class="s">"give spacing - {0:10}, {1:7}"</span><span class="o">.</span><span class="nb">format</span><span class="p">(</span><span class="s">"longer"</span><span class="p">,</span> <span class="s">"bye"</span><span class="p">)</span> <span class="s">'give spacing - longer    , bye    '</span> </code></pre></div>    </div>

Let’s return to our example. Below is the function definition for ‘pretty printing our dictionary.

def display_dictionary(dictionary) :
    """Pretty print 'dictionary' in key sorted order.

    Parameters:
        dictionary dict: Dictionary to be pretty printed.
    """
    keys = dictionary.keys()
    keys = sorted(keys)
    for k in keys :
        print('{0}  :  {1}'.format(repr(k), dictionary[k]))

The function first gets the keys of the dictionary and sorts them. The sorted function returns a sorted list of the given sequence. Now, when the dictionary is printed, there is a nice order to the characters. Then it iterates through the keys list and prints them using format. The keys are printed using the repr function. This makes the strings print with the ‘ ‘ around them, if we had printed the strings directly we would have lost the quotes. repr returns the representation of the argument. This can be seen when the disp_dict.py file is saved and tested as below.

>>> freq = freq_count('sgame.txt')
>>> display_dictionary(freq)
'\n'  :  9
' '  :  125
'1'  :  5
'2'  :  3
'3'  :  3
'4'  :  3
'5'  :  3
'6'  :  3
'7'  :  2
'8'  :  2
'9'  :  4

Now go away or I will taunt you a second time

Histogram Example

Let’s look at an example that combines the use of dictionaries and exception handling. This example is about statistical analysis of data. Specifically, we want to read data from a file (one floating point number per line) and produce a histogram of the data. To do this we want to subdivide numbers into ‘buckets’ and count how many times data values fall in each bucket. For the program, we will ask the user for the name of the file containing the data to be processed, and the width of each bucket. For this example we will use the file data1.txt.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;&gt;&gt;</span> <span class="kn">import</span> <span class="nn">random</span> <span class="o">&gt;&gt;&gt;</span> <span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'data1.txt'</span><span class="p">,</span> <span class="s">'w'</span><span class="p">)</span> <span class="o">&gt;&gt;&gt;</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span> <span class="p">:</span>
    <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">random</span><span class="o">.</span><span class="n">normalvariate</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">10</span><span class="p">))</span> <span class="o">+</span> <span class="s">'</span><span class="se">\n</span><span class="s">'</span><span class="p">)</span>

For this problem, we will produce a stand-alone program. The program consists of three parts: getting input from the user, computing the histogram, and writing out the results. Later we will write a GUI version of the same program.

We start with the main part — a function that takes a file name and a bucket size and returns a dictionary of bucket counts. The key for the dictionary will be the ‘bucket position’ — i.e. how many buckets away from 0 this bucket is. So, for example, if bucket size = 10, a value between 0 and 10 will be in bucket 0 and a value between -10 and 0 will be in bucket -1.

Here is the function definition.

def make_histogram(filename, bucketsize) :
    """Compute the histogram of the data in 'filename' with given bucket size.

    An error message will be produced if either the file cannot be opened
    for reading or an invalid data value is found.

    Parameters:
        filename (str): Name of the file from which data is read.
        bucketsize (int): Size of the data bucket.
        
    Return:
        dict<int, float>: Histogram of occurrences of data in file.

    Preconditions:
        bucketsize > 0
    """
    try :
        file = open(filename, 'r')
    except IOError :
        print('Error: cannot open ' + filename + ' for reading')
        return {}

    hist = {}
    for data_element in file :
        try :
            val = float(data_element)
        except ValueError :
            print('Error: cannot convert "' + data_element + '" to a float')
            return {}
        if val < 0 :
            bucket = int((val-bucketsize) / bucketsize)
        else :
            bucket = int(val / bucketsize)
        hist[bucket] = hist.get(bucket, 0) + 1

    return hist

The first thing this function does is attempt to open the file. Unlike previous examples, we have surrounded this with a try statement. This allows us to catch the error raised if the file cannot be opened for reading. If an IOError is raised then we print a simple error message and return an empty dictionary. We then create an empty dictionary, hist to store our data in. We then go through every line in the file directly with a for loop. The first part of the body of the for loop is to attempt to convert the line to a float. This is also surrounded in a try statement to test if the line can be converted. If it cannot then the ValueError is caught and again a simple error message is printed and an empty dictionary is returned. We then check the sign of the number val. If it is negative, we use one formula for which bucket it would belong in; otherwise, we use a different formula. This is because we use rounding when we convert from a float to an integer, as Python always rounds down we need to treat the different signed numbers differently. The last part of the for loop is to increase the count of how many items are in that particular bucket. This is the same as used in the character frequency example above. Finally, we return hist.

Note the power of dictionaries here — we could have used lists to store the bucket counts BUT we need to know the range of data values first. This would have meant reading the file twice or loading all the data into another list. The processing would also have been more complicated as well. Now we can save our program so far, histogram1.py, and test the function. We also create a file data2.txt and test the function using this file (it is a good idea to test small first).

>>> make_histogram('data2.txt', 1.0)
{0: 2, 1: 1, -1: 1}
>>> make_histogram('data1.txt', 1.0)
{0: 41, 1: 29, 2: 35, 3: 33, 4: 45, 5: 35, 6: 43, 7: 27, 8: 33, 9: 23,
10: 17, 11: 15, 12: 10, 13: 12, 14: 23, 15: 10, 16: 6, 17: 6, 18: 7, 19: 4, 20: 8, 21: 4, 22: 4, 23: 1, 24: 5, 25: 2, 26: 2, 27: 1, 28: 1, 29: 2, 30: 1, 34: 1, 
35: 1, -1: 36, -32: 1, -30: 1, -28: 2, -26: 1, -25: 2, -24: 6, -23: 5, -22: 4, 
-21: 6, -20: 2, -19: 6, -18: 6, -17: 10, -16: 18, -15: 11, -14: 17, -13: 20, 
-12: 24, -11: 30, -10: 23, -9: 25, -8: 39, -7: 40, -6: 28, -5: 33, -4: 50, -3: 35, -2: 32}

The next step is to get user input. Given we want a positive number for the bucket size we write the following function that checks user input.

def get_bucketsize() :
    """Return the bucket size asked from the user.

    Ensures that a valid bucket size is entered.

    Return:
        float: Number entered for the bucket size.
    """
    while True :
        user_input = input("Bucket size: ")
        try :
            size = float(user_input)
        except ValueError :
            print('Not a number')
            continue
        if size > 0 :
            break
        print('Number must be positive')
    return size

This function starts with an infinite while True loop; this enables us to keep asking the user until correct input is given. Then the user is prompted for the bucket size. The function then attempts to convert the bucket size given by the user into a float. This is surrounded by a try statement to catch a ValueError in case the user does not input a number. If the user input is not a number then a message is printed and then on the next line is a continue statement. continue starts at the begining of the loop body. This effectively skips all the code after the continue, then since we are at the top of the loop again the function asks the user again for a bucket size. If the input is a number, then it is tested to see if it is positive. If it is positive we break out of the while loop and return the bucket size. If it is not then we print out a message and the loop starts again. Saving this code into histogram2.py we can do a few tests.

>>> get_bucketsize()
Bucket size: a
Not a number
Bucket size: -10
Number must be positive
Bucket size: 3
3.0

The last part is to pretty print the resulting histogram. The next function (similar to the frequency count one) does the job.

def pp_histogram(histogram, bucketsize) :
    """ Pretty prints the histogram using the size of the buckets."""
    keys = histogram.keys()
    keys = sorted(keys)
    for key in keys :
        print('({0:7.2f}, {1:7.2f}) : {2:3}'.format(
            key*bucketsize, (key+1)*bucketsize, histogram[key]))

This function first gets the list of the keys from the dictionary, histogram, using the keys method of dictionaries. This enables us to perform the next line, which is to sort this keys dictionary so that it is in order from smallest to largest. We then iterate over this list and print out the histogram information. The first two substitutions of the format string use two methods of the extra format string options discussed in the notes about the dictionary data structure. These segments of the format string look like :7.2f. The 7 means to have spacing of 7 characters. The .2f means to print as floats to 2 decimal places. The last substitution area contains the index number with a spacing of 3 characters. As the item to be printed here is just an integer it needs no float formatting.

Here is the result of saving our histogram3.py code and applying this function to the histogram for data1.txt.

>>> pp_histogram(make_histogram('data1.txt', 5.0), 5.0)
( -35.00,  -30.00) :   1
( -30.00,  -25.00) :   4
( -25.00,  -20.00) :  23
( -20.00,  -15.00) :  42
( -15.00,  -10.00) : 102
( -10.00,   -5.00) : 155
(  -5.00,    0.00) : 186
(   0.00,    5.00) : 183
(   5.00,   10.00) : 161
(  10.00,   15.00) :  77
(  15.00,   20.00) :  33
(  20.00,   25.00) :  22
(  25.00,   30.00) :   8
(  30.00,   35.00) :   2
(  35.00,   40.00) :   1

To complete the stand-alone program we just need to add the following code to the end of the code that we have written so far.

print('Print a histogram\n')
filename = input('File name: ')
bucketsize = get_bucketsize()
print('\n\n--------------------------------------------\n\n')
pp_histogram(make_histogram(filename, bucketsize), bucketsize)

The complete code for this example is in histogram.py. If we run the module from IDLE we get the following output in the interpreter window.

Print a histogram

File name: data1.txt
Bucket size: 5.0


--------------------------------------------


( -35.00,  -30.00) :   1
( -30.00,  -25.00) :   4
( -25.00,  -20.00) :  23
( -20.00,  -15.00) :  42
( -15.00,  -10.00) : 102
( -10.00,   -5.00) : 155
(  -5.00,    0.00) : 186
(   0.00,    5.00) : 183
(   5.00,   10.00) : 161
(  10.00,   15.00) :  77
(  15.00,   20.00) :  33
(  20.00,   25.00) :  22
(  25.00,   30.00) :   8
(  30.00,   35.00) :   2
(  35.00,   40.00) :   1

What is going on? Well, all the expressions in the file are evaluated in the interpreter. The definitions are evaluated and as a consequence are added to the interpreter. The other expressions are then evaluated — this is really the program being executed.

Brave Sir Robin ran away, bravely ran away away.

When danger reared his ugly head, he bravely turned his tail and fled.

Yes, brave Sir Robin turned about, he turned his tail, he chickened out.

Bravely taking to his feet, he beat a very brave retreat.

A brave retreat by Sir Robin.

I/O

Files

It is very common in programming that input is received from a file instead of, or as well as, user input. It is therefore necessary to be able to open, read and save to files. Python has a file object that enables us to perform these tasks.

Reading a File

To show how to use the file object in Python here are a few examples using the text.txt file.

>>> f = open('text.txt', 'r')
>>> type(f)
<class '_io.TextIOWrapper'>
>>> f.read()
'Python is fun,\nit lets me play with files.\nI like playing with files,\nI can 
do some really fun stuff.\n\nI like Python!\n'
>>> f.close()
>>> f.read()
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    f.read()
ValueError: I/O operation on closed file
>>>

The first line shows opening a file using the open function and assigns it to the variable f. The open statement takes two string parameters; one is the name of the file to be opened. (The file name may include the path to the file if it is in a different directory to the Python program.) The other is the open mode, we will discuss this soon. The second example shows that f is a file object. The third example shows one of the multiple methods for getting the data from the file. The fourth example shows closing the file, it is important to close a file after the program is finished with it. The last example shows that it is not possible to perform operations on a closed file.

More Reading

As before, dir(f), will list all the methods available for use with files. The more useful ones for us are the ones that allow us to read from and write to the file. Below are some examples of the read methods we have not seen yet.

>>> f = open('text.txt', 'r')
>>> dir(f)
['_CHUNK_SIZE', '__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_checkClosed', '_checkReadable', '_checkSeekable', '_checkWritable', '_finalizing', 'buffer', 'close', 'closed', 'detach', 'encoding', 'errors', 'fileno', 'flush', 'isatty', 'line_buffering', 'mode', 'name', 'newlines', 'read', 'readable', 'readline', 'readlines', 'seek', 'seekable', 'tell', 'truncate', 'writable', 'write', 'writelines']
>>> f.readline()
'Python is fun,\n'
>>> f.readline()
'it lets me play with files.\n'
>>> f.readline()
'I like playing with files,\n'
>>> f.readline()
'I can do some really fun stuff.\n'
>>> f.readline()
'\n'
>>> f.readline()
'I like Python!\n'
>>> f.readline()
''
>>> f.close()
>>> f = open('text.txt', 'r')
>>> f.readlines()
['Python is fun,\n', 'it lets me play with files.\n', 'I like playing with files,\n', 'I can do some really fun stuff.\n', '\n', 'I like Python!\n']
>>> f.close()
>>> f = open('text.txt', 'r')
>>> for line in f :
	print(line)

Python is fun,

it lets me play with files.

I like playing with files,

I can do some really fun stuff.



I like Python!

>>> f.close()
>>>

The first few examples are the use of readline, it returns one line of the file each time it is called. When there are no lines left readline returns an empty string. The next example is of readlines (notice the extra s). readlines returns a list with each element in the list being a string of each line in the file. Notice how each line of the file ends in a \n. This is the new line character; it is the character that is, invisibly, inserted when we hit the Enter (return) key. The last example is the use of a for loop to directly iterate over the file one line at a time. Notice how there is an extra line in between each line that we print. This is caused by the new line character, \n. print interprets the new line and inserts it, but print also inserts its own new line, therefore we end up with two new lines printed.

Writing

Now let’s have a look at a method for writing to a file:

>>> f = open("lets_write.txt", "w")
>>> f.write("I'm writing in the file\n")
25
>>> text = ['look', 'more', 'words']
>>> for word in text :
	f.write(word)

>>> f.close()
>>>

Now if we look at the file it should look like lets_write1.txt. Notice how we need to put in newlines manually when required, they are not automatically inserted. The write method returns the number of bytes or characters that it wrote to the file, in our example the string "I'm writing in the file\n" is 25 characters long. Also, notice how we did not have to first create the destination file. Opening the file for writing creates the file if it does not already exist.

Let’s look at another method of writing to a file:

>>> f = open("lets_write.txt", "w")
>>> text = ['many, many \n', 'lines\n', 'are\n', 'easily\n', 'inserted\n',
        'this way!']
>>> f.writelines(text)
>>> f.close()

Now if we look at the file again it should look like lets_write2.txt. First, notice that the data we wrote before to the file is no longer there, this is because ‘w’ writes from the start of the file and writes over anything that is already there. Notice again that the newlines had to be inserted manually.

Read and Write

Let’s look at an example that uses both reading and writing, with some functionality to process the data. We are going to write a function that will take two filenames as arguments and turn all the characters in one file into uppercase and write them into the other file. Here is the code:

def make_all_caps(in_filename, out_filename) :
    """Convert all characters in 'in_filename' to caps and save to 'out_filename'.

    Parameters:
        in_filename (string): Name of the file from which to read the data.
        out_filename (string): Name of the file to which the data is to be saved.

    Preconditions:
        The files can be opened for reading and writing.
    """
    fin = open(in_filename, 'r')
    fout = open(out_filename, 'w')
    for line in fin :
        fout.write(line.upper())
    fin.close()
    fout.close()

The first thing we do is open the in_filename in Universal read mode for portability. We then open the out_filename in write mode. Next, we use a for loop to iterate over the lines in fin. The body of the for loop writes the uppercase version (using strings upper method) of each line of the input file (fin) to the output file (fout). Both files are then closed so that the file pointers are not left open. Saving as make_all_caps.py we can run an example using our text.txt file from earlier.

>>> make_all_caps('text.txt', 'text_caps.txt')

If we now have a look at text_caps.txt. it should have the contents of text.txt all in upper-case.

Files often have blank lines in them in one form or another. When dealing with files it is generally easier to ignore blank lines than to attempt to process them. However, blank lines are not really blank, they contain the new line character ‘\n’.

Let’s modify our code above to remove blank lines from the file as well as make it all upper case.

def all_caps_no_blanks(in_filename, out_filename) :
    """Changes every character in 'in_filename' to all caps, 
       removes blank lines and saves to 'out_filename'.

    Parameters:
        in_filename (string): Name of the file from which to read the data.
        out_filename (string): Name of the file to which the data is to be saved.

    Preconditions:
        The files can be opened for reading and writing.
    """
    fin = open(in_filename, 'r')
    fout = open(out_filename, 'w')
    for line in fin :
        if line != '\n' :
            fout.write(line.upper())
    fin.close()
    fout.close()

Notice the change to the function is the addition of the if statement to check if the current line is equal to a new line character. If it is not then we write the upper case version of the line to the fout file. Now if we run all_caps_no_blanks1.py on our text.txt file like below:

>>> all_caps_no_blanks('text.txt', 'text_caps_nb.txt')

We should now have a file that looks like text_caps_nb.txt.

Blank lines might not just include new line characters, they may also include other forms of whitespace, such as spaces and tabs. If we run the previous function on words.txt we will notice that there are still blank lines as some of them have spaces.

We can modify the example again to work in these cases using the string method strip. strip returns a copy of the string with all whitespace removed from the beginning and end of the string. If the string incorporates only whitespace then an empty string is returned.

def all_caps_no_blanks(in_filename, out_filename) :
    """Changes every character in 'in_filename' to all caps, 
       removes blank lines and saves to 'out_filename'.

    Parameters:
        in_filename (string): Name of the file from which to read the data.
        out_filename (string): Name of the file to which the data is to be saved.

    Preconditions:
        The files can be opened for reading and writing.
    """
    fin = open(in_filename, 'r')
    fout = open(out_filename, 'w')
    for line in fin :
        if line.strip() :
            fout.write(line.upper())
    fin.close()
    fout.close()

Now if we run all_caps_no_blanks2.py on our words.txt file and words_allcaps_nb.txt then it should be capitalised with no blank lines.

Notice how the if statement is simply if line.strip(). This is because the boolean representation of an empty string is False while a string with anything in it is True. The same can be said for objects such as lists, tuples and more.

>>> bool('')
False
>>> bool(' ')
True
>>> bool('sfg')
True
>>> bool([])
False
>>> bool([0])
True
>>> bool([2,5,6,3])
True
>>> bool(())
False
>>> bool((2,3,4))
True

We interrupt this film to apologise for this unwarranted attack by the supporting feature.

Luckily, we have been prepared for this eventuality, and are now taking steps to remedy it.

Sudoku Example

We are now going to have a look at an example that will incorporate list and file processing. This example will be of a Sudoku game stored in the file sgame.txt. The pretty-printed form of this game is sgame.pdf.

The first step is to write a function that takes the name of a file that contains a Sudoku game and returns a representation of the game. The data structure we use to store the game is a list of lists — the entries are rows and each entry in a row represents a column entry of the row. To do this we also write a function to turn a string representing a row into a row.

def row2list(row_string) :
    """Convert a string representing a row in a Sudoku game into a list.

    Parameters:
        row_string (string): Represents a single row in a Sudoku game.

    Return:
        list<string>: Containing the numbers in a row of a Sudoku game.

    Preconditions:
        Numbers in 'row_string' are single digits
        and there is a single space separating the numbers.
    """
    row = []
    for i in range(0, 18, 2) :
        row.append(row_string[i])
    return row

This function initialises an empty list for collecting the row entries. Then uses a for loop with range that goes through every second number, up to but not including 18. As each entry is a space apart in the string we only want every second character in that string. The character at each entry position is then appended into the row list.

def read_game(filename) :
    """Read the data for a Sudoku game from 'filename'.

    Parameters:
        filename (string): Name of the file from which to read the game data.

    Return:
        list<list<string>>: Representation of a Sudoku game as a matrix
                            (list of lists) of strings.

    Preconditions:
        The files can be opened for reading and writing.
        File contains 9 lines and each line represents one row of the game.
    """
    file = open(filename, 'r')
    game = []
    for line in file :
        game.append(row2list(line))
    file.close()
    return game

This function opens the file in universal read mode and then initialises an empty list representing the game data structure. Then for every line in the file the row2list function is called on the line to get the row list. That row list is then appended to the game list to create the full game.

Our code currently looks like sudoku1.py. A couple of tests of the functions are below.

>>> row2list('1 2 3 4 5 6 7 8 9')
['1', '2', '3', '4', '5', '6', '7', '8', '9']
>>> row2list('1 2   4 5 6 7 8  ')
['1', '2', ' ', '4', '5', '6', '7', '8', ' ']
>>> for i in read_game('sgame.txt'):
        print(i)
		
[' ', ' ', '1', ' ', ' ', ' ', ' ', '9', ' ']
['7', ' ', ' ', '1', ' ', '3', ' ', '5', ' ']
[' ', ' ', ' ', ' ', '9', '8', '1', ' ', '6']
[' ', '9', ' ', ' ', '4', ' ', ' ', ' ', ' ']
[' ', '8', '4', ' ', ' ', ' ', '6', '1', ' ']
[' ', ' ', ' ', ' ', '5', ' ', ' ', '2', ' ']
['6', ' ', '5', '9', '3', ' ', ' ', ' ', ' ']
[' ', '7', ' ', '4', ' ', '2', ' ', ' ', '1']
[' ', '2', ' ', ' ', ' ', ' ', '3', ' ', ' ']
>>>

Now we need functions to extract a given row, column or 3x3 block. Adding the following functions to our code will give us this functionality. Keep in mind that the game data structure can be thought of as a 2-dimensional matrix.

def get_row(row, game) :
    """Return the indicated 'row' from 'game'.

    Parameters:
        row (int): Index of the row to extract from 'game'.
        game (list<list<string>>): Matrix representation of the Sudoku game grid.
		
    Return:
        list<string>: The numbers in a row of a Sudoku game.

    Preconditions:
        0 <= 'row' <= 8
        'game' is a list representation of a Sudoku game.
    """
    return game[row]

This function simply needs to return the row^th index of game as game is a list of rows.

def get_column(col, game) :
    """Return the column indicated by 'col' from 'game'.

    Parameters:
        col (int): Index of the column to extract from 'game'.
        game (list<list<string>>): Matrix representation of the Sudoku game grid.
		
    Return:
        list<string>: The numbers in a column of a Sudoku game.

    Preconditions:
        0 <= 'col' <= 8
        'game' is a list representation of a Sudoku game.
   """
    column = []
    for row in game :
        column.append(row[col])
    return column

This is a bit more tricky than get_row as we do not have a list of columns. Therefore, the function needs to go through every row and collect the values at the column index of the row.

def get_block(row, col, game) :
    """Return the 3x3 block starting at index [row, col] from 'game'.

    Parameters:
        row (int): Row index of the start of the block to extract from 'game'.
        col (int): Column index of the start of the block to extract from 'game'.
        game (list<list<string>>): Matrix representation of the Sudoku game grid.
		
    Return:
        list<string>: The numbers in a 3x3 block of a Sudoku game.

    Preconditions:
        0 <= r < 3 and 0 <= c < 3
        'game' is a list representation of a Sudoku game.
    """
    block = []
    for block_row in range(3*row, 3*row+3) :
        block.extend(game[block_row][3*col:3*col+3])
    return block

This function is even more complicated. We need to be able to get a 3x3 block, therefore we need to get three adjacent rows and the three adjacent entries from each of those rows. The for loop gets the rows using range to generate indices that correspond to the block required. The 3*row is used as the block rows are numbered 0, 1 or 2 down the game data structure. This moves the start index to the required starting row of the board that corresponds to the start of that block. The end row is simply 3 more rows down. As game is a list of lists representing rows it can be directly indexed to get the row of interest. Then slicing can be used on that row to get the 3 adjacent columns. This is done using the same system as obtaining the rows. The extend method of lists modifies the list it is called on by joining the list in its argument to the first list to form one list.

Our Sudoku code now looks like sudoku2.py. Here are some tests of the functions that we have just written.

>>> game = read_game('sgame.txt')
>>> get_row(1, game)
['7', ' ', ' ', '1', ' ', '3', ' ', '5', ' ']
>>> get_column(0, game)
[' ', '7', ' ', ' ', ' ', ' ', '6', ' ', ' ']
>>> get_block(1, 2, game)
[' ', ' ', ' ', '6', '1', ' ', ' ', '2', ' ']
>>>

To finish off this problem we look at the problem of determining what possible values can be put in an empty square. To do this, we need to determine what are the possibilities based on entries already in a given row, column or block. This can be done by finding the difference between the given entries and the valid entries. Below are the functions that will allow us to do this.

def list_diff(list1, list2) :
    """Return the list of entries in list1 that are not in list2 (list1 - list2).

    A general-purpose list function that works for lists of any type of elements.
    """
    diff = []
    for element in list1 :
        if element not in list2 :
            diff.append(element)
    return diff

This function creates an empty result list, diff. The function then goes through every element in list1 and checks if it is in list2. If element is not in list2 then element is appended to our result list diff.

# all the valid choices
ALL_CHOICES = ['1', '2', '3', '4', '5', '6', '7', '8', '9']

def choices(row, column, game) :
    """Return choices that are possible at position indicated by [row, column].

    Identify and return all the choices that are possible at position 
    [row, column] in 'game' - i.e. each choice should not occur in the 
    indicated 'row', 'column' or in the block containing this position.

    Parameters:
        row (int): Row index of the position to check.
        column (int): Column index of the position to check.
        game (list<list<string>>): Matrix representation of the Sudoku game grid.
		
    Return:
        list<string>: The valid choices at this position.

    Preconditions:
        0 <= row <= 8 and 0 <= column <= 8  game[row][column] == ' '
    """
    block_row = row / 3
    block_col = column / 3
    choices = list_diff(ALL_CHOICES, get_row(row, game))
    choices = list_diff(choices, get_column(column, game))
    choices = list_diff(choices, get_block(block_row, block_col, game))
    return choices

This function starts by finding the block that the coordinate [row, column] is in and setting this to block_row and block_col. Then it finds all the choices for the row by calling list_diff with the list ALL_CHOICES and the list obtained from get_row as arguments. Then choices is updated by passing it into list_diff along with the result from get_column and assigning the result back into choices, this removes all the non-possible choices from the column from the choices list. This is again done with the block. This then leaves all the choices available for that square so choices is returned.

Finally, our sudoku code looks like sudoku.py. Here results of some tests of the last few function we wrote.

>>> list_diff(ALL_CHOICES, get_row(0, game))
['2', '3', '4', '5', '6', '7', '8']
>>> list_diff(ALL_CHOICES, get_column(0, game))
['1', '2', '3', '4', '5', '8', '9']
>>> list_diff(ALL_CHOICES, get_block(0, 0, game))
['2', '3', '4', '5', '6', '8', '9']
>>> list_intersection(['2', '3', '4', '5', '6', '7', '8'],
                      ['1', '2', '3', '4', '5', '8', '9'])
['2', '3', '4', '5', '8']
>>> choices(0, 0, game)
['2', '3', '4', '5', '8']
>>>

My hovercraft is full of eels

Error Handling

We have seen many errors that the interpreter has produced when we have done something wrong. In this case, we say that the interpreter has raised an error. A typical example is when we wish to use user input. We cannot guarantee that the user will input values correctly. If we are not careful, user input might cause an error. When this happens, we want to be able to catch the error (Exception) and deal with it ourselves (users do not like to see red).

To be able to do this we surround the code that is likely to raise an error in a try, except statement. The following gives a couple of examples of error handling on the int function that attempts to convert user input into an integer. To do this we shall write a small function definition as follows.

def int_exception() :
    """Asks for user input and attempts to convert it into an integer.

    Returns:
        int: Integer value entered by user;
             or -1 if they did not enter an integer.
    """
    num = input("Enter a number: ")
    try :
        return int(num)
    except Exception :
        print("{0} is not a number".format(num))
        return -1

This function simply asks the user for a number. It then attempts to convert the input into an int and return it. See how the return int(num) is inside a try statement and followed by an except Exception statement. This is how the exception is caught. If the block of code between the try and except lines raises an exception then the rest of the block is not executed and the code after the except line is executed instead. In this case, we print a short message and return -1. Saving the file as int_exception.py we can perform a couple of test as below.

>>> int('10')
10
>>> int_exception()
Enter a number: 10
10
>>> int('ten')

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    int('ten')
ValueError: invalid literal for int() with base 10: 'ten'
>>> int_exception()
Enter a number: ten
'ten is not a number'
-1

The first example simply shows that the string '10' can be turned into an integer. The next example shows our function and we give 10 as user input and see that it returns 10. The next example shows then the string 'ten' raises an error as it cannot be converted into a string. We then run our function again and see that we do not get the error but instead get the message 'ten is not a number' and -1 is returned. Note how we used except Exception, this catches any error that happens. It is OK to do this in this case as there are not many other ways this code can raise an error. It is not always good practice to do this, especially in complex code. Sometimes we end up catching errors raised by mistakes in our code, not the user input. It would therefore, be better to state the exception we want to catch rather than the “catch all” Exception.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">try</span> <span class="p">:</span>
<span class="n">body1</span> <span class="k">except</span> <span class="n">ExceptionName</span> <span class="p">:</span>
<span class="n">body2</span> </code></pre></div>    </div>

Using the Exception

It is possible to store the exception caught so that information is able to be extracted for use. This is done using as. To demonstrate this we are going to modify our code from above to look like the function definition below.

def int_exception(in_num) :
    """Asks user for a number and divides 'in_num' by the input.

    Parameters:
        in_num (int): Number to be divided by user input.

    Returns:
        float: 'in_num' divided by the number entered by the user;
               or -1 if input is not an integer or if input is 0.
    """
    num = input("Enter a number: ")
    try :
        num = int(num)
        return in_num / num
    except Exception as e :
        print("Error: {0}".format(str(e)))
        return -1

The function now takes a number argument. It still asks the user for a number. The number from the argument is then divided by the number given by the user and returned. If an error is raised then it is caught and assigned to e. An error message is then returned including the string representation of the error. Saving the file as int_exception2.py we can perform a couple of test as below.

>>> int_exception(15)
Enter a number: 10
1.5
>>> int_exception(15)
Enter a number: ten
Error: Invalid literal for int() with base 10: 'ten'
-1
>>> int_exception(15)
Enter a number: 0
Error: division by zero
-1

The first example works as expected, the function returns 1.5. The next two examples show the function outputting an error message; the first being 'Error: could not convert string to float: ten' from a ValueError, which is from attempting to float a non numerical string from the input and the other is 'Error: float division by zero' from a ZeroDivisionError, which happens if 0 is input (we can not divide by zero).

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">try</span> <span class="p">:</span>
<span class="n">body1</span> <span class="k">except</span> <span class="n">ExceptionName</span> <span class="k">as</span> <span class="n">var</span> <span class="p">:</span>
<span class="n">body2</span> </code></pre></div>    </div>

Catching Different Exceptions

We have seen that it is possible to catch every error raised and that we can get information about the error. So how do we catch a specific error that we know is possible from incorrect input? It is possible to specify what errors we want to catch and deal with. This way we only catch the errors that are caused by incorrect input, rather than everything and miss the errors raised by any incorrect code.

To demonstrate this let’s look at the function definition below:

def get_numbers(dividend) :
    """Asks the user repeatedly for numbers and calculates 'dividend' 
    divided by each number.

    Results of division are stored in a list.
    The list is returned if nothing is entered.

    Parameters:
        dividend (int): Number to be divided by the numbers entered by user.

    Return:
        list<float>: List of dividend divided by each input number.
    """
    results = []
    while True :
        num = input("Enter Number: ")
        if not num :
            break
        try :
            num = float(num)
            results.append(dividend / num)
        except ValueError :
            print("That is not a number")
        except ZeroDivisionError :
            print("Can't divide by zero")
    return results

This function takes a number as an argument. It then repeatedly asks for user input with input. If there is nothing typed then the while loop breaks. If something is typed then it attempts to convert the input to a float and then attempts to divide the argument number by the input number, appending it to the list. It checks for two different errors, ValueError and ZeroDivisionError In both cases a simple message is printed to tell the user that they have given wrong input.

After saving the file as get_numbers.py we can run a few tests.

>>> get_numbers(10)
Enter Number: 2
Enter Number: a
That is not a number
Enter Number: 4
Enter Number: 0
Can't divide by zero
Enter Number: 3.5
Enter Number: 
[5.0, 2.5, 2.857142857142857]

It can be seen in this example that if non numerical inputs are given then we get That is not a number. If 0 is input then the message is Can’t divide by zero. Any numerical based inputs in this test work.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">try</span> <span class="p">:</span>
<span class="n">body1</span> <span class="k">except</span> <span class="n">ExceptionName1</span> <span class="p">:</span>
<span class="n">body2</span> <span class="o">.</span> <span class="o">.</span> <span class="o">.</span> <span class="k">except</span> <span class="n">ExceptionNameN</span> <span class="p">:</span>
<span class="n">bodyn</span> </code></pre></div>    </div>

Dealing with Unknown Exceptions

Sometimes we need to be prepared for any exception that may happen. We have seen a way of dealing with this in the first couple of exception handling examples with except Exception. However, sometimes there are errors that we wish to deal with specially and just have a single case for any other exception that may occur. We can modify the function definition above to be able to demonstrate this.

def get_numbers(dividend) :
    ""Asks the user repeatedly for numbers and calculates 'dividend' 
    divided by each number.

    Results of division are stored in a list.
    The list is returned if nothing is entered.

    Parameters:
        dividend (int): Number to be divided by the numbers entered by user.

    Return:
        list<float>: List of dividend divided by each input number.
    """
    results = []
    while True :
        num = input("Enter Number: ")
        if not num :
            break
        try :
            num = float(num)
            results.append(dividend / num)
        except ValueError :
            print("That is not a number")
        except ZeroDivisionError :
            print("Can't divide by zero")
        except Exception as e :
            print("Unknown Error {0}".format(str(e)))
            return []
    return results

This function definition is the same except that we have added an extra except statement. This except statement is another except Exception like we have seen before. The way this function now works is, if a ValueError or ZeroDivisionError error occurs then it will behave the same as the previous example. If any other exception occurs then the last except statement will catch it and print out an error message and return an empty list.

Saving now as get_numbers2.py we can test the function.

>>> get_numbers(10)
Enter Number: 2
Enter Number: f
That is not a number
Enter Number: 0
Can't divide by zero
Enter Number: 
[5.0]
>>> get_numbers("g")
Enter Number: 2
Unknown Error: unsupported operand type(s) for /: 'str' and 'float'
[]

The first few examples are as before. The last example is an example of the new functionality. The function itself was given a string argument when it was meant to be given numbers only. This created a different exception when we attempted to divide a string by a number. Therefore, our function prints an error message and returns an empty list.

Raising Exceptions

There are many situations where we might want an error to occur. If a function receives incorrect input, or some other invalid action occurs, it is better to let the function raise an exception, which forces another part of the code deal with the problem (by using try-except statements). As an example, we will revisit the prime numbers example from earlier. To test if a number is prime, we wrote this function:

def is_prime(num) :
    """Returns True iff 'num' is prime.

    Parameters:
        num (int): Integer value to be tested to see if it is prime.

    Return:
        bool: True if 'num' is prime. False otherwise.

    Preconditions:
        num > 1
    """
    i = 2
    while i < num :
        if num % i == 0 :
            return False
        i = i + 1
    return True

We discussed that the precondition n > 1 means that inputs which don’t fit this criteria could have unknown consequences. This function would return True if the input was invalid, and it would be the user’s responsibility to check that the input was valid. If the user doesn’t do this, there could be more disastrous consequences later in the program. We will modify the function to raise an error when the input is invalid; this will make sure any mistakes don’t quietly pass by.

Different types of exception in Python serve different purposes. For example, we use ValueError to represent an inappropriate value (for example, if n <= 1), and TypeError to represent an input of the wrong type (for example, a float).

def is_prime(num) :
    """Returns True iff 'num' is prime.

    Parameters:
        num (int): Integer value to be tested to see if it is prime.

    Return:
        bool: True if 'num' is prime. False otherwise.

    Preconditions:
        num > 1
    """
    if num <= 1 :
        raise ValueError("Input must be > 1")
    if num != int(num) :
        raise TypeError("Input must be an integer")

    i = 2
    while i < num :
        if num % i == 0 :
            return False
        i = i + 1
    return True

When we test the function on incorrect inputs, we see the function raises the appropriate errors:

>>> is_prime(-2)
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    is_prime(-2)
  File "is_prime.py", line 9, in is_prime
    raise ValueError("Input must be > 1")
ValueError: Input must be > 1
>>> is_prime(3.14)
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    is_prime(3.14)
  File "is_prime.py", line 11, in is_prime
    raise TypeError("Input must be an integer")
TypeError: Input must be an integer

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">raise</span> <span class="n">ExceptionName</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> </code></pre></div>    </div>

Listen. Strange women lying in ponds distributing swords is no basis for a system of government.

Supreme executive power derives from a mandate from the masses,

not from some farcical aquatic ceremony.

Variable Scope

Now that we have written some programs we should describe how Python keeps track of all the variables in our programs — what values they have, and which variable are being talked about at a specific point in our programs. This is an issue in every programming language and is typically managed by using environments (a data structure) to maintain information about the variables and values that can be accessed in a particular part of a program (program scope) such as a function body. The scope of a variable describes where in a program a particular variable can be used.

Python uses dictionaries for its environments — each dictionary maintains the mapping from variable names to values. Python uses the term namespace to refer to these dictionaries. There is one global namespace that keeps information about everything at the ‘global’ level. When each function is called a new local namespace is constructed to keep track of variables inside the function. Python comes with two functions that extract the global and (current) local namespaces: globals() and locals(). Below is some code and some results of running the code that gives an insight into how Python keeps track of variables.

>>> a = 10
>>> b = 11
>>> def foo(b) :
    print('Global namespace = ', globals())
    print('Local namespace = ', locals())
    return a + b

>>> foo(3)
Global namespace =  {'__loader__': <class '_frozen_importlib.BuiltinImporter'>, 'foo': <function foo at 0x7f2cb35f58c8>, '__builtins__': <module 'builtins' (built-in)>, 
'__spec__': None, '__package__': None, '__doc__': None,
'__name__': '__main__', 'b': 11, 'a': 10}
Local namespace =  {'b': 3}
13
>>> 

We can see that variables a and b are defined in the global namespace and b is also defined in the local namespace of foo. So, why is the value of foo(3) equal to 13? Python first looks in the local namespace to see if a given variable is defined there. If so, it uses the corresponding value. If not it looks in the global namespace for a value. If the variable is not in the global namespace we get a familiar error message. In the case above, b is defined in the local namespace and its value (3) is used. The variable a is not defined in the local namespace so Python looks in the global namespace, getting the value 10.

Compare the above example with the following.

>>> a = 10
>>> b = 11
>>> def foo(b) :
        a = a + 3
        return a + b

>>> foo(3)

Traceback (most recent call last):
  File "<pyshell#58>", line 1, in <module>
    foo(3)
  File "<pyshell#57>", line 2, in foo
    a = a + 3
UnboundLocalError: local variable 'a' referenced before assignment
>>> 

In this case we are trying to modify a global variable (i.e. one not in the local scope). Python sees this assignment as follows — on the left hand side is a and so Python treats this as a local variable. On the right hand side is an occurrence of a, but it has not been given a value in the local scope and so our favorite error is produced.

It is possible to modify global variables as the following example shows.

>>> a = 10
>>> b = 11
>>> def foo(b):
    global a
    a = a + 3
    return a + b

>>> foo(3)
16
>>> foo(3)
19
>>> 

The architects themselves came in to explain the advantages of both designs.

Class Design

In the previous sections, we have used existing classes (e.g. str and list) that enable us to be able to write functions and programs that do what we wish. In this section, we will begin to design our own classes. Being able to write our own classes is a useful tool as not everything is in a format that we like or will find easy to use for a particular program that we may wish to write. We will start with the design of a simple ADT for a 2D point (an x and y coordinate).

2D Point Class

For our first example will we write the class definition for a 2D point. This class will require a constructor (to be able to create instances of the class), a couple of accessors that get the x, y coordinate of the point and a mutator to move the point by a certain distance. The class definition is as follows. Below, we will discuss this definition in detail.

import math

class Point(object) :
    """A 2D point ADT using Cartesian coordinates."""

    def __init__(self, x, y) :
        """Construct a point object based on (x, y) coordinates.
        
        Parameters:
            x (float): x coordinate in a 2D cartesian grid.
            y (float): y coordinate in a 2D cartesian grid.
        """
        self._x = x
        self._y = y

    def x(self) :
        """(float) Return the x coordinate of the point."""
        return self._x

    def y(self) :
        """(float) Return the y coordinate of the point."""
        return self._y

    def move(self, dx, dy) :
        """Move the point by (dx, dy).

        Parameters:
            dx (float): Amount to move in the x direction.
            dy (float): Amount to move in the y direction.
        """
        self._x += dx
        self._y += dy

Class definitions start with the keyword class, followed by the class name, and (object):. Following this, and indented, is what looks like function definitions. These are the method definitions for the class. Notice the first argument of each method is self: self is a reference to the object itself. This argument is needed so that the method can access and modify components of the object. Class names, like function names, follow a naming convention. This convention is that the first letter of each word in the class name is a capital letter. Methods follow the same naming convention as functions.

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">ClassName</span><span class="p">(</span><span class="nb">object</span><span class="p">)</span> <span class="p">:</span>
<span class="s">"""Comment"""</span>

<span class="k">def</span> <span class="nf">method_1</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="p">[</span><span class="n">args</span><span class="p">])</span> <span class="p">:</span>
    <span class="n">method_1_body</span>

<span class="k">def</span> <span class="nf">method_2</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="p">[</span><span class="n">args</span><span class="p">])</span> <span class="p">:</span>
    <span class="n">method_2_body</span>

<span class="o">...</span> </code></pre></div>    </div>

Setting up a Point

Earlier in the course, we have used the dir function to list the methods of an object, and saw that many of the methods had double underscores at the start and end of their names. These methods each have a special meaning in Python, which allow the object to work with existing Python syntax, such as arithmetic, slicing, and built-in functions.

The first method that the Point class has is the __init__ method. This is the constructor method of the class, which is executed when the object is created. __init__ takes as arguments any data that is to be required to make the instance of the class and creates the instance variables and any data structures required for the instance to function. In this case __init__ takes in the x, y coordinate of the point and creates two instance variables, self._x and self._y. Note the self. at the start of the variable names. self is required to access any variables and methods of the instance, as it is the reference to the object instance. self._x and self._y are given the values of x and y respectively from the inputs into the constructor (__init__ method) of the Point class.

The underscores on the instance variables also have an informal meaning. In object-oriented programming it is often useful to have private variables and methods. This is data and methods that are not meant to be accessed from outside the class except perhaps via a non-private method. Python does not have a way of setting variables and methods private. Python instead uses underscores to ‘hide’ variables and methods. Note: This is simply a naming convention used by programmers to signify to readers of the code that this variable or method is meant to be private. It is possible to access these variables and methods directly if the number of underscores is know, though it is not recommended in case the class definition changes.

The next two methods (the x and y methods) are our accessors — they provide us with an interface to access the class instance variables from outside the class definition. These methods simply return the corresponding coordinate (either x or y). The method move is a mutator — it modifies the data stored in the object. Other than self, move also has the arguments dx and dy. These are added to the current self._x and self._y, respectively, to ‘move’ our point along a certain vector to a new location.

Now we can save our code as point1.py and run a few tests to see it in action.

>>> p = Point(2, -5)
>>> p
<__main__.Point object at 0x011D4710>
>>> str(p)
'<__main__.Point object at 0x011D4710>'
>>> type(p)
<class '__main__.Point'>
>>> p.x()
2
>>> p.y()
-5
>>> p.move(-3, 9)
>>> p.x()
-1
>>> p.y()
4

The first line creates an instance, p, of our new Point class. When we create a Point instance, we are instantiating the class. The example shows a print out of how Python represents our Point class, while the one after it shows Pythons string version of our Point class. The following example shows the type of our Point class. We then go and call the x and y methods to view the current state of the class instance. We then move our Point, p, and then have a look at the new state of the class instance.

String Representations

The second and third examples above showed how Python represents our Point class. The Python interpreter is using the default methods for str and repr for the string representations of our class. These are not particularly nice or useful representations of this class as it does not tell us much about the instance state. We can make our class have a better representation by writing our own __str__ and __repr__ methods into our class definition. These two method names are used to define the behaviour of the built-in str and repr functions respectively. The following method definitions will provide good __str__ and __repr__ representations of our Point.

    def __str__(self) :
        """The 'informal' string representation of the point."""
        return '({0}, {1})'.format(self._x, self._y)

    def __repr__(self) :
        """The 'official' string representation of the point."""
        return 'Point({0}, {1})'.format(self._x, self._y)

The __str__ and __repr__ methods both use a similar formatted string to produce a string that is a nice representation of our Point class. These methods are used when the functions str and repr, respectively, are used on our Point class. __repr__, ideally, should represent all the data important to the object’s state and to be able to recreate the object with that same state. If possible we should also make it so that if the interpreter read the repr string back in and evaluated it, it would construct a copy of the object.

After adding the above code to the class definition our class we can now save our point2.py code and test our two new methods.

>>> p = Point(-1, 4)
>>> p
Point(-1, 4)
>>> str(p)
'(-1, 4)'
>>> repr(p)
'Point(-1, 4)'

The first example shows what Python now returns if we simply ask for p. When something is evaluated the interpreter uses the __repr__ method of the class as is shown in the third example. The second example shows the string representation of our Point class. Notice that the repr string looks just like the line created to make the Point instance originally.

Arithmetic on Points

>>> Point(1, 3) == Point(1, 3)
False
>>> p1 = Point(2, 3)
>>> p2 = p1
>>> p1 == p2
True
>>> p3 = Point(2, 3)
>>> p1 == p3
False