Security and Privacy in a Networked World/Programming in Python

NOTE: This topic makes use of Wikipedia articles to provide the basic understanding of the matters. These articles deal with technology and are not much disputed, having reached the common knowledge stage. They also provide some good links for further study under the reference sections.

Python?

Python is a programming language developed by Guido van Rossum in 1989. It is a full-fledged, generic programming language with many different uses, in networking security it is often used as a tool to write short scripts for managing (and sometimes also, attacking) computer networks. Other reasons for choosing Python as the example language for this course include:

Python is often deemed as a good choice to start learning programming with, as its syntax is relatively simple.
Python directs its users towards good programming style by using indenting to mark structural blocks in programs (most other well-known languages - e.g. Java, C or Pascal - allow free placement of program statements and denote blocks with keywords or symbols).
As fully free and open-source software, it is included by default in most modern Unix-based operating systems (including OS X as well as almost all Linux distributions and BSD Unixes).
Python is also used to write some system software, e.g. the graphic installers used by Ubuntu, Red Hat Enterprise Linux and Fedora.
It is used for writing scripts in LibreOffice (similar to the role of Visual Basic for Applications used in MS Office), GIMP graphics package and a number of graphics/animation packages like 3D Max, Maya, Lightwave and others.

Note that Python is currently available in two parallel flavours. Python 2 was released in 2000 and its current stable version is 2.7. However, in 2008 the creator of Python decided to fix some inconsistencies in the language, resulting in the new Python 3 being incompatible with Python 2. Most systems that include Python have opted for Python 2 so far - therefore, this version (2.7) is also used at our course.

Basics of programming

Note: this section gives a very rudimentary overview of programming, as plenty of online materials (on Python and other languages) is available.

Languages

Programming is typically done in so-called high-level languages - they are command systems that could be described as something in between of math and human languages. First widely used languages date back to the 60s (FORTRAN, COBOL), current popular choices include C (with its derivatives like C++, C# and others), Java, Python, Ruby, BASIC (while the language has long history, its current versions like Microsoft's Visual Basic are modern languages) and others.

High-level languages are designed to be understood by people rather than computers. To be understood by the computer (or rather, the operating system), they must be translated into machine code. Based on the method of translation, programming languages can be roughly divided into two categories (today, several intemediate and hybrid methods also exist):

interpreted languages - the program is read and executed by special software (the language interpreter) row by row (more exactly, statement by statement).
compiled languages - use another kind of software (the compiler) that translates the whole program (provided it is error free), resulting in a machine-language version of the program. The compiled program is a separate entity that can be run, copied etc (the .exe files in MS Windows are a well-known example of compiled programs). However, in order to make changes, the original high-level version (called the source code) is typically needed - after making changes, the source program must be re-compiled.

Python in its basic form is an interpreted language - we can launch Python in OS X or Ubuntu, and use simple commands which are obeyed immediately. However, compiled versions of Python do exist as well.

Main elements of a program

At the most general level, a program consists of three kinds of "building blocks":

sequence
selection (choice)
iteration (cycle)

We may illustrate it with a simple recipe for oat porridge:

"Take a cup of oat flakes and a glass of milk. Pour milk to a pot and heat it. Put oats into the pot and stir. If the mix starts to boil, cut down on heat and let it simmer for 10 minutes. Keep stirring until the porridge is ready. Add some salt and sugar - if you plan to eat the porridge with hot butter, keep it more salty, otherwise make it sweet."

Here, most instructions are given as a sequence - "do this, then that". However, there are two selections - the first is at the condition of "the porridge boils" - in a programming language, it can be written as

IF porridge boils THEN cut down on heat

Another selection is at the end - we may write it like this:

IF I plan to eat the porridge with hot butter THEN
   Add more salt than sugar
ELSE
   Add more sugar than salt

Finally, there is also an iteration - a part of program that needs to be repeated until an ending condition appears. We may write

DO
  let the porridge simmer
UNTIL 10 minutes have passed

(Note: this is one of the three main variants of iteration in programming; see below)

Basics of Python

In Unix-like systems, the interactive environment (the Python shell) is launched from a terminal (command line) with just "python". Launching a Python program (typically with .py extension) along with the environment is done with "python programname.py".

First steps in the Python shell

After lauching the Python shell, it displays a prompt. An example:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Here, the ">>>" shows the system being ready to receive commands.

A simple use for the Python shell is as a calculator. We can enter a formula and it is immediately calculated by the interpreter:

>>> 5 + 2
7
>>> 3 * 4 + 10 / 2
17
>>>

NB!!! Somewhat awkwardly, exiting the Python shell is done via the key combination of Ctrl-D. More logical commands like "exit" and "quit" also work, but need empty parentheses added - quit() and exit().

Variables

Variables are used in most programming languages. Technically, they are named locations in computer memory that can hold a value. The main purpose is the increased flexibility, as the value of an variable can be changed on the run. Variables can be of different types (e.g. integers or textual values), contain single elements or more complex structures. The exact way of handling variables depends on the programming language used.

In Python, we can again start with the shell:

>>> a = 3
>>> b = 6
>>> a + b
9
>>>

In this example, the variable "a" is assigned the value of 3 and the variable "b" the value of 6. Now, instead of summing the values, we can sum the variables - the interpreter sums their values and outputs the result.

Noteː the first statement should be read as "assign 3 to a" rather than "a equals 3" - many programming languages (Python included) use different symbols for checking equality (the equality check is often written in the form of "x == 2"). However, in some languages (e.g. BASIC) both use the same symbol, making it easier to get confused.

If we now assign "b" a new value of 2, the result will be:

>>> b = 2
>>> a + b
5
>>>

While "a" remained at 3, "b" received a new value and the resulting sum is different.

Outputting values and the first programs

The Python interpreter prints out resulting values by default. However, in programs, we need to use the "print" command. The classic first program in all programming languages is called "Hello World!" - often, it is a single-line program like in Python:

print "Hello World!"

Note: in Python 3, parentheses are needed: print("Hello World")

We can try it out by writing the line into a file named e.g. hello.py and then launching the program along with the Python shell with "python hello.py".

For the second example, let's do the same addition as above, but now with a separate program.

Note: Python programs can be written with any clear-text editor. In Windows, Notepad is fine - however, WordPad or Word need to be told separately to save the file as pure text, as they add formatting information to their documents by default. In Unix-like systems, we can edit the files in the same terminal in parallel with using the Python shell (Nano is probably the easiest to use and is provided by many systems), or use graphical editors like Gedit or Kate. In OS X, TextEdit works (in newer versions, it will write "rich text" - with formatting - by default. Switching to cleartext mode is done with pressing CMD-Shift-t).

Open an editor of choice and create a file named sumitup.py:

a = 3
b = 6
print a + b

Launch it with "python sumitup.py"

Note: this is the simplest form of output. In practice, the output is usually formatted using some extra settings. Also notice that textual values are in quotes while numerical ones are not.

Input

Many programs need to obtain some data from the program user and use it to calculate results. In Python 2, this is done with "raw_input" (substituted by just "input" in version 3):

print "We will now add some numbers."
a = raw_input("Please enter the first number: ")
b = raw_input("... and now the second one: ")
total = int(a) + int(b)
print "The total of these is ", total

Notes:

In Python, the "raw_input" is technically a function - a subprogram that accepts parameters in parentheses and returns a value. Here, the variable "a" and "b" are assigned the values entered by the user.
the input is initially interpreted as textual. Therefore, to convert the values into numbers, we need the int() function (meaning "turn the value into an integer"). The conversion could have been done also on input - in this case, e.g. the second line would start as "a = int(raw_input("Please... " and the fourth would be just "total = a + b" (but if the conversion is omitted altogether, the result would be concatenation of two textual values: "3" and "6" result in "36"). The reverse - from integer to text - is obtained by str() function (in programming, textual values are commonly referred to as "strings").

Selection

In most programming languages, the simple selection is known as the if-statement. A simple example:

mynumber = 7
yournumber = int(raw_input("Please enter a number: "))
if (mynumber > yournumber):
  print "My number is bigger!"

This program also introduces an important concept of program structure. So far, our examples were simple sequences. Selection, however consists of several lines of code - in this case, the condition and statement(s) that are used only when the condition is met (these statements form a condition block). In Python, program blocks are defined by indentation. Therefore, the last "print" belongs to the if-block and is shifted right. Notice also the colon at the end of the condition line - it shows that the line is part of a multi-line structure.

Note: this example uses the int-conversion at the raw_input() as mentioned above.

NBǃ Indents of the same level (the program can contain several nested sub-blocks) must be in line (starting at the same position) throughout the program - otherwise, the program will not runǃ For example

main program
block 1
  block 1 sequence - shifted to the right by two positions
main program continues - not shifted
block 2
  block 2 sequence - shifted by 2
  block 3
    block 3 sequence - shifted by 4
  block 2 sequence continues - shifted by 2
main program continues - not shifted

A more complete version of the example would be:

mynumber = 7
yournumber = int(raw_input("Please enter a number: "))
if (mynumber > yournumber):
  print "My number is bigger!"
else:
  print "My number is smaller or equal to yours."

This adds the "else" part that is used when the initial condition is not met. Both if- and else- blocks can consist of any number of lines - but all the lines must be likewise indented.

Finally, when the number of options is more than two, we can use the "elif" clause (shortened from "else if"):

mynumber = 7
yournumber = int(raw_input("Please enter a number: "))
if (mynumber > yournumber):
  print "My number is bigger!"
elif (mynumber < yournumber):
  print "My number is smaller then yours."
else:
  print "Our numbers are equal!"

There can be several elif- clauses (e.g. in a choice between 10 different options).

NB!!! In Python, the single equal sign (=) denotes assigning a value to a variable. To write a comparison (e.g. "if x equals 5"), a double sign must be used - "if (x == 5):".

Iteration

Typically, there are three kinds of iteration used in programming languages:

counted iteration (for)
pre-test iteration (while)
post-test iteration (do while)

However, Python only uses the two former ones, there is currently no post-test iteration in Python (although it is possible by combining other statements).

The counted iteration, called "for" in most languages, uses a special variable (counter) to count the steps (thus the "for" means "for every value of the counter (do something)"). The version used in Python is a little different from many other languages (in some of them, a "for each" does similar things to "for" in Python). In Python, the iteration is used to move through a range of numbers or elements of a set. For example, we can count numbers from 1 to 12 (e.g. for months in a year):

for x in xrange(1,13):
  print x

Note: The range limit must be larger by one!

We can also use the same statement to print a sentence letter by letter:

sentence = "I like Python!"
for x in sentence:
  print x

The pre-test and post-test iteration are rather similar, but the latter is always carried out at least once, while the former can also be totally ignored. They do not use a special control variable, rather they check the value of a variable already in use. In Python, "while" is used for the pre-test iteration:

counter = 1
while (counter < 5):
  print "The counter is now at ", counter
  counter = counter - 1

In this example, the while-block will check whether the value of the counter is less than 5 (it is, as it is initially set to 1) and then proceed until the condition is not met anymore (the counter reaches 5).

Note: the last line of the example is a classic confuser for beginners - they tend to interpret it as comparison ("How can something equal itself plus something?"), but it is actually assigning a "larger by one" value to the variable, or adding one to it.

If we set the counter to the initial value of 8, the program will not display anything at all - the initial check fails (the counter is already larger than 5) and the whole while-block is ignored.

There are also two commands which are often found in combination with the while-block:

break - stops the iteration and moves to the next statement after the while-block.
continue - ends the current round, moving to the main condition for the next round's check.

An example: Text entry with the length check (must be at least 5 characters):

while True:
  text = raw_input("Enter text (at least 5 symbols): ")
  if text == 'stop':
    break
  if len(text) < 5:
    continue
  print 'The text length was OK'

Note: this example uses a technique called "eternal loop". In this case, the main while-block keeps repeating until the ending condition ("stop" is entered) is met.

Additional reading and links

Note: there is plenty of materials online. Care must be taken though with the ones that rely on too old versions of Python.

The official home page. http://www.python.org/
Python tutorial. http://docs.python.org/2/tutorial/
Python Wiki: Python for Non-Programmers. https://wiki.python.org/moin/BeginnersGuide/NonProgrammers
6 Free E-Books on Learning to Program with Python. http://readwrite.com/2011/03/25/python-is-an-increasingly-popu

Study and Blog

Pick one of the programming tasks:
- Total beginners: write a simple game of "Sell Me an Animal!". You are a zookeeper, the program is a potential buyer of animals. It should ask you "What do you offer?" and react in three possible ways: a) exclaim "I take it!" on a certain animal, b) declare "I hate those!" on another, or c) say "Not interested!" on all the rest. HINT: Assign the two special cases (good and bad) to variables and compare the input values to these.
- Intermediate: the same as above, plus a) keep asking (using an iteration) until either the "good" or "bad" animal is reached, b) count the total number of animals offered and output it at the end.
- Advanced (NB! this text does not cover the necessary techniques - takes either previous knowledge or studying the manuals!): the same as above, plus a) read the list of animals from an external text file, and b) in case of the neutral answer, ask for another animal on the list whose name begins with the same letter. For example: "What do you offer?", "Monkey", "Not interested, do you have a moose?". Note that for this to work properly, the data file must have at least two animals with all letters (no need to do all the alphabet though - e.g. two with "a", two with "k" and two with "s" should work well).

Sample data for the lazy ones:

ant
ara
cat
cow
crocodile
dog
dolphin
donkey
elephant
elk
pig
puppy
snail
skunk
warthog
wolf
wolverine

Back to the main course page