7  Writing Pythonic Code

ImportantLearning objectives for this lesson

After this lesson you should…

  • be able to write your own custom functions
  • know the difference between positional and keyword arguments
  • be experts in the use of slice notation when indexing
  • understand how the splat operation works when unpacking collections
  • be comfortable with the use of the set collection and its special features
  • be able to use list and dict comprehensions in place of simple for-loops, including if-clauses
  • be able to enumerate over a collection, as well as zip multiple collections together
  • do nested for-loops to iterate over multiple directions
  • know how to specify default argument values in a function
  • know how to write a “docstring” for your custom functions
Figure 7.1: Programmers really care about coding looking nice. Some people consider code to be poetry.

When Guido created Python he was partly motivated by the realization that code is read more often than it is written. With this insight in mind he created a syntax that was as close to spoken English as possible. We have seen examples of this with statements like for item in collection. When code is written in a clean, concise, and consistent manner it becomes easy to read your own code as well as someone elses. Writing clean code is called “pythonic code”.

Pythonic code means two things. First is that the formatting of the code follows the standard conventions:

To facilitate nice and consistent code, there is a style guide generally referred to as pep8.

Notepep: Python Enhancement Proposal

The Python community created a process where new features can be proposed for the language. These proposals are called “PEPs”, an acronym for “Python Enhancement Proposal”. A numbered list of proposals is available, along with their current status.

The second aspect of Pythonic code is how the code is written, meaning the method used for certain common tasks uses the features of Python to their fullest. Examples include:

7.1 PEP8, The Python Style Guide

PEP8 was created very early on, and has been a core part of Python ever since. This PEP basically describes how Python code should look. A condensed version of this guide, including examples is given here.

Here are a few highlights:

  • Indent each new logical block by 4 spaces
  • Use 2 blank lines between top-level function or class definitions, and 1 between methods within a class
  • Use “snake_case” for variable and function names (‘snake_case’ instead of ‘camelCase’ or ‘PascalCase’)

Most of these are style recommendations, not Python syntax requirements. Consistent indentation is required by Python, but it could be 2 or 10; using specifically “4” spaces per level is the style convention.

Instead of listing off all the rules, the following table demonstrates several (but not all):

Table 7.1: Summary of some coding convention do’s and don’ts.
Yes No
spam(ham[1], {eggs: 2}) spam( ham[ 1 ], { eggs: 2 } )
spam(1) spam (1)
dct['key'] = lst[index] dct ['key'] = lst [index]
i = i + 1 i=i+1
submitted += 1 submitted +=1
x = x*2 - 1 x = x * 2 - 1
hypot2 = x**2 + y*y hypot2 = x ** 2 + y * y
c = (a+b) * (a-b) c = (a + b) * (a - b)
c = (a+b) / (a-b) c = (a + b)/(a - b)
def complex(real, imag=0.0): def complex(real, imag = 0.0):
return magic(r=real, i=imag) return magic(r = real, i = imag)

For a block containing several statements, put each statement on its own indented line:

if x == 4:
    print(x, y)
    x, y = y, x

7.1.1 Linters

Since there are so many rules, it takes quite a while to learn them all. Luckily, there is a tool we can use called a linter. In IDE’s such as Spyder and VSCode it is possible to enable linting using a plugin or optional feature. The result looks like this:

Figure 7.2: Screenshot of the Spyder IDE showing orange triangle on each line with a PEP8 violation, and a pop-up describing the specific problem.

7.2 A Deeper Dive into Collections

7.2.1 Slicing

We briefly covered slicing, but it will come up a lot more when we start to talk about numpy, so let’s take a closer look. We can use the : symbol to indicate a range of indices.

It works like this: start:stop:step. Each of these can be excluded, and defaults will be used:

  • Default for start is 0.
  • The default step is 1
  • The default stop is None which means it goes to the end
Figure 7.3: A visual representation of indexing and slicing. The green squares are what is retrieved with normal indexing. The blue squares indicate the elements that will be retrieved when indexing, but counting backward. Finally, the orange squares indicate the elements that will be retrieved using normal slicing.

Using the defaults gets us the entire list:

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1b = a[:]
print(b)
1
A single : is a shortcut that means the whole list. It is like writing a[0:len(a)]. In contrast, a[0:-1] excludes the last item because the stop is excluded. 
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

We can change the start and stop values to get a section of the list:

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1b = a[2:6]
print(b)
1
Here we start at 2 and stop at 6. It is crucial to note that it stops before 6, not at 6. 
[2, 3, 4, 5]

The end point can be specified as distance from the start, or distance from the end by using a negative number:

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1b = a[2:-3]
print(b)
1
Here we start at 2 and stop at 3 from the end. 
[2, 3, 4, 5, 6]

Changing the step size is also possible if we want every Nth item:

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1b = a[2:8:2]
print(b)
1
The third value is the step size, so in this case it retrieves every second item between 2 and 8. The default is 1, so it can be excluded to get the normal behavior. 
[2, 4, 6]

And of course the step can be negative if we want to walk through the collection backwards:

a = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
1b = a[6:2:-1]
print(b)
1
The step size can be negative too, which is useful for reversing the order of the items 
[6, 5, 4, 3]

Example 7.1 (Using complicated slicing notation) Given a string, retrieve every third letter, starting 5 from the end, and stopping at the beginning.

Solution

s = 'The_quick_brown_fox_jumped_over_the_lazy_dog'
s2 = s[-5::-3]
print(s2)
ylhroeux_o_i_T

Note that writing s[-5:0:-3] would exclude index 0; the omitted stop in s[-5::-3] allows us to reach it.

Comments

To check this independently, the string has 44 characters, so index -5 is index 39. Starting there and subtracting 3 repeatedly gives the following indices, including index 0.

l2 = [39, 36, 33, 30, 27, 24, 21, 18, 15, 12, 9, 6, 3, 0]
print(l2)
[39, 36, 33, 30, 27, 24, 21, 18, 15, 12, 9, 6, 3, 0]

Now we can retrieve the letters from s using the indices in l2:

s3 = ''
for i in l2:
    s3 = ''.join((s3, s[i]))
print(s3)
ylhroeux_o_i_T

And finally, let’s officially confirm that the s3 is equal to s2:

print(s2 == s3)
True
WarningThe Splat Operator

This falls under the category of “you may never use this, but you might see it in someone elses code”

Python allows us to unpack values in a collection into separate variables, but provides an option to keep some variables together. This is sometimes called the “splat” operator:

a, *b = 1, 2, 3
print('a is:', a)
print('b is:', b)
a is: 1
b is: [2, 3]

Or:

a, *b, c = 1, 2, 3, 4
print('a is:', a)
print('b is:', b)
print('c is:', c)
a is: 1
b is: [2, 3]
c is: 4

7.2.2 Sets

Another type of collection is called a set. This word refers to the mathematical concept of a set. A set contains no duplicate elements and is unordered. Use set() for an empty set, since {} creates an empty dictionary.

A set is defined by putting curly braces around a comma-separated sequence of values:

1s = {1, 2.0, 'text'}
print("Result:", type(s))
1
The use of curly braces is a bit unfortunate since this looks a lot like the definition of a dict. The difference is that the collection only contains values, wheres in a dict it contains key:value pairs. There is actually some logic behind this: a set cannot contain any duplicate items, and similarly a dict cannot contain any duplicate keys. So {'a': 1, 'a': 2} will become {'a': 2}. 
Result: <class 'set'>

Python also includes a built-in function for creating sets:

1s = set([1, 2.0, 2.0, 2.0, 'text'])
print("Result:", s)
1
Note that we have included multiple instances of 2.0, yet the resulting set only has one instance. 
Result: {1, 2.0, 'text'}

Probably the most useful application of sets is for “Venn-diagram” style logic. For instance, if you have two lists of values, you might want to know what they have in common, or what are their differences. This can be accomplished with sets.

A Venn diagram that might be more relatable to a room full of engineers!
Figure 7.4: Examples of Venn diagrams
Figure 7.5: Visual representation of operations between 2 sets

Consider the following 2 sets:

a = {'bob', 'dave', 'jane'}
b = {'dave', 'sue', 'taylor'}

We can find what the two sets have in common by finding the intersection:

a = {'bob', 'dave', 'jane'}
b = {'dave', 'sue', 'taylor'}
c = a.intersection(b)
print("Result:", c)
Result: {'dave'}

Or we can find how they differ using difference:

a = {'bob', 'dave', 'jane'}
b = {'dave', 'sue', 'taylor'}
c = a.difference(b)
print("Result:", c)
Result: {'jane', 'bob'}

Table A.5 gives a list of the methods available on a set object which can be used for performing a variety of other “set operations” such those shown in Figure 7.5. An abbreviated list is given below with just the ones associated with “Venn-diagram”-type logic.

Table 7.2: Partial list of methods on a set object, including the shortcut keys which have been defined
Method Shortcut Description
difference() - Returns a set containing the difference between two or more sets
intersection() & Returns a set, that is the intersection of two other sets
issubset() <= Returns whether another set contains this set or not
< Tests for a proper subset: all items belong to the other set, but the sets are not equal
issuperset() >= Returns whether this set contains another set or not
> Tests for a proper superset: this set contains the other set, but the sets are not equal
symmetric_difference() ^ Returns a set with the symmetric differences of two sets
union() | Return a set containing the union of sets

Example 7.2 (Using Sets) We can write chemical formulae in the following form: ['C', 'H', 'H', 'H', 'H'] which would be methane. Write a function which accepts 2 such lists and returns a list of elements which are in common.

Solution

def common_elements(chem1, chem2):
    chem1 = set(chem1)
    chem2 = set(chem2)
    common = chem1.intersection(chem2)
    common = list(common)
    return common

d = dict()  # Could also use empty curly braces - {}

d['ethane'] = ['C', 'C', 'H', 'H', 'H', 'H', 'H', 'H', ]
d['methanol'] = ['C', 'O', 'H', 'H', 'H', 'H'] 

print(common_elements(chem1=d['ethane'], chem2=d['methanol']))
['C', 'H']

Comments

Here we have used a dictionary to store the chemical formulae, and taken advantage of the ability to use readable words as the keys. Converting the intersection to a list does not guarantee the order of the returned elements. Use sorted(common) instead of list(common) if alphabetical order is required.

7.3 For-Loops Again

7.3.1 Enumerating while Iterating

We have seen For-Loops which iterate over the items in a collection:

c = ['a', 'b', 'c']
for item in c:
    print(item)
a
b
c

And we have seen For-Loops which using an index into a collection:

c = ['a', 'b', 'c']
for i in range(len(c)):
    print(c[i])
a
b
c

But we can do both!

c = ['a', 'b', 'c']
for i, item in enumerate(c):
    print(i, item)
0 a
1 b
2 c

Having access to the item and to the index where is located is quite useful:

c = [22, 33, 44]
for i, item in enumerate(c):
    c[i] = item/11
print(c)
[2.0, 3.0, 4.0]

7.3.2 Looping over Two Collections Simultaneously

We sometimes have 2 collections which we want to scan together, for example to combine the values into a dict. We can use the zip function as follows:

names = ['a', 'b', 'c']
nums = [1, 2, 3]
d = {}
for key, value in zip(names, nums):
    d[key] = value
print(d)
{'a': 1, 'b': 2, 'c': 3}

By default, zip stops when its shortest input is exhausted.

7.3.3 Nested For-Loops

We have been mostly looking at linear collections like lists and strings. However, we often have “multidimensional” data, like a table or spreadsheet, which can be thought of as “list of lists”. In such cases we need to use one for-loop to scan over the “outer” list, then a second for-loop to scan each “inner” list.

Another way to think about this is a 2D (or ND!) array of data, where one for-loop scans each row, then the second loop scans the columns of that row. Consider the following case of a 5-by-5 matrix:

Example 7.3 (Process each number in a list-of-lists) Compute the square-root each number in the following 2D array:

data = [[1, 2, 3, 4, 5],
        [4, 2, 6, 9, 1],
        [9, 4, 7, 8, 4],
        [1, 6, 2, 8, 2],
        [3, 4, 8, 4, 4]]

Solution

for row in range(len(data)):
    for col in range(len(data[row])):
        data[row][col] = round(data[row][col]**0.5, 5)

Now let’s print our data array, one row at a time:

for row in data:
    print(row)
[1.0, 1.41421, 1.73205, 2.0, 2.23607]
[2.0, 1.41421, 2.44949, 3.0, 1.0]
[3.0, 2.0, 2.64575, 2.82843, 2.0]
[1.0, 2.44949, 1.41421, 2.82843, 1.41421]
[1.73205, 2.0, 2.82843, 2.0, 2.0]

Comments

In languages other than Python, it is common to have 2 or 3 levels of nested for-loops which corresponds to 2D and 3D arrays. Beyond that, the code and the data become quite hard for us simple humans to understand.

In Python, doing for-loops to analyze arrays of data like above is quite rare! The reason is that Python is quite slow at this, for reasons to be discussed in the next lecture when we are introduced to numpy.

7.4 Comprehensions

7.4.1 List Comprehensions

Python offers an alternative way to build a collection in a single line. This can make simple transformations easier to read, though fewer lines do not automatically mean clearer code:

result = [item/4 for item in [-1, 0, 2, 3]]
print("Result:", result)
Result: [-0.25, 0.0, 0.5, 0.75]

Technically this is called a list comprehension. They can be confusing at first, but eventually you may appreciate the conciseness they offer. It often helps readability to condense logic to a single line.

Example 7.4 (Compare for-loop and list comprehension) Given a list of strings, each a single word, capitalize each word using both a for-loop and a list comprehension

Solution

strings = ['firstname', 'lastname']

1new_1 = []
for i in strings:
2    new_1.append(i.capitalize())

3new_2 = [i.capitalize() for i in strings]
1
We must start by defining a new, empty list to which we’ll add the capitalized words
2
Here we do the capitalization and append it to new_
3
This version creates a new list inside the [ ], then we catch it in new_2

Comments

Doing this on a single line is helpful for readability, but only if the for-loop was simple. It is technically possible to do list-comprehensions inside other list comprehensions, which is equivalent to nested for-loops, but these are quite difficult to read/understand, so defeat the purpose.

7.4.2 Dict Comprehensions

Python supports list, dictionary, and set comprehensions. There is no tuple comprehension.

Dictionary comprehensions are helpful for combining two different data sources into a single dictionary. For instance if we have a list of names, and another list of values, we can do:

name = ['a', 'b', 'c', 'd', 'e']
data = [1, 2, 3, 4, 5]
1d = {name[i]: data[i] for i in range(len(data))}
print(d)
1
Note that this looks a lot like defining a dict, like d = {'a': 1, 'b': 2, 'c': 3}. 
{'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5}

7.4.3 Combining If-Statements and Comprehensions

Recall the “One-line If-Statement”:

limit = 5
value = 'a' if 4 < limit else 'b'
print(value)
a

We can combine this if condition in comprehensions too:

vals = [1, 2, 3, 4]
limit = 3
1a = [i for i in vals if i < limit]
print(a)
1
The final if filters the input: only values below limit are included, so this result is [1, 2] and may be shorter than the original list. 
[1, 2]

7.5 Error Catching

In previous weeks we have discussed the differences between types and how Python tries to treat each variable accordingly. However, sometimes this just isn’t possible for Python and an error occurs. So far we have focused our energy on trying to intercept the variables and decide how to treat each one, but Python offers a way to handle this: try and except.

The basic idea is that we try to run some code first, and only if it fails do we do something else.

try:
    c = a + b
except:
    print('a and b are not compatible')

This will catch every error; however, it is good practice to only catch errors which you are expecting. In the above case our print statements suggests that we are only expect TypeErrors, so we should write:

try:
    c = a + b
except TypeError:
    print('a and b are not compatible')

Let’s look at an example:

Example 7.5 (Using try/except) Write a function which accepts a finite built-in int, float, or complex number and converts it to an int. A float should be truncated toward zero; for a complex number, truncate its real part toward zero.

Solution

def convert_to_int(a):
    try:
1        b = int(a)
    except TypeError:
2        b = int(a.real)
    return b

x = 3.5 + 2.2j
y = convert_to_int(x)
print("The result is:", y)
1
We know this will cause a TypeError if a is complex
2
real is a attribute of complex numbers which contains the real part 
The result is: 3

Comments

This approach attempts the operation directly and handles a specific failure.

NoteTry-Except blocks

There are more clauses we can add to a try statement. The else block runs if the try block completes normally; finally runs when leaving the statement, including when an exception propagates. This schematic uses pass as a valid placeholder for code:

try:
    pass  # Attempt an operation
except ValueError:
    pass  # Handle an anticipated ValueError
else:
    pass  # Execute if try completed normally
finally:
    pass  # Cleanup when leaving the statement

These extra clauses are not very common, but are mentioned so if/when you see them in the wild, you’ll know what you’re seeing.

7.6 Writing Custom Functions

In addition to the built-in functions we learned about (i.e. float(3)), and the massive collection of extras we can install and import (i.e. math.cos(3)), we will still want to create our own custom functions. The construction of such a function is quite simple:

1def sqrt(x):
2    y = x**0.5
3    return y
1
There are three important things happening on this line. (1) The keyword def means define as in here we define a function. (2) We will call our function sqrt, which is the name we will use when we call it. (3) Inside the round brackets we list all the variables that our function requires or at accepts.
2
Here we write the logic of our function. This can be as many lines long as we want. Note indentation of the lines following the :! Whitespace like this crucial in Python. Refer to here: Important 7.1 for more info. The code is is identical to what we’ve been putting the cells of our Jupyter notebooks.
3
Finally we return the result of the calculation back to the caller.
Important 7.1: Whitespace in Python

Python is quite flexible about many things, but it is not flexible about whitespace at all!. When Guido van Rossum created Python in 1991 he had several guiding principles. One of them was the realization that “code is read more often than it is written”. For this reason he designed the syntax to use whitespace to denote logical blocks rather than brackets and braces like other languages (i.e. C/C++). The mandatory indentation makes code quite easy for a human to read since the logical blocks are instantly identifiable.

We can use this function in our program as follows:

1def sqrt(x):
    y = x**0.5
    return y

2a = 9
3c = sqrt(x=a)
1
We have to define a function before we use it, so place the definition at the top of the script.
2
Note that the variable name a has nothing to do with the variable name x inside the function.
3
Just because we assign the result to y inside the function, does not mean that y is available outside the function. The numerical value gets returned, not the variable y. We catch the value of 3 and assign it to any variable name we like, in this case c.
WarningCommon Errors
  • return exits the function immediately and passes the value to the caller.
  • A function that reaches its end without return, or uses return alone, returns None.
  • Printing a value inside a function only displays it; it does not return that value.
  • Names assigned inside the function are local to that function. A variable stored in a can be passed to b, and a inside the function does not change the original a.

Example 7.6 (Create a custom function for solving the quadratic equation) The quadratic equation occurs a lot in engineering. It arises when trying to solve equations of the form:

\[ ax^2 + bx + c = 0 \]

Write a function that accepts real coefficients \(a\), \(b\) and \(c\), with \(a \ne 0\), and returns both roots. A repeated root is returned twice; a negative discriminant produces complex roots, but we don’t even need to check for this since Python will produce complex numbers.

Solution

def roots(a, b, c):
    A = 4*a*c
    B = b**2
    C = 2*a
    ans1 = (-b + (B - A)**0.5)/C
    ans2 = (-b - (B - A)**0.5)/C
    return ans1, ans2

x1, x2 = roots(5, 1, -1)
print("Result:", x1, x2)
Result: 0.35825756949558396 -0.558257569495584

Comments

This function has a few features we have not seen yet:

  • It accepts 3 values, (a, b and c). These are called “arguments”.

  • It has multiple lines of logic within it, which more clearly demonstrates the point of hiding complex logic inside a function. We only need to write one statement (x1, x2 = roots(5, 1, -1)) yet 5 lines of logic are run.

  • It returns two values, separated by a ,. We also catch the two values into separate variables simultaneously (x1 and x2). This follows the rules of tuple handling which we’ve seen in previous chapters.

There are a few more details about how functions behave that we we can learn from our roots function:

  • If we supply arguments only as values then Python will assume their order matches the order in the function definition. This means that roots(5, 1, 0) will make on to a=5, b=1 and c=0.

  • If we explicitly name the arguments, we can put them in any order. For instance roots(c=0, a=5, b=1) will work just fine.

  • If we choose to mix both of the above, then the unnamed arguments must go first, like roots(5, c=0, b=1). In this case 5 is assumed to mean a, then c and b are assigned to their respective arguments.

7.7 Advanced Functions

7.7.1 Positional vs Keyword Arguments

To start with, let’s identify the difference between position and keyword arguments. Consider the following function:

def add_values(a, b):
    return a + b

We can call it using positional arguments:

c = add_values(2, 2)

Or keyword arguments:

c = add_values(a=2, b=3)

When using keywords we can do this:

1c = add_values(b=3, a=2)
1
Note that the order of the arguments is not the same as the function definition. This works because Python will map a=2 onto a and b=3 onto b.

Lastly, note that positional arguments must come first, and then keyword arguments can follow:

c = add_values(2, b=3)

7.7.2 Default Arguments

In many functions there are some arguments which we don’t usually want to change. An example is something like mode='encrypt'.

This can be done in the function definition as follows:

def add(a, b, mode='normal'):
    if mode == 'normal':
        d = a + b
    elif mode == 'absolute':
        d = abs(a) + abs(b)
    else:
        raise ValueError("mode must be 'normal' or 'absolute'")
    return d

print(add(-2, 3))  # Uses the default mode='normal'
print(add(-2, 3, mode='absolute'))
1
5

The supported modes are 'normal' and 'absolute'. An unsupported mode raises an informative ValueError rather than reaching return with d undefined.

7.7.3 Unpacking Arguments

If you have a list or tuple of values, such coefficients (e.g. [1, 3, 5]), that you want to pass into a function, you can do the following:

def func(a, b, c):
    print(a, b, c)

vals = [1, 3, 5]
1func(*vals)
1
The * (asterisk) tells python to unpack the values in vals and place them into a, b, and c of the function. Obviously, the length of vals has to match the number of arguments in func. This is equivalent to the “splat” operation mentioned early. 
1 3 5

And taking this to the next level, we can use dicts as keyword arguments, where the key corresponds to the argument name.

def func(a, b, c):
    print(a, b, c)

vals = {'a': 1, 'c':5, 'b': 3}
1func(**vals)
1
The ** (double asterisk) unpacks vals as keyword arguments, so the value in vals['a'] (which is 1) is passed to parameter a. For this function, the dictionary must supply each of a, b, and c, with no unexpected keys. 
1 3 5

This functionality is important because there are many packages where functions are designed to work together. The next example is a preview of NumPy arrays, SciPy statistics, and plotting, which we will study later.

Example 7.7 (Using results from one function as arguments to another) Consider that we have data points, such as pumpkin weights, and want to fit a normal distribution to them. We will generate 1000 artificial observations from a normal distribution with mean 10 and standard deviation 2:

import numpy as np

data = np.random.normal(size=1000, loc=10, scale=2)

Solution

Now let’s use the functions in the scipy.stats module to fit a normal distribution to our data:

from scipy import stats

params = stats.norm.fit(data)
print(params)
(np.float64(9.986765323323013), np.float64(1.9545642402692476))

Finally, let’s plot a graph of the cumulative distribution curve using the values of params we found above:

import matplotlib.pyplot as plt

1x = np.linspace(0, 20, 100)
2y = stats.norm.cdf(x, *params)

3fig, ax = plt.subplots(figsize=(4, 4))
ax.plot(x, y, c='tab:blue')
ax.set_xlabel('Value')
ax.set_ylabel('Fraction of values smaller than stated value');
1
Here we generate 100 values between 0 and 20, equally (i.e. linearly) spaced
2
Here ask scipy.stats.norm to generate y values for the given x values using the params we found for this particular dataset.
3
This is a call to the matplotlib library which we’ll cover in more detail in a few chapters

Comments

Plotting data is a huge part of why we crunch numbers on a computer. There are a lot of libraries for plotting in Python, each has their own angle, but matplotlib is the OG. Everyone else needs to justify why you should use their library rather than matplotlib. Below is a list of the more popular ones, some of which we’ll revisit later.

7.7.4 Accepting an Unlimited Number of Arguments

There are times where we want to write a function that accepts an undetermined or unlimited number of arguments. Python has a two ways to accept an undetermined number and/or name of arguments. They both use a variation of the * and ** notation mentioned above.

We can supply a set of positional arguments like this:

1def func(*args):
2    return args

3print(func(1, 2, 3))
1
Note that we use *args with a single *. Also note that we don’t have to use the variable args, but this is convention.
2
Here we simply return args to the user
3
When we print the returned result we see it is a tuple containing the values which we sent into func, in order. 
(1, 2, 3)

And we can supply “keyword” arguments like this:

1def func(**kwargs):
    return kwargs 

2print(func(a=1, b=2, c=3))
1
Here we use two **’s. Again note that using the variable name kwargs to catch these arguments is not mandatory but is convention.
2
When we print the result we see it is a dict with the given arguments as keys. 
{'a': 1, 'b': 2, 'c': 3}

The main thing to note with both of the above is that inside the function, args and kwargs both become just a normal tuple or dict. In other words, the code inside the function does not know that a, b and c were passed. These get rolled into a dict called kwargs which can be used inside the function.

Example 7.8 (Computing a polynomial with an arbitrary number of coefficients) Write a function which computes a polynomial with an arbitrary number of coefficients, like:

\[ y = a_0 + a_1x + a_2x^2 + \cdots + a_nx^n \]

Solution

1def poly(x, *args):
    y = 0
    for i, a in enumerate(args):
        y = y + a*(x**i)
    return y
1
In this definition, x receives the first positional argument and args collects all remaining positional arguments as a tuple of coefficients. 
coeffs = 3, 2, 4, -3, 8, 0, 0, 5
x = 3
y = poly(x, *coeffs)

And the answer is:

11547

Comments

Note that we can check that this function is behaving correctly by inputting coefficient values for which we know the answer. For instance we know:

\[ 3^2 = 9 \]

So:

coeffs = 0, 0, 1
x = 3
y = poly(x, *coeffs)
print(y)
9
NoteNext-level function definitions

It is also possible to mix and match *args and **kwargs if you really want to get fancy:

def func(*args, **kwargs):
    return args, kwargs
print(func(1, 2, 3, a=4, b=5, c=6))
((1, 2, 3), {'a': 4, 'b': 5, 'c': 6})

And to really make things messy, you can accept some keyword arguments explicitly, and even supply default values to some, then collect the rest into **kwargs:

1def func(*args, x, y, z=5, **kwargs):
    return args, kwargs
2print(func(1, 2, 3, a=4, b=5, c=6, x=1, y=2))
1
The parameters x, y, and z are keyword-only because they follow *args. Supplying x and y is mandatory; z defaults to 5. Extra positional arguments go into args, and extra keyword arguments go into kwargs.
2
The call supplies x and y by name and uses the default for z. These three parameters are separate from kwargs, so they are not included in this function’s returned dictionary. 
((1, 2, 3), {'a': 4, 'b': 5, 'c': 6})

The main use case for behavior like this is when calling another function from within a function, and using kwargs to collect the arguments for the inner function. For instance, let’s write a function which calls another function a specified number of times:

def inner(a, b, c):
    return sum((a, b, c))

def outer(n, func, **kwargs):
    s = 0
    for i in range(n):
        s = s + func(**kwargs)
    return s

result = outer(5, func=inner, a=1, b=1, c=1)
print(result)
15

The crucial thing to note above is that outer accepts inner as an argument and is able to call it without knowing any of its argument names. It just passes kwargs to func and assumes that the user supplied the correct arguments.

One place where we see **kwargs a lot is in Matplotlib functions, which we’ll introduce in a few weeks.

7.8 Writing Docstrings

We are now quite familiar with writing our own functions. Some functions can be complicated and need to be explained. This is what “docstrings” are for. A docstring is a string placed at the top of a function body, building on the strings introduced in Working with Text.

A basic docstring can be a single sentence:

def square(x):
    """Return x squared."""
    return x**2

For a longer description, scientific Python projects often use the NumPy documentation convention below. The section headings and their underlines are a documentation convention, not a Python syntax requirement:

def calc_area_and_perimeter(r):
    """Compute the area and perimeter of a circle.

    Parameters
    ----------
    r : int or float
        Nonnegative radius of the circle.

    Returns
    -------
    result : tuple of floats
        Area and perimeter, in that order. If r is in metres,
        area is in square metres and perimeter is in metres.
    """
    A = 3.14159*r**2
    P = 2*3.14159*r
    return (A, P)

Recall that triple quotes allow the string to span several lines.

Python’s built-in help function displays this documentation:

help(calc_area_and_perimeter)
Help on function calc_area_and_perimeter in module __main__:

calc_area_and_perimeter(r)
    Compute the area and perimeter of a circle.

    Parameters
    ----------
    r : int or float
        Nonnegative radius of the circle.

    Returns
    -------
    result : tuple of floats
        Area and perimeter, in that order. If r is in metres,
        area is in square metres and perimeter is in metres.

The full docstring can optionally contain any of the following section, if applicable:

Table 7.3: The defined sections in the official Numpydoc specification
Section Description
Short summary A one-line summary that does not use variable names or the function name
Deprecation warning To warn users that the object is deprecated
Extended Summary A few sentences giving an extended description.
Parameters Description of the function arguments, keywords and their respective types.
Returns Explanation of the returned values and their types.
Yields Explanation of the yielded values and their types.
Receives Explanation of parameters passed to a generator’s .send() method
Other Parameters Used to describe infrequently used parameters.
Raises Details which errors get raised and under what conditions
Warns Details which warnings get raised and under what conditions, formatted similarly to Raises.
Warnings Cautions to the user in free text/reST.
See Also Used to refer to related code.
Notes Provides additional information about the code, possibly including a discussion of the algorithm.
References References cited in the Notes section may be listed here
Examples For examples using the doctest format
NoteNumpydoc vs Google

There are several accepted ways ways to format docstrings, as well described by thist SO Discussion. I strongly suggest you use the numpydoc option since you’ll be dealing with numpy and other packages in the numpy universe which all use it.

A docstring as it appears in the source code

A docstring being converted to a webpage and placed on the project’s documentation site

A docstring being rendered by Spyder and shown in the help panel

A docstring being rendered by VSCode and shown as a popup window
Figure 7.6: Examples of the docstring being rendered by different user interfaces.

7.9 Creating Your Own Modules

Figure 7.7: The difference between a “script” and a “module”

Here are some definitions to help

  • A script is a file we execute to run a program. A notebook which we’ve been using up to now is a form of a script.

  • A module is a unit of Python code that can be imported. Our examples are .py files containing reusable functions. The same file can act as a script when executed and as a module when imported.

  • A package organizes modules under a common name, allowing imports such as numpy.linalg. A library is a broader term for reusable code and may contain several packages or modules. We often use existing libraries such as NumPy, pandas, SciPy, and Matplotlib.

Now let’s create a module with two functions: one for taking the square root of every element in a list, and one for squaring each element.

def square_it(collection):
    for i, val in enumerate(collection):
        collection[i] = val**2
    return collection


def root_it(collection):
    for i, val in enumerate(collection):
        collection[i] = val**0.5
    return collection

The above code should be placed in a new file called something like “myfuncs.py”, as shown below:

Figure 7.8: Screenshot of the myfuncs.py file containing our custom functions

We can then use the functions in this module in a script as follows:

from myfuncs import root_it, square_it

vals = [1, 3, 5, 9]
squares = square_it(vals)
print(vals)  # [1, 9, 25, 81]: the original list changed
print(squares is vals)  # True: both names refer to the same list

And the script would look like this:

Figure 7.9: Screenshot of myscript.py showing some calculations using the functions in myfuncs.py

7.9.1 Using if __name__ == "__main__" to Test Your Module

It is as important to test our code as it is to write it in the first place. In fact, in large important projects, more time is spent writing tests than writing code.

Like everything, there are many ways we can do tests, from simple (a script) to complicated (unit testing suites that are run by continuous integration services!).

The simplest way to include tests in our code is to add them at the end of our file; HOWEVER, when a module is imported everything in it is run. If our tests are long, or perhaps they are broken, then importing our module will be stalled or broken. We can fix this by using another of Python’s features: if __name__ == "__main__".

Let’s update our myfuncs.py file:

def square_it(collection):
    for i, val in enumerate(collection):
        collection[i] = val**2
    return collection


def root_it(collection):
    for i, val in enumerate(collection):
        collection[i] = val**0.5
    return collection


1if __name__ == "__main__":
    # The code inside this "if block" is ONLY executed 
    # when you "run" this file. It is NOT run when you 
    # import the functions from this file in a script
2    vals = [1, 2, 3]
3    ans = [1, 4, 9]
4    test = square_it(vals)
5    print(test == ans)

    vals = [1, 4, 9]
    ans = [1, 2, 3]
    test = root_it(vals)
    print(test == ans)
1
Python sets __name__ to '__main__' when it executes the file as the main program. When another file imports myfuncs, its __name__ is 'myfuncs', so the guarded block is skipped. Function definitions and other top-level statements outside the guard still execute during import.
2
Here we define some data which we use in our test
3
This step is crucial: here we define the EXPECTED output given the test data
4
Next we run our function on the test data
5
Lastly, we check that our function gave the expected answer. In this case we print True/False, so can visually look at the output and see if anything failed. 
True
True
Important

With the above if __name__ == "__main__" block added to our module file, we can run the file after changes to check that these particular examples still work.

We can add as many tests as we want/need to check all the “edge cases”.

7.10 Assignment 6

Submit one Python file named A6.py containing the six functions A6Q1 through A6Q6.

Each function must return its result; printing the result is not sufficient. The checker imports your file and calls each function with its own inputs. Keep any code used to test your functions inside an if __name__ == "__main__": block so that it does not run when the file is imported. Notebook cell tags and assignments to answer are no longer needed.

Tip

Install or update the checker using the instructions in Appendix E. Run python -m ne111_checker A6 on Windows or python3 -m ne111_checker A6 on macOS from any folder. You no longer need to start the checker in the same directory as your file. Just launch the checker from anywhere then cClick Browse… in the checker to select your saved A6.py, then select a question tab and press its Run button. Save changes to your file before running the check again; the checker rereads the selected file automatically.

Given a list, tuple, or string, return a subset starting and stopping at the specified locations, using the specified step size. Use slice notation. If start, stop, or step is omitted, use Python’s default slicing behaviour.

Function: A6Q1(c, start=None, stop=None, step=None)

Examples:

A6Q1([1, 2, 3, 4, 5], 1, -2)  # Returns [2, 3]
A6Q1(c='abcdefghi', start=-4, stop=2, step=-1)  # Returns 'fed'

Given 2 lists, find the elements in common and the elements present in only one of the lists. Return a tuple containing these two sets, in that order. Repeated elements appear only once in each set.

Function: A6Q2(first, second)

Examples:

A6Q2([1, 2, 3], [2, 3, 4])  # Returns ({2, 3}, {1, 4})

Fix all the whitespace mistakes in the following function according to PEP 8. Preserve the parameter names, default values, calculations, and order of the returned roots.

def A6Q3(a, b=1, c = 5):
    ans1=( -b + (b**2-4*a*c) ** 0.5 ) /(2*a) # One root
    ans2 = (-b-(b**2-4*a*c)**0.5)/(2*a)  # The other root
    return(ans1, ans2)

Write a function which accepts two arbitrary values and attempts to multiply them together. If multiplication is not possible, return None.

Function: A6Q4(a, b)

Examples:

A6Q4(2, 5)  # Returns 10
A6Q4('abcd', [1, 2, 3])  # Returns None

Write a function which accepts between 1 and 4 integers and multiplies them all together. Use default arguments so that omitted factors do not change the product. Calling the function with no arguments or more than 4 arguments should raise TypeError.

Function: A6Q5(a, b=1, c=1, d=1)

Examples:

A6Q5(1, 2, 3)  # Returns 6
A6Q5(3, 3)  # Returns 9

Update the following function to include a docstring with both Parameters and Returns sections. Describe the parameters and the returned value. Preserve the function’s signature and calculation.

def A6Q6(a, b, c=0):
    hypt = (a**2 + b**2 + c**2)**0.5
    return hypt