Let’s make it immediately clear what this chapter is and what isn’t.
This is not a fully-fledged, self consistent Python course—there are so many good ones on the web that coming up with a new one would be just like solving a problem that did not exist in the first place. If anything, this chapter is a semi-random collection of things that you might stumble across as you work your way through becoming proficient in developing scientific software.
For each and single one of those thing, first of all you have to acknowledge its existence. By the end of the semester you will not qualify as a software engineer by any reasonable metrics. Hopefully, though, as more and more of the things we touch upon will come biting at you, your reaction will be: “hey, wait a moment—I’ve heard this one before”. And that’s where the real study will begin, with patience and practice.
Over the course of this notes we shall see tons of snippets of (mostly Python) code. Each one comes with a small copy button on the top right that you are encourage to use. Copy and past them in your favorite text editor, change them, play with them and develop the familiarity that you need in order to be able to use the different techniques that they illustrate whenever you need them.
And, since we are at it, here is the first one.
import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Take a second to go through all the sentence in this easter egg originally devised by early Python adopter and developer Tim Peters. It is very possible that any of these make much sense to you, now; by the end of the course you should be able to get something out of each one, and laugh out loud to at least a few of them.
2.1 Coding conventions
Oh. My. God. This has been going on for several pages, already, and we have not seen anything useful, yet. Shouldn’t we be concerned with variables, functions and all of that?
You were warned: this is not a Python course for beginners in the usual sense. Of course we will get to the good stuff in due time, but remember our goal, here, is to develop the necessary habits to write decent code—code that you won’t necessarily have to throw away tomorrow—rather than just code that does something. And, since we are at it, I have to tell you: you will have to survive in a hostile environment, where your supervisor will come to your office every other day asking for something important (the most important thing on the face of the Earth, in fact) to be completed by the end of the day. When that happens make sure you have clear in your heart that every shortcut you take in order to speed up the process will come hunt you down sooner or later. In the long term, it will be a net negative. (Whether you can convince your supervisor of that is a totally different story.)
Although never is often better than right now. Rings a bell?
Now back to the coding conventions. Coding conventions are guidelines about how to write code and, it goes without saying, they are different for different languages and, in many cases, different across different teams. Look at this small snippet of code.
the spacing around the = operator is inconsistent.
And yet the things runs correctly. This is the bad things about coding conventions: you are encouraged to stick to them, but your code will happily run if you don’t. Then why should one care, you are asking? Well, this stems from one the insights popularized by Python creator and former BDFL (benevolent dictator for life) Guido van Rossum—namely: code is read much more often than it is written. In other words: it makes sense to spend time crafting the code while you are writing, because that will benefit all the persons that will be reading the code in the years to come.
Readability counts. (And there it is, the Zen of Python again.)
The one-line summary, here, is: think about it but don’t be obsessed by it The bible of coding conventions for Python code is the Python enhancement proposal PEP 8. Print a copy and keep it under your pillow.
By the way: there are automatic tools out there, called linters, to help you but we shall get back to that in due time.
2.2 Python basic types
If you have ever programmed in a compiled language such as C or rust, Python is different in many respects. For one thing, you don’t have to declare variables, as the type is inferred at the time of assignment based on the value being assigned. Python offers a number of native type, among which the most common are
integers;
floating-point numbers;
lists and tuples;
dictionaries.
i =3x =3.0s ="Hi there!"l = [1, 2, "a string"]t = (1, 2, "a string")d = {"key1": 1, "key2": 2}for item in (i, x, s, l, t, d):print(item, type(item))
3 <class 'int'>
3.0 <class 'float'>
Hi there! <class 'str'>
[1, 2, 'a string'] <class 'list'>
(1, 2, 'a string') <class 'tuple'>
{'key1': 1, 'key2': 2} <class 'dict'>
Also, unlike C, variables can change type in the lifetime of a program. Despite that, every variable has a unique and well defined type at each point in the execution.
a =11print(a, type(a))a ="Hello!"print(a, type(a))
11 <class 'int'>
Hello! <class 'str'>
2.2.1 Numeric types
The most interesting fact about numeric types in Python is perhaps the fact that, unlike most languages, integers come with infinite precision. In practice this means that their footprint in memory is not pre-determined in size (e.g., 8, 16, 32 or 64 bits) but can grow automatically as the RAM memory permits, depending on the actual value that the variable is storing—no need to worry about overflows.
Small piece of advice: do not get used to that, as this is a luxury that you can hardly afford in the vast majority of other languages. Actually, when we shall start operating with numpy arrays, we shall see that we are thrown right back into the fixed-size arithmetic hell. We shall discuss the pros and cons when we get there.
The floating-point arithmetic is such an important subject for us, that we shall defer its discussion to a separate chapter.
2.2.2 Strings and string formatting
Technically speaking, strings are immutable sequences of unicode code points. (If any of this is unclear to you, do your research and get up to speed, as strings are fairly common in many different settings.)
One of the things that, sooner or later you will have to master is how you actually format strings. From this standpoint, Python does not have a particularly straightforward story to tell, as there are at least four different ways to achieve the same things.
name ="Luca"age =42# The ugly way.print("My name is "+ name +" I am "+str(age) +" year(s) old.")# The old way (% operator)print("My name is %s I am %d year(s) old."% (name, age))# The new way (.format)print("My name is {} I am {} year(s) old.".format(name, age))# The newer way---new in Python 3.6. This is awesome!print(f"My name is {name} I am {age} year(s) old.")
My name is Luca I am 42 year(s) old.
My name is Luca I am 42 year(s) old.
My name is Luca I am 42 year(s) old.
My name is Luca I am 42 year(s) old.
There should be one– and preferably only one –obvious way to do it. This is very un-Pythonic. Also, this suddenly reminds me that I have been teaching this class for seven years and I am now 49; damned.
What do I have to commit to memory in all this? Simple: use f-strings whenever possible, which is in Python 3.6 and later, which should be pretty much in any Python installation these days. And, of course, there is a lot of legacy Python code out there, so chances that you might stumble across any of the old variance, sooner or later, are fairly high.
2.2.3 Containers
Containers, as the name suggests, is a data structure meant to hold other objects. As usual Python is fairly flexible when it comes to containers—they typically can include any kind of objects (with a few limitations we shall discuss in due time) and they don’t even have to the homogeneous, as we have seen already.
(This, again, is at odds with numpy arrays. As we shall see, in loose terms, we pay this extra flexibility in speed.)
When it comes to the container zoology, the most important top-level difference is between sequences and mappings. Lists and tuples are sequences: their elements are identified by position, starting from 0. Dictionaries are mappings: their values are identified by keys. They all preserve the insertion order, and you access the elements with the [] syntax.
(Stop for a second and think hard what the relevant use cases might be for each one of them. What would you use for a series of measurements from a data acquisition system? And for a phone book?)
If you are wondering what the difference is between a list and a tuple… easy: the former is mutable, while the latter is immutable.
l = [1, 2, 3]print(l)l[0] =12print(l)t = (1, 2, 3)print(t)# Ops... this causes a traceback!t[0] =12
[1, 2, 3]
[12, 2, 3]
(1, 2, 3)
---------------------------------------------------------------------------TypeError Traceback (most recent call last)
CellIn[7], line 11 8print(t)
10# Ops... this causes a traceback!---> 11t[0] = 12TypeError: 'tuple' object does not support item assignment
2.2.4 More fun with types
There’s a couple more Python basic type we should mention. The first one is the boolean type that pops out when we compare objects
print(3==3, type(3==3))print(3==4, type(3==4))
True <class 'bool'>
False <class 'bool'>
The other is the None singleton. It does absolutely nothing, it does not compare equal to anything other than itself, and does not support any specific operation.
Dumb as it seems, it is ubiquitous in Python code, mainly used a sentinel value for function arguments that have no other obvious default, or as a return value for functions that do not actually return anything.
2.3 Control flow
Although we have to go super-fast on this one, there’s a few interesting bits of Python control flow that are worth mentioning.
At the tree-level, Python offers the same basic control mechanisms that most of the other languages do offer: the for loop, the while loop and the if statement.
i =2# Conditional expressionsif i ==2:print("Apple")elif i ==3:print("Peach")else:print("Cheese")# For loopsfor i in [1, 2, 3]:print(i)i =4# While loopswhile i !=0:print(i) i -=1
Apple
1
2
3
4
3
2
1
Quite often, you will find for loop iterating over the range().
for i inrange(3):print(i)
0
1
2
This is where you should start paying attention, because the semantic of the for loop in Python is not necessarily what you would expect if you come from a different language, e.g.,
for i inrange(10):print(i)if i ==5: i =8
0
1
2
3
4
5
6
7
8
9
And, since we are now proficient with the Python container, we can mix and match them with our control flow facilities in a few interesting ways:
list1 = ["a", "b", "c"]list2 = [10, 11, 12]# Horrible (and very un-Pythonic, too)!for i inrange(len(list1)):print(i, list1[i])# Nice-looking.for i, item inenumerate(list1):print(i, item)# Zipping iterablesfor item1, item2 inzip(list1, list2):print(item1, item2)# List comprehension---this works with dictionaries too.print([x**2for x in list2])
0 a
1 b
2 c
0 a
1 b
2 c
a 10
b 11
c 12
[100, 121, 144]
Boy, that was quick.
2.3.1 Two types of comparisons
We have seen lots of comparisons so far, in this chapter; we have used the == operator every single time. Now, if you read enough Python you will see instances of
a =Noneif a isNone:print("Busted!")
Busted!
Wondering why that is not spelled as
a =Noneif a ==None:print("Busted!")
Busted!
Well, it turns out == and is test two different things: the first will tell you if two objects compare equal, while the second will tell you if two objects are really the same object. What does it mean to be the same object, you ask? Well, it means to point to the same address in memory, as you can easily find out yourself using the id() built-in function.
a = [1, 2, 3]b = [1, 2, 3]print(id(a), id(b))print(a == b)print(a is b)
139824243846848 139824243845312
True
False
This also explains why is is more idiomatic to test whether an object is None: the latter is a singleton in Python, that is, there exists one and only one None object within the scope of a Python program. If something compares equal to None, then it is the onlyNone that exists.
2.4 Functions
DRY (Don’t Repeat Yourself) is better than WET (Write Every Time), so computer science go. When you find yourself doing the same thing over and over again (that is: more than once) in a program, it is always best to encapsulate that thing into a function for re-usability.
In their simplest form, Python functions are defined with the def keyword.
def square(x):return x * xprint(square(2.))
4.0
Let’s take this chance to build a little bit of common nomenclature. In this case square is the name of the function; x is the first (and only) argument; function can return one or more values via the return statement. (If no such statement is explicitly present, the function returns None.)
Generally speaking, the use of functions comes with a few major advantages:
you end up with less lines of code;
if there is a bug in your function, you have one single place to fix, not multiple;
it is easier to convey the intent of what we are doing.
The last point was probably not obvious in the first, simple example, but it becomes clear as soon as the body of the functions stops being trivial:
Function definition in Python is explained in all the gory details here, but a cursory look at some of the more important facts is worth a few minutes.
2.4.1 Default argument values
Each argument can be provided a default value, the contract being: if we do not provide a value for a given argument, the default is used.
def square(x=1.):return x * xprint(square(2.))print(square())
4.0
1.0
All the arguments we have seen so far are called positional arguments, as their meaning is determined by the position in the function signature. There is a catch to keep in mind: all the argument without default values must come before any argument with a default value—this works just fine
CellIn[21], line 1def power(x=2., exponent):
^
SyntaxError: parameter without a default follows parameter with a default
2.4.2 Variadic functions
It doesn’t take too much familiarity with Python to realize that some functions are special in that accept an arbitrary number of arguments. Take the builtin print() function, for instance:
print("a")print("a", "b")print("a", "b", "c")
a
a b
a b c
(We could go on forever, but you got the point.)
What is is that makes the magic in this case, and how can I achieve that with a user-defined function? Well, first of all these functions are called variadic functions, and implementing this behavior in Python is significantly easier than in other languages—all you really need is the magic of the star
def add(*values): total =0.for value in values: total += valuereturn totalprint(add(1.))print(add(1., 2.))print(add(1., 2., 3.))
1.0
3.0
6.0
If you paid attention you might have noticed that what we have done here is a poor man equivalent of the sum() builtin, and this is not something that would ever want to do in real life, but the point is: you prepend a * to your argument name and all of a sudden you can provide how many arguments as you want. Nice, isn’t it?
You can get a clearer picture of what happens in the body of your variadic function by a printing experiment
This is easier than anticipated: the * in the function definition collects all positional arguments into a tuple that you can use in the body of the function itself.
Interestingly enough, the magic of the * also works in the other direction—that is you can unpack an iterable in place within a function call using essentially the same syntax. This is something that most likely you have seen when fitting with scipy. Take for instance
[omissis]# Define the fitting modeldef model(x, m, q):return m * x + q# Run the fit.popt, pcov = curve_fit(model, x, y, sigma=dy)# Unpack the best-fit parameters.m_hat, q_hat = popt# Plot the data points.plt.errorbar(x, y, dy)# Plot the model.plt.plot(x, model(x, m_hat, q_hat))
This all makes sense because our fitting models takes exactly three arguments (the \(x\) data points, along with the slope and the intercept of the line) and therefore we have to unpack the two-element numpy array with the best-fit slope and intercept.
But now that we know the trick we might as well do business in a more expressive fashion:
[omissis]# Run the fit.popt, pcov = curve_fit(model, x, y, sigma=dy)# Plot the data points.plt.errorbar(x, y, dy)# Plot the model.plt.plot(x, model(x, *popt))
which works out of the box, no matter how many parameters has our fitting model. Cool, isn’t it?
2.4.3 Keyword arguments
There is one last important thing when it comes to Python functions: keyword arguments. Keyword arguments are passed to a function by explicitly naming the corresponding parameter, which is something that can be generally done
def square(x=1.):return x * xprint(square(1.))print(square())print(square(x=1.))
1.0
1.0
1.0
Now: in this particular case the things may look silly, as there is only one argument—naming it explicitly does not add much context. But when a function has 12 arguments, 8 of which have default values, and we only want to set one among these, well… all of a sudden calling it by name makes a lot of sense—think about curve_fit()!
As a function becomes more and more complex, this can be so useful that Python provides a mechanism to make an arbitrary subset of the arguments keyword only—you just add a phony* argument as a separator:
---------------------------------------------------------------------------TypeError Traceback (most recent call last)
CellIn[26], line 5 2return x**exponent
4print(power(2., exponent=2.))
----> 5print(power(2.,2.))
TypeError: power() takes 1 positional argument but 2 were given
(Of course this is a silly example, but if you do look carefully you will realize that curve_fit() is actually doing this for real.)
Finally, Python also provides a ** syntax that collects additional keyword arguments into a dictionary, pretty much in the same way that * collects positional arguments in a tuple
While this can be useful at time (e.g., when you have to dispatch arguments from one function call to another), you should use this feature with caution, as all of a sudden you can pass literally anything to your function, and there is no way for the interpreter to tell, e.g., whether you mistyped one of the names.
I know, this was a lot. And we did skip over a number of details, too. This might be a good time to take a thorough look at the actual documentation.
2.4.4 Type annotations
One last thing about functions. If you happen to read over some modern, idiomatic Python, you might stumble across something along the lines of
def square(x: float=1.) ->float:return x * xprint(square(1.))
1.0
Now, wait a moment! What the hell is this extra float stuff—were not we clear than in Python you don’t need to declare the variable type?
Well, yes; and this is still true. These extra bits are called type annotations.
The funny thing is that the interpreter does absolutely nothing with the annotations. (Well, sort of. There are some odd corner of modern Python, such as dataclasses where type annotation have a structural function, but we shall pretend this is not the case, for the time being.) What are they for, then? Type annotations serve two main purposes:
they help you reason about the code;
there are tools, such as mypy that can do clever things via a combination of static analysis and type annotations.
We shall come back on the matter later, but for the time being be aware that annotations exists—nowadays they are fairly popular.
2.5 The Python import system
We have already stumbled across the import statement, and it’s now the time to take a closer look at the thing.
When you fire up the Python interpreter, there is a limited number of things you have access to: the language syntax, a handful of keywords and a number of builtin types and functions, which we have partially explored. (And, by the way, it is somewhat amusing that we got this far with these limited facilities.) Anything else must be imported in order to be usable.
The fact that you need to opt-in for most of the stuff should not be surprising, as it is a sensible way to limit the footprint in memory of the interpreter session—presumably there is no point in loading the Python module handling dates and times if you are working on combinatorial mathematics, would you think? Loading only a small subset of the things that are most commonly used and letting the user actively decide on anything else they might need seem only reasonable.
The mechanism through which additional Python modules are loaded in memory on top of the builtins is called the import system. And, although the mechanism is exactly the same, it is somewhat customary to divide the Python module ecosystem in two distinct levels:
The main difference is: you get the standard library as part of the Python installation, with no question asked; and you have to install any third-party module by yourself, typically via pip or some other package installer. Just to be clear: the math module is part of the standard library, while numpy and scipy are third-party.
And, before we forget: the import system will let you import your own files too.
2.5.1 Import basics
The syntax of the Python import system is (unsurprisingly) rich. Let’s go through some of the possibilities and try and establish the best practices.
Let’s say you want to use the sqrt() function in the math module. Python will let you use a wildcard to minimize the hassle
from math import*print(sqrt(4.))
2.0
This is literally like saying: “get me anything that is defined within a given module”, and, to this days, you still see it being done. Avoid it, unless it’s in a small script that you plan on throwing away. Doing this will pollute your scope with names that you don’t event know are there and increase the likelihood of clashes.
By the way: you really want to what you have done?
Lots of names, eh? Do this for a dozen of modules and it will be hard for you to find a name for you own functions that is not in use.
A slightly better alternative is
from math import sqrtprint(sqrt(4.))
2.0
Now you are only importing one name from the module. Sure enough, if you happen to need another function you will need to add an import, but it’s definitely better than before. You still have the potential issue that, if you have lots of code and you are using the sqrt() function at line 823, with the corresponding import at the top of the file, you might need a lot of back and forth to figure out where each function is coming from.
Look at this instead:
import mathprint(math.sqrt(4.))
2.0
This gives you the best of both worlds—you only import one name (math) and it is immediately obvious which module the sqrt() function belongs to. I will give you that you also have to type 5 extra characters every time you use sqrt, but remember: code is written one time and read many times. Whenever you can, go for it; it’s a no brainer.
2.5.2 Import aliases
If you start using numpy and matplotlib regularly, you will see that most of the time they are imported in the form of an alias with the as keyword
import numpy ad npfrom matplotlib import pyplot as plt
This is literally all over the place. Every single example on the web is spelled like that.
You might wonder why? Well, this saves you a few keystrokes any time you actually use a name exported by the module. Going from numpy to np gives you exactly 3 less keystrokes. Is that a good idea? Well, yes and no—it depends on the context. For recognizable aliases that everybody is using there is no particular harm, but if you start using aliases a lot, chances are you will sooner or later run into something like
from math import*import logging as log# ... 1000 lines of code in the middlex = log(2.)
---------------------------------------------------------------------------TypeError Traceback (most recent call last)
CellIn[32], line 5 2importloggingaslog 4# ... 1000 lines of code in the middle----> 5 x = log(2.)TypeError: 'module' object is not callable
Good luck debugging that in a large codebase.
2.5.3 Importing your own modules
There will be a time in your career when you will start importing your own modules. When that happens you might start wondering how the hell you can signal Python where your module lives, and where the interpreter should look for it.
If your module lives in the present working directory things work out of the box, and you should be all right.
If you have properly installed your module, e.g., via pip, then again you are fine, because the files are where they should be. By the way, you can always tell where a Python module is physically located looking at the __file__ attribute
Ideally this should be the case, and if you structure your repository correctly you should be all set.
If everything else fails, all you need to know is that you can add more places for Python to look for modules by properly setting the PYTHONPATH environmental variable. In 2026 this should be hardly necessary, but keep this in mind.
2.6 The Python standard library
Python has a name for coming with batteries included. When you install Python you get the full standard library. And, just to be clear, the standard library is huge. Just scroll the documentation web page, and you will find hundreds of modules. Many of the things that you might think of are there.
Here a short list of modules that you might find interesting, sooner or later:
Lots of interesting stuff—take the habit of browsing around in your spare time.
2.6.1 Beyond the standard lib
Awesome how the standard library is, there are use cases where you will grow to prefer third-party modules.
For mathematical functions and random number generation, for instance, the math and random modules in the standard library are ok in simple settings, but for most applications you will prefer numpy.
Some specific modules of the standard library have grown old, and are now largely obsolete. While they are still maintained for the sake of backward compatibility, they are rarely used for new code and more modern and streamlined third-party packages are to be preferred. Two prominent examples are loguru for logging and pytest for unit testing, which largely replace the standard library modules logging and unittest, respectively.
Some other, more specific applications are just not covered by the standard library, in which case you have to resort to a third-party package, unless you prefer implementing the functionality from scratch.
In all these cases, the Python Package Index, or pypi, comes to the rescue. With almost a million different projects hosted, it has you covered, no matter what you need, although it is important to say that not all of them are actively developed and surely not all to the same standard.