Programming basics - Software Engineering For Scientific Computing

This is a quick overview of Programming and Python. Hopefully this is mostly review, or possibly an intro to Python syntax if you are coming from a different language.

This also includes an intro to Jupyter, which we will be sometimes using in the course.

Objectives:¶

Learn the 5 components of the Python Syntax
Understand Python’s object system
Learn to interact with the IPython prompt in the Jupyter notebook

Jupyter, IPython, Python, oh my!¶

Just to keep the computational model straight, we are using:

Python 3.12 (3.11+ recommended), a interpreted programming language
A REPL, which Reads a line, Evaluates it, then Prints it (in a Loop)
- print not needed to see the last un-captured value in a cell
IPython, which is an enhancement to make Python more Interactive
- You can’t use these enhancements in a library, but they generally are only useful interactively anyway
Jupyter notebooks, which is a web based display of a Kernel, such as IPython (or many other language kernels).
Jupyter lab, which is a place were multiple notebooks and other screens are shown together.

Jupyter lab  -> Jupyter notebook ->      IPython       ->  Python
-----------     ----------------   ------------------    ---------
This is the     This is the file   Checks for special    Real work
visible part    format             syntax, then sends
                                   it on

Basics of programming¶

Computing can be broken down to two things:

Variables: These hold data
Functions: These are procedures (usually operating on variables)

# This is a variable
variable = 3


# This is a function
def function(x):
    return pow(x, 2)

function(variable)

9

Don’t worry about the details of the Python syntax quite yet, just focus on the two concepts: data and procedures. And, yes, x**2 is nicer than pow(x,2).

Try it yourself¶

The great thing about programming in an interpreted language like Python is that you can try everything yourself, in real time. Play around with the above variables and answer these questions:

Can you name a variable def? Why or why not?
Can you name a variable pow? Why or why not?
Why might you not want to make a variable named pow?
How can you get rid of your variable named pow? (Hint: Try del or restarting your kernel (top menu)).

Python’s syntax¶

Operators: special symbols or words that have a specific meaning. You can usually control what they do in classes. Some general parts of the syntax, like whitespace, commas, colons, and brackets, are not called operators.
Keywords: special words that have language level meaning and cannot be used elsewhere or changed.
Soft Keywords: special words that sometimes have language level meaning (new in 3.10, currently only match and case)
Builtins: functions and objects that are pre-defined, but not special.
Numbers: several different number types available
Strings: add text to the program or help
Comments: start with a # and are ignored by Python

Note: Builtins can’t be undefined due to the way the Python lookup system works.

# Does not take anything or return anything
def funct():
    "This is the function documentation"
    print("Called f", 1 * 1, "time")

Operators, Keywords, Builtins, Numbers, Strings, Comments

Answer this:¶

What color is each of these in your notebook?

Objects and Types/Classes¶

We can go one step further from variables and functions: objects and classes.

Object: A related collection of data and functions for that data
- Members: Variables that hold the data for the object
- Methods: Functions that operate on the object data
Class: The “type” of an object; objects that have the same methods/members have the same class

Why are objects so important in Python? Because unlike many other languages, everything in Python is an object! This is very powerful.

Note: Syntax such as keywords are not objects. Builtins are objects.

number = 1.5
string = "hello world"
complex_number = 2 + 3j

Try running some of the methods and look at the members of the objects defined above.

IPython + Jupyter keyboard hints:

Use Tab to see completions, either when typing a word or after a .
Use Shift-Tab to see the help for a function after typing it
Use Shift-Enter to execute a cell

We can use type to see the class of an object:

type(number)

float

Names¶

There are certain conventions for the names of objects and classes:

Case	Convention	Example
Objects and methods	start with lower case letter	`like_this`
Classes	Start with upper case letter	`LikeThis`
Globals	all uppercase	`LIKE_THIS`
Hidden	start with underscore	`_like_this`
In one class only	start with two underscores (rarely used)	`__like_this`
Special python names	start and end with two underscores (called “dunder” for double underscore)	`__like_this__`

Older parts of Python do not always follow conventions; notably most Python built-in types are all lowercase. And singletons True, False, None, and Ellipsis are objects but are capitalized.

Built in Python Types¶

The following are a mostly comprehensive list of built-in Python types

Basic types¶

py_int = 101
py_float = 3.7
py_complex = 1j
py_str = "Unicode string: Python 3 is 😀"
py_bytes = b"Binary string"
py_bool = True
py_none = None

These types are all also constructible by calling the name of the type, such as int(1). A few interesting notes:

Numbers can be constructed from strings too, like int("1")
Strings can start with special characters, like b, r, or f. We’ll see this again someday.
Strings use " or '. Three quotes can be used to include new lines.
None is the only none type. How lonely.

Try it yourself:¶

How large is the largest int value allowed?

Compound types¶

These are composed, but are also very important

py_tuple = (1, 2, 3)
py_list = [1, 2, 3]

py_dict = {"Python 3": "😍", "Python 2": "😖"}

py_set = {"no", "duplicates", "in", "here"}

Accessing values is done with brackets:

print(py_tuple[0])
print(py_dict["Python 3"])

1
😍

Indexing starts from 0, as it should
A tuple is just like a list, but can’t be changed: immutable

Mutable	Immutable
Can be changed	Can only be replaced
Can’t be a key or in a set	Can be a key or in a set
`list`, `dict`, `set`	`tuple`, `str`, `int`, etc.

A set does not have an order (so you can’t use [] to get items from it)
Dictionaries were not ordered before Python 3.6 (even now you still only have direct access to keys)
The keys of a dictionary are conceptually a set (with an order).

There are other types in the standard library, but we’ll restrict ourselves to talking about these for now.

If you could put a mutable item in as a key or set item, you could then change it later and have two identical key items or set items!

Try it out: Mutation vs. replacement¶

Can you guess what the final value of a is? What about b?

a = [1, 2, 3, 4]
b = a
b[2] = "😡"
b = [3, 2, 1, 3]
b[2] = None

print(a, b)

[1, 2, '😡', 4] [3, 2, None, 3]

Key points to understanding this example:

Normal assignment replaces the variable on the left
Python never copies, but with immutable values and the above statement, it might seem that way

There are a limited number of expressions allowed on the left (but they can be chained):

An object (replaces): a =
An index into an object (mutate) a[x] =
An object attribute (mutate): a.x =

Places were = is not allowed can use := from Python 3.8+.

Slicing¶

One of the most important things in computation is indexing, and Python is really good at it. You’ve already seen integers being used to index lists (strings work the same way too), but let’s peek at another way to index lists and tuples: slicing.

a = "abcdefg"
print(a)
print(a[0:7:1])  # start : stop (not included) : step
print(a[0:7])  # step not needed
print(a[1:])  # you can leave out values
print(a[:-1])  # like indexing, - means from end
print(a[::2])

abcdefg
abcdefg
abcdefg
bcdefg
abcdef
aceg

Operators¶

Python’s operators are pretty standard. Here are the ones that tend to be a little odd:

x**y: Raise x to a power y
x == y: See if x is equal to y
x is y: See if x is the same object (in the same spot in memory) as y, can’t be overridden
~x: Invert x (might be different than unary -)
+x: Python does have a unary + operator, too. Not sure why you’d want it.
x += y: Add y to x, in place only if mutable

x / y: True mathematical division (Python 2 used to do integer division sometimes)
x // y: Truncating division 3 // 2 == 1
x @ y: Matrix product of x and y (Python 3.5+, only in libraries)
x and y: Boolean operator, short-circuits and can’t be overridden (or and not too)
x & y: Bitwise and - this one can be overridden, but has odd precedence
x | y: Bitwise or

Control¶

We’ve already seen functions. What other control statements are out there for us to use?

x = True
if x:
    print("x was true")
else:
    print("x was not true")

x was true

The for each loop works on any iterable:

a = "abcdefg"
for item in a:
    print(item, sep=", ")

a
b
c
d
e
f
g

You can also use for item in range(N) to count over a range of values, or while CONDITION:. Python 3.10 adds a new “pattern matching” control structure match that takes case conditions.

You can use for (and if) inside [], (), or {} to build the data structures mentioned above inplace - this is called a “comprehension”.

bad_way_to_make_a_set_of_tens = {x for x in range(100) if x % 10 == 0}
bad_way_to_make_a_set_of_tens

{0, 10, 20, 30, 40, 50, 60, 70, 80, 90}

Decorators¶

This is likely the simplest syntactic sugar you’ll see in Python, but maybe one with some of the furthest reaching consequences. Let’s say you have a bit of code that looks like this:

def f(): ...
f = g(f)

So g is a function that takes a function and (hopefully) returns a function, probably a very similar one since you are giving it the same name as the old f. In Python 2.5, we gained the ability to write this instead:

@g
def f(): ...

That’s it. That’s all it does. The thing after the @ “decorates” (or transforms) the function you are defining and the output is saved with the same name f.

The thing you return doesn’t have to be a function, actually:

def bad_decorator(func):
    print(f"You don't need {func.__name__}!")
    return 2


@bad_decorator
def f(x):
    return x**2

You don't need f!

What is f now?

2

Don’t do this! (Unless you don’t want any friends.)

It’s best to think of this as a “modifier” (or “decorator”) for functions (and classes). Don’t worry to much about how it works, and especially how to write them. My favorite is a “decorator factory”, which is simply a function that returns a function that takes a function that returns a function! But in practice, it just looks like a decorator that takes arguments.

Let’s look at an example: functools.lru_cache:

import functools
import time


@functools.lru_cache(maxsize=128)
def slow(x: int) -> int:
    time.sleep(2)
    return x

Now try using it:

slow(1)

1

slow(1)

1

With¶

There are other useful things in Python, but the with statement is too good to wait. With lets you take code like this:

f = open(filename)
txt = f.read()
f.close() # Don't forget me!

and write instead:

with open(filename) as f:
    txt = f.read()
# File automatically closed here!

with simply runs code at the start and at the end of a block. It also promises the code at the end runs, even if there’s an error! (More about errors later)

Other languages, like C++, do not have a with statement, but they have similar constructs. C++ uses RAII, or Resource Acquisition Is Initialization instead. Python can’t use RAII because you are never guaranteed when or even if an object will be destroyed. And Python doesn’t have a way to add a scope block wherever you want like C++. Ruby and JavaScript use executable blocks, but Python doesn’t have multiline lambdas.

But design wise, much of the same ideology applies.

There’s a handy trick for writing new context managers: a decorator in contextlib! (contextlib has some really great stuff in it)

import time
import contextlib


@contextlib.contextmanager
def timer():
    old_time = time.monotonic()
    try:
        yield
    finally:
        new_time = time.monotonic()
        print(f"Time taken: {new_time - old_time} seconds")

with timer():
    time.sleep(1)

Time taken: 1.0003499689999984 seconds

As a special bonus for using this shortcut, contextlib.contextmanager context managers are also usable as decorators!

@timer()
def slow_func():
    time.sleep(1)

slow_func()

Time taken: 1.000109529999989 seconds

Design in programming¶

You should:

Make small, understandable pieces that can be run by themselves (Easier to debug and test)
Write similar code once (fewer places for bugs)
Reuse existing functions or libraries when possible (let someone else design and debug what they are good at)

Objects are very good for the second two points (and so-so for the first).

Also notice the running theme above?