Exceptions and the Debugger

These notes expand on exceptions and the debugger, and are meant to supplement other notes on similar topics. There isn’t a corresponding lecture, but I’m putting the release date as October 27th, when I may be traveling.

Last time, we defined a method in our LinkedList class to obtain the element at the $n^{th}$ location. It looked something like this:

    def nth_from(self, node: ListNode, n: int):
        if n == 0:
            return node.data
        return self.nth_from(node.next, n - 1)

    def nth(self, n: int):
        if n < 0 or self.length() <= n:
            raise Exception("index out of range")
        return self.nth_from(self.first, n)

We raised an exception if the index given was out of bounds for the list.

The Bad Old Days

An alternative approach would have been returning some sort of agreed-upon special “error” value, like 0 or -1. And if you were using Tim’s first programming languages (archaic versions of C and BASIC) that’s probably exactly what you’d do.

So, why didn’t we? Why did we need an Exception, and why do we want them in a language to begin with?

Discussion!

Ok, so maybe a special notion of exception is generally better for exceptional circumstances, like errors or bad inputs.

But there are a few questions remaining, like:

What makes one exception better than another?
How should we think of using exceptions?
How do exceptions even work?

Why is one exception better than another?

Haven’t we just replaced one exception with another?

Let’s see what happens if we take out the check that Rob added in class on Friday. If we ask for, say, l.nth(3) in the tests from last time:

l = LinkedList()
l.append("hello")
l.append("world")

print(l.nth(3))

we get:

Traceback (most recent call last):
  File "/Users/tim/repos/teaching/112/lectures/oct25/prep.py", line 83, in <module>
    print(l.nth(3))
  File "/Users/tim/repos/teaching/112/lectures/oct25/prep.py", line 42, in nth
    return self.nth_from(self.first, n)
  File "/Users/tim/repos/teaching/112/lectures/oct25/prep.py", line 37, in nth_from
    return self.nth_from(node.next, n - 1)
  File "/Users/tim/repos/teaching/112/lectures/oct25/prep.py", line 37, in nth_from
    return self.nth_from(node.next, n - 1)
  File "/Users/tim/repos/teaching/112/lectures/oct25/prep.py", line 37, in nth_from
    return self.nth_from(node.next, n - 1)
AttributeError: 'NoneType' object has no attribute 'next'

Understanding the default Python error

This error has 2 parts: the error message AttributeError: 'NoneType' object has no attribute 'next' and the call stack for the error, which shows how we got to the point that the error occurred.

The message tells us that we tried to access the next field of something that wasn’t a ListNode but rather the None value, and this takes place on the line:

return self.nth_from(node.next, n - 1)

The call stack shows the list of nested function calls that Python took to reach this point in the program. It looks like the error happened in the third recursive call to nth_from, and this chain of recursion started with the invocation of nth by the offending top-level print statement.

That’s what the error tells us, by itself.

But why did the error happen?

As the writers of this code, we have the knowledge to go further and infer a more specific cause. Our recursive calls reached the end of the list, but the value of n was still greater than zero. So Python tried to make another recursive call, and fell off the end of the list.

Note that what we just inferred was different from what the raw error told us. We took our knowledge of the program and used it to interpret the error. Now we have something less low-level and more behavioral. Compare the two:

“This series of nested function calls happened, and then the program tried to get the next field of None, which didn’t exist.”
“The program had already reached the end of the list, but the user’s input wanted us to go further.”

Is everyone who gets the error going to possess the intimate knowledge of our program that we have? If not, there’s a good chance that the default Python error might be unhelpful. The user might say: “Yeah, I know that None doesn’t have a next field, but so what?”

And if your user isn’t a programmer, but just someone trying to use a script that calls a library that uses your LinkedList class, the error is likely even more confusing! What’s None? What’s next? They weren’t trying to do anything but order dinner online…

The takeaway here is that your code should always consider what its intended user may be trying to do. This isn’t as easy as it sounds: if you’re writing a data structure, it might be used for just about any purpose. But you can do better than the default Python error: here, the intent of the user (or library using your class) was definitely to index into a LinkedList and get the nth element.

So shouldn’t the error be in terms of that goal?

Hence the custom exception. (But we can and will do better, still.)

`Try`ing to work with Exceptions

Python gives us a way to catch an exception that something we’ve called produces. We call these a “try/except”, or, sometimes, a “try/catch”. Concretely, let’s try wrapping the call to nth_from_ in a construct that will stop an exception if one happens:

try:
    return self.nth_from(self.first, n)
except (AttributeError):
    pass # don't do anything. Will return None

If code within the try: block raises an exception of the appropriate type (here, AttributeError), the the except block runs. Just pass here means that the function will return None. That’s not great, but we could also have returned -1 or 3 or 17000. We could also raise our own exception!

The key is that once an exception has been caught by an except block, it’s assumed to have been handled. It won’t propagate any further up through your program unless the code raises it again. The block is there specifically to define fall-back behavior in unusual circumstances, and exceptions embody those circumstances.

How would I re-raise the same exception?

You’re allowed to give a name to the exception in the except block. We won’t be talking much about this sort of more advanced exception-engineering in this class, but if you move on to 0200 you’ll see more.

How do exceptions actually work?

It’s easiest to see this live.

Let’s try that access from a few minutes ago in the debugger:

print(l.nth(3))

This time, instead of clicking the Run button, we’ll go to the Run menu and click “Start debugging”:

The debugger will stop at the point that the exception gets thrown, and add a lot of new stuff to the screen:

Let’s pause and look. We see:

the code window, highlighting the offending call of nth_from;
a giant red error popup, telling us that an AttributeError has happened;
something that looks like a TV remote control has appeared on the top right of the window.

That remote control contains a bunch of useful buttons for stepping through your code slowly, line by line.

Let’s try the process over from the beginning. We’ll put a breakpoint on the top-level print statement. In VSCode, we can do that by moving the mouse to the left of the line numbers until we see a red circle:

Clicking the red circle will add a breakpoint: a point in your program where, if you’re debugging rather than just running, the debugger will pause execution:

Let’s start the debugger again. Now, we see the program paused:

Now click the “step into” button:

The debugger calls nth but then stops again: the “step into” button lets a single method or function call happen:

On the left, you’ll see the current values of variables in the program dictionary:

Since we want to follow Python into the call of nth_from, we’ll click on step into again. Then we see:

Another click and we’ll learn that, since n isn’t zero, execution steps forward, over the return statement:

If we keep following this process, we’ll see the value of n reduce by 1 every time we recur into nth_from. This is where the call stack arises from: a series of function or method calls, nested within one another.

But then, disaster strikes:

and we see the power of exceptions. The exception immediately takes over execution, and flows back up the previous method calls, looking for a matching except block. If no such block is found, the program terminates with an error.

Takeaway

Up until now, we’ve treated an error as if it were the end. The program would have to crash! And indeed, that’s what happens if the exception propagates all the way to the top-level of hte program: the program halts and prints an error. But try and except blocks let us write code that handles, and even possibly recovers, from those errors.

Could we make a better exception?

The exception we had before was vague in two ways:

It was an Exception, which is the most general way of representing “Something bad happened!”; and
it only contained an error a string (“Bad index”).

Fix 1

One option to improve on this would be to throw the defined Python IndexError exception type, but with an improved error message. An IndexError describes the nature of the problem far more faithfully than an AttributeError: you’re saying that the index the caller provided was wrong.

Fix 2

Another option is to define our own exception. We use inheritance for this.

class LinkedListBadIndexError(Exception):
    pass

We could put more here if we wanted. It’s just a class, and so we could add an __init__ method and anything else we wanted. We could carry the bad index value as a field, etc.

Personally, I prefer the first option (an IndexError) to a custom exception class for this particular application. It’s easy (just raise an IndexError) and it communicates the right information. But often there isn’t a good exception for the idea you want to convey; then you’d make your own.

Testing with Exceptions

What happens if we try to test with the bad index? What can we even assert? We use Pytest’s helper:

from linkedlist import *
import pytest
with pytest.raises(LinkedListBadIndexError):
  lst.nth(1)

We’re going to ask you to include tests for both normal cases and error cases from now on!