.. _ch05-python-exception-debugging: =============================================================== Exception handling and debugging in Python =============================================================== You have probably already seen Python's exception handling in action from trying to execute invalid syntax, or some other illegal operation. For instance: .. code-block:: python >> 1/0 Traceback (most recent call last): File "", line 1, in ZeroDivisionError: division by zero The first part in the last line says ``ZeroDivisionError``. This is one of the Python's built-in exceptions for arithmetic errors. Exceptions are raised by different kinds of errors when executing invalid Python code. You may also throw your own errors, and even define custom error types specific to your code base. Please see a more complete list of Python's built-in exceptions at https://docs.python.org/3/library/exceptions.html. Below, we shall take a look at some examples which will teach us to how to use exception handling in Python programs. In addition, we are going to study a few debugging strategies to help deal with exceptions that arise when you aren't expecting them. .. _ch05-python-exceptions: -------------------------------------------- Exceptions -------------------------------------------- A few of the more common built in exceptions are listed here: * ``TypeError``: unsupported operation .. code-block:: python >>> 1+'apple' Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' * ``KeyError``: invalid use of key .. code-block:: python >>> eng2kor = {'three': 'set', 'two': 'dool', 'one': 'hana'} >>> eng2kor[0] Traceback (most recent call last): File "", line 1, in KeyError: 0 * ``IndexError``: invalid use of index .. code-block:: python >>> a = [1, 2, 3] >>> a[4] Traceback (most recent call last): File "", line 1, in IndexError: list index out of range * ``AttributeError``: attribute reference or assignment failure .. code-block:: python >>> eng2kor.append('foo') Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'append' * ``SyntaxError``: the most basic type of error when the Python parser is unable to understand a line of code .. code-block:: python >>> print "I love scientific computing" SyntaxError: Missing parentheses in call to 'print'. Did you mean print("I love scientific computing")? The above syntax error due to missing parentheses happens mostly when migrating Python2 codes to Python3. -------------------------------------------- Catching exceptions -------------------------------------------- ^^^^^^^^^^^^^^^^^^^^^^^ try/except ^^^^^^^^^^^^^^^^^^^^^^^ The ``try`` statement works as follows: #. Python attempts to execute the statement(s) between ``try`` and ``except`` #. If no exception occurs, the ``try`` block terminates and the ``except`` block is ignored #. If an exception does occur during the execution of the ``try`` block, the rest of the block is skipped, and the exception is passed to the ``except`` block. #. If its type matches the exception named after the ``except`` keyword, the body of the ``except`` clause is executed, and then execution continues after the ``try-except`` statement. #. If the type does not match, then the exception will get forwarded to additional ``except`` blocks if they are present. Otherwise, it gets passed up a layer (potentially all the way to the interpreter). If the interpreter catches the exception then you'll see one of the above messages: :ref:`ch05-python-exceptions`. Now let's take a look at few examples. In the first example, you will see the error message if you enter an input that is not a number: .. literalinclude:: ./codes/try_except1.py :language: python :linenos: :download:`Download this code <./codes/try_except1.py>` Exception handlers don’t just handle exceptions when they occur immediately within the try clause, but also when they occur inside functions that are called (even indirectly) in the try clause. For example: .. literalinclude:: ./codes/try_except2.py :language: python :linenos: :download:`Download this code <./codes/try_except2.py>` ^^^^^^^^^^^^^^^^^^^^^^^ try/finally ^^^^^^^^^^^^^^^^^^^^^^^ A ``finally`` block can optionally be used with a ``try/except`` pair, which is intended to define clean-up actions that must be executed under all circumstances. For instance: .. literalinclude:: ./codes/try_finally.py :language: python :linenos: :download:`Download this code <./codes/try_finally.py>` Such a clean-up step can be useful for resource management, e.g., closing a file. ^^^^^^^^^^^^^^^^^^^^^^^ Raising exceptions ^^^^^^^^^^^^^^^^^^^^^^^ Let's now take a look at how to *raise* an exception in your code. In the following example, we run an iteration using our old example of Newton's root finding method. One idea is to raise an exception of ``ArithmeticError`` (see line 33 in the example below) when the method fails to converge. .. literalinclude:: ./codes/try_except3.py :language: python :linenos: :download:`Download this code <./codes/try_except3.py>` Try running this code using the initial guesses of :math:`x=-1/2,0,1,1.1634`. .. _ch05-python-debugging: -------------------------------------------- Python debugging -------------------------------------------- ^^^^^^^^^^^^^^^^^^ Print statements ^^^^^^^^^^^^^^^^^^ Print statements can be added almost anywhere in a Python code to print things out to the terminal window as it goes along. You might want to put some special symbols in debugging statements to flag them as such, which makes it easier to see what output is your debug output and also makes it easier to find them again later to remove from the code, e.g. you might use "+++" or "DEBUG". As an example, suppose you are trying to better understand Python scoping and namespaces, as well as the difference between local and global variables. Then this code might be useful: .. literalinclude:: ./codes/debugging1.py :language: python :linenos: :download:`Download this code <./codes/debugging1.py>` .. note:: In the above example, we see two tokens, ``%s`` for strings and ``%d`` for numbers, as placeholders of what comes after the ``%`` sign, e.g., ``% (x, y, z)``. See more examples `here `_. We'll see more about this when we cover C. Here the print function in the definition of ``f(x)`` is being used for debugging purposes. Executing this code gives .. code-block:: console $ python3 debugging1.py +++ before calling f: x = 3.0, y = -22.0 +++ in function f: x = 13.0, y = -22.0, z = 3.0 +++ after calling f: x = 3.0, y = 13.0 If you are printing large amounts you might want to write to a file rather than to the terminal. .. _ch05-python-pdb: ^^^^^^^^^^^^^^^^^^ The pdb debugger ^^^^^^^^^^^^^^^^^^ Inserting print statements may work best in some situations (particularly for debugging logic errors), but it is often better to use a *debugger*. The Python debugger ``pdb`` is very easy to use, often even easier than inserting print statements and is well worth learning. See the `pdb documentation `_ for more information. The syntax is very similar to ``gdb`` which we discussed before. You can insert *breakpoints* in your code (see line 17 in the below) where control should pass back to the user, at which point you can query the value of any variable, or step through the program line by line from this point on: .. literalinclude:: ./codes/debugging2.py :language: python :linenos: :download:`Download this code <./codes/debugging2.py>` Of course one could set multiple breakpoints with other ``pdb.set_trace()`` commands. For the above example we might do this as below. Upon running the above example, we get the prompt for the ``pdb`` shell when we hit the breakpoint .. code-block:: console $ python3 debugging2.py > /home/ian/UCSC/AM129Lectures/Python/debugging2.py(18)f() -> return x (Pdb) p x 13.0 (Pdb) p y -22.0 (Pdb) p z 3.0 Note that ``p`` is short for ``print`` which is the same as ``gdb`` command. You could also type ``print(x)`` but this would then execute the Python print command instead of the debugger command (though in this case it would print the same thing). There are many other ``pdb`` commands, such as ``next`` to execute the next line, ``continue`` to continue executing until the next breakpoint, etc. .. code-block:: console $ python3 debugging2.py > /home/ian/UCSC/AM129Lectures/Python/debugging2.py(18)f() -> return x (Pdb) p z 3.0 (Pdb) continue x = 3.0 y = 13.0 (See `the pdb documentation `_ for more details.) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Debugging after an exception occurs ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Often code has bugs that cause an exception to be raised, causing the program to halt execution. Consider the following example: .. literalinclude:: ./codes/pdb_example.py :language: python :linenos: :download:`Download this code <./codes/pdb_example.py>` In `IPython` you can run this by calling .. code-block:: python >>> %run pdb_example.py --------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) ~/UCSC/AM129/LectureF21/Python/Fundamentals/pdb_example.py in 12 for i in range(5,0,-1): 13 # pdb.set_trace() ---> 14 soln = division_by_zero(float(i)) ~/UCSC/AM129/LectureF21/Python/Fundamentals/pdb_example.py in division_by_zero(x) 6 7 def division_by_zero(x): ----> 8 x/=x-1 # equiv. to x=x/x-1 9 return x 10 ZeroDivisionError: float division by zero At some point there must be a division by zero. To figure out when this happens, we could insert a ``pdb.set_trace()`` command in the loop and step through it until the error occurs and then look at ``i``. Alternatively, and more easily, we can use a *post-mortem* analysis after it dies via ``pdb.pm()`` .. code-block:: python >>> import pdb >>> pdb.pm() > (8)division_by_zero() (Pdb) p i 1 (Pdb) p soln 2.0 (Pdb) p x 1.0 This starts up ``pdb`` at exactly the point where the exception is about to occur. We see that the divide by zero happens when ``i = 1``. Compare this to our usage of ``bt`` in ``gdb``.