CC-BY-SA 4.0, Rolf Kranz

There is a logical explanation to bugs.

Debugging as an experimental science

Experimental science = theory + experiments

Don't try to fix your bug

Understand what happens

Theory

• Assumptions about how your code is working
• Hypotheses about what happens during the bug

Theory

Can be wrong

Theory

The grid to interpret your experiments

Experiment

What is actually happening when you run your code

Experiment

Used to test your hypotheses

How?

Scientists use a research log

Debugging log

• Write down experiment settings and results
• Write down your assumptions and hypotheses explicitly
• Write down ideas you can't follow right now

Debugging log is great to write post-mortems

Let's see how that applies to a toy example

I need to iterate on cumulative lists:

[0], [0, 1], [0, 1, 2], ...

I need to add a special value at the beginning of each list:

-100

                    def prepend(l: list, beginning: list = [-100]):
                        beginning.extend(l)
                        return beginning


                    for i in range(5):
                        zero_to_i = list(range(i))
                        l = prepend(zero_to_i)

                    print(len(l))
                    # Expected: 5; i = 4, zero_to_i = [0, 1, 2, 3], l = [-100, 0, 1, 2, 3]
                    # Actual: 11
                

Assumption 1

prepend is adding -100 in front of the list.

Experiment 1

                    def prepend(l: list, beginning: list = [-100]):
                        beginning.extend(l)
                        return beginning

                    print(prepend([0, 1, 2]))
                    # [-100, 0, 1, 2]
                

Experiment 2

Let's print the values of zero_to_i

Experiment 2

                    def prepend(l: list, beginning: list = [-100]):
                        beginning.extend(l)
                        return beginning


                    for i in range(5):
                        zero_to_i = list(range(i))
                        print(zero_to_i)
                        l = prepend(zero_to_i)

                    # []
                    # [0]
                    # [0, 1]
                    # [0, 1, 2]
                    # [0, 1, 2, 3]
                

Experiment 3

Let's print the values of l

Experiment 3

                    def prepend(l: list, beginning: list = [-100]):
                        beginning.extend(l)
                        return beginning


                    for i in range(5):
                        zero_to_i = list(range(i))
                        l = prepend(zero_to_i)
                        print(l)

                    # [-100]
                    # [-100, 0]
                    # [-100, 0, 0, 1]
                    # [-100, 0, 0, 1, 0, 1, 2]
                    # [-100, 0, 0, 1, 0, 1, 2, 0, 1, 2, 3]
                

Assumption 4

prepend is always behaving in the same way.

Experiment 4

                    def prepend(l: list, beginning: list = [-100]):
                        beginning.extend(l)
                        return beginning

                    print(prepend([0, 1, 2]))
                    # [-100, 0, 1, 2]

                    print(prepend([0, 1, 2]))
                    # [-100, 0, 1, 2, 0, 1, 2]
                

Experiment 5

Investigate what is happening with multiple successive calls to prepend.

Experiment 5

                    def prepend(l: list, beginning: list = [-100]):
                        print("l: ", l)
                        print("beginning: ", beginning)
                        beginning.extend(l)
                        return beginning

                    prepend([0, 1, 2])
                    # l: [0, 1, 2]
                    # beginning: [-100]

                    prepend([0, 1, 2])
                    # l: [0, 1, 2]
                    # beginning: [-100, 0, 1, 2]
                

What is happening

• Default arguments are instantiated once.
• If a default argument is changed, the new value will be used as default for subsequent calls.
• beginning.extend(l)
• DO NOT use mutable default arguments.

Fixed code

                    def prepend(l: list, beginning: list | None = None):
                        if beginning is None:
                            beginning = [-100]
                        beginning.extend(l)
                        return beginning


                    for i in range(5):
                        zero_to_i = list(range(i))
                        l = prepend(zero_to_i)

                    print(len(l))
                    # Expected: 5
                    # Actual: 5
                

Hints and tips

Qualifying the bug - in theory

• What is the "bug"?
• Is it a bug?

Qualifying the bug - in practice

• Is it reproducible?
• Was it working before?

Building your theory 1/5

Rely on Occam's razor

Building your theory 2/5

Use documentation and architecture diagrams

But don't trust them

Building your theory 3/5

Follow the flow

Building your theory 4/5

Divide and conquer

Building your theory 5/5

Remember: at least one of your assumptions is wrong

Designing efficient experiments 1/7

Invest in tooling

Observability for debugging

• Good logs can already validate or invalidate hypotheses
• Metrics can help in hard-to-reproduce scenarii

Designing efficient experiments 2/7

Trust nothing / print everything

Designing efficient experiments 3/7

Divide and conquer

Designing efficient experiments 4/7

Inspect values at interfaces

Designing efficient experiments 5/7

Leverage temporality

Designing efficient experiments 6/7

Identify source of randomness

Designing efficient experiments 7/7

Debugging is a team effort

Writing code easy to debug

The science of debugging in 3 principles

• There's a logical explanation.
• Understand what happens, do not fix the bug.
• Trust nothing, check all your assumptions.

And a debugging log to bind them all

Questions?

See also The Pocket Guide to Debugging, by Julia Evans