Your party has been wandering in
the Swamp of Sluggish Iterations for several hours...
... when the mage stumbbles upon a grimoire.
Book the Secrets of NumPy
Hi, my name is Sarah Diot-Girard,
and I'll be your game master today.


So you were wandering through
the Swamp of Sluggish Iterations.
picture of the paladin
"We should convert all of our loot values in silver pieces,"
said the paladin,
"we will be so much wealthier."
1 gold piece = 12 silver pieces
And now, it's taking forever to compute...
picture of the mage
"Uh, oh, it's fascinating,"
says the mage,
"NumPy arrays are stored much more efficiently in memory."
Image from Pythonic Perambulations
Oh, look, here stands proudly a tall tower.
That might be your way out of this stinky swamp.

The Tower of Vectorization

You should never use for loops when using NumPy arrays.
Well, maybe we can use this newly found talent to compute loot share for all members of the party.
Oh no, it is not working. You approach the Tower of Vectorization only to find yourself trapped into...

The Maze of Indexing and Broadcasting

picture of the mage
"Oh, I now understand how vectorization is working!"
Broadcasting is the mecanism used by NumPy
to match the dimensions of two arrays.
(Scalars are just zero-dimension arrays.)

Vectors are of dimension 1.

We can switch from row vectors to column vectors
using np.newaxis.

Or None.

For higher dimensional vectors,
np.transpose or .T do the trick (no copy is needed).
image of the party leader
"I want to lead this party
in a data-driven way.
Let's compute the weighted attributes for each member of the group."
image of the bard
"Can I point out that we only considered positive outcomes so far?
Here is the full data."
Integer or boolean arrays can be used
to index NumPy arrays.

Can we select loot values between 5 and 30?

image of the ranger
"I have compiled difficulty levels for the closest dungeons."

What are the difficulty values of dungeons 1, 4 and 7
for floors 0, 2 and 3?

image of the ranger
"I have added which objects can be found in each floor."

Where can I find object 8?
Which objects are available in the range 10 to 20?

image of the party leader
"Can I sort character attributes by heath points?"

From a detailed table, can I get how many pieces each one got in last floors of dungeons?

We want to compute the contribution of each attribute point to the loot.

You can also use Einstein notation.

image of the ranger
"Oops, we have forgotten change rates!"
image of the mage
"I've got one last performance trick."

The Dark Forest of Strides

image of the ranger
"I've recorded our speed at regular intervals during the past days.
However, the measurements are noisy!"

We want to compute the windowed average on 5 measurements.

Strides are virtual views on your array
that add virtual dimensions without copying anything.

They are tricky to use
but can give huge performance boosts.

Image from Software Carpentry

The Village of Ufuncs

image of the mage
"Uh oh, listen to that!
Universal functions are vectorized functions that operates on arrays element-wise."

NumPy comes with a whole lot of unary and binary ufuncs (add, multiply, exponential, trigonometric functions, arithmetic operations, comparisons).

They support broadcasting,
reduce and accumulate operations.

What about more exotic ufuncs?

The Shrine of Jit compiling

image of the ranger
"What's the most optimal path between dungeons?
We'll need to compute an awful lot of distances"
image of the ranger
"What if I want to use a more complex cost function?"

Remember the running average we computed earlier?
Numba can help with that too!


image of the bard
"As this story
comes to its end,
hereby lies
what you should remember!"

For best performance, get out of Python as quickly as possible, and do not go back till the end.

Embrace the world of NumPy fully!

Almost everything you want to do
probably already has an implementation...
in NumPy, in SciPy or in scikit-learn.

Use it, it'll always be faster (and not bugged)!

When in difficulty, Numba will help you.

At least, it's worth a try.

Thank you for participating
in this adventure!

We're hiring !
And we have a RPG club!

Characters illustrations from here .