Your party has been wandering in
the Swamp of Sluggish Iterations for several hours...

... when the mage stumbbles upon a grimoire.

Hi, my name is Sarah Diot-Girard,
and I'll be your game master today.

Prologue

So you were wandering through
the Swamp of Sluggish Iterations.

"We should convert all of our loot values in silver pieces,"
said the paladin,
"we will be so much wealthier."

1 gold piece = 12 silver pieces

And now, it's taking forever to compute...

"Uh, oh, it's fascinating,"
says the mage,
"NumPy arrays are stored much more efficiently in memory."

Image from Pythonic Perambulations

Oh, look, here stands proudly a tall tower.
That might be your way out of this stinky swamp.

The Tower of Vectorization

You should never use for loops when using NumPy arrays.

Well, maybe we can use this newly found talent to compute loot share for all members of the party.

Oh no, it is not working. You approach the Tower of Vectorization only to find yourself trapped into...

The Maze of Indexing and Broadcasting

"Oh, I now understand how vectorization is working!"

Broadcasting is the mecanism used by NumPy
to match the dimensions of two arrays.
(Scalars are just zero-dimension arrays.)

Vectors are of dimension 1.

We can switch from row vectors to column vectors
using np.newaxis.

Or None.

For higher dimensional vectors,
np.transpose or .T do the trick (no copy is needed).

"I want to lead this party
in a data-driven way.
Let's compute the weighted attributes for each member of the group."

"Can I point out that we only considered positive outcomes so far?
Here is the full data."

Integer or boolean arrays can be used
to index NumPy arrays.

Can we select loot values between 5 and 30?

"I have compiled difficulty levels for the closest dungeons."

What are the difficulty values of dungeons 1, 4 and 7
for floors 0, 2 and 3?

"I have added which objects can be found in each floor."

Where can I find object 8?
Which objects are available in the range 10 to 20?

"Can I sort character attributes by heath points?"

From a detailed table, can I get how many pieces each one got in last floors of dungeons?

We want to compute the contribution of each attribute point to the loot.

You can also use Einstein notation.

"Oops, we have forgotten change rates!"

"I've got one last performance trick."

The Dark Forest of Strides

"I've recorded our speed at regular intervals during the past days.
However, the measurements are noisy!"

We want to compute the windowed average on 5 measurements.

Strides are virtual views on your array
that add virtual dimensions without copying anything.

They are tricky to use
but can give huge performance boosts.

Image from Software Carpentry

The Village of Ufuncs

"Uh oh, listen to that!
Universal functions are vectorized functions that operates on arrays element-wise."

NumPy comes with a whole lot of unary and binary ufuncs (add, multiply, exponential, trigonometric functions, arithmetic operations, comparisons).

They support broadcasting,
reduce and accumulate operations.

What about more exotic ufuncs?

The Shrine of Jit compiling

"What's the most optimal path between dungeons?
We'll need to compute an awful lot of distances"

"What if I want to use a more complex cost function?"

Remember the running average we computed earlier?
Numba can help with that too!

Epilogue

"As this story
comes to its end,
hereby lies
what you should remember!"

For best performance, get out of Python as quickly as possible, and do not go back till the end.

Embrace the world of NumPy fully!

Almost everything you want to do
probably already has an implementation...
in NumPy, in SciPy or in scikit-learn.

Use it, it'll always be faster (and not bugged)!

When in difficulty, Numba will help you.

At least, it's worth a try.

Thank you for participating
in this adventure!

sarah.diot-girard@people-doc.com

We're hiring !
And we have a RPG club!

Characters illustrations from here .