Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Open In Colab Open In Kaggle

A Python for loop over a million temperature readings runs a million separate additions, each paying the interpreter’s overhead; numpy replaces the loop with one array operation that runs in compiled code, often one to two orders of magnitude faster, and reads closer to the mathematical statement of the problem. That combination — speed and a notation that matches the science — is why numpy underlies the rest of the scientific Python stack, from pandas to scikit-learn to xarray. This notebook works with one running example, a small two-dimensional temperature field on a latitude–longitude grid, to cover array creation, dtype and shape, indexing and masking, vectorised math and broadcasting, reductions, and the handling of missing data — the same topics numpy’s own beginner documentation covers, worth a look for a second pass at these ideas. It closes with a generated-code bug that silently truncates results because of a dtype mistake.

You already know a container for a sequence of numbers: the list. An ndarray differs in three ways that matter for scientific work.

listndarray
Dimensionsone (nested lists to fake more)any number, as a real shape
Contentsanything, mixedone dtype for every element
Arithmetic* repeats, + joinselement-wise maths
Speeda Python loop per elementone compiled loop over the whole block

The single dtype is what buys the speed: because every element has the same type and size, numpy stores them in one contiguous block of memory and hands the loop to compiled code.

[1, 2, 3, 4, 1, 2, 3, 4]
[2 4 6 8]
int str float
int64

1.3.1 Creating arrays

An array is created from data — a nested list — or from a constructor. Every array carries a shape (its size along each axis) and an ndim (the number of axes).

[[ 5.2  4.8  6.1  3.9  2.7  4.4]
 [ 1.3  0.5 -0.8 -1.2  0.9  2.1]
 [-2.6 -3.1 -1.9  0.2 -0.5  1.1]
 [ 6.4  7.   5.5  8.1  4.2  5.8]]
shape: (4, 6) | ndim: 2
Diagram of the temp_celsius array with its shape and ndim labelled

Figure 2:temp_celsius as a grid of 4 rows by 6 columns: shape is the size along each axis, ndim is the number of axes.

1.3.2 Other ways to create arrays

Beyond a literal, numpy provides constructors for the patterns you need most often.

[[0. 0. 0.]
 [0. 0. 0.]]
[1. 1. 1. 1.]
[[7. 7.]
 [7. 7.]]
[0 2 4 6 8]
[0.   0.25 0.5  0.75 1.  ]
[0.40166345 0.14912346 0.76739579]

Coordinates often come as two 1D axes, but a field defined on the grid needs a value at every (row, column) pair. meshgrid expands the two axes into two 2D arrays of matching shape — one holding the longitude of each cell, the other its latitude. You will use these constantly when plotting fields in the next subchapter.

(4, 6) (4, 6)
[[6.  6.6 7.2 7.8 8.4 9. ]
 [6.  6.6 7.2 7.8 8.4 9. ]
 [6.  6.6 7.2 7.8 8.4 9. ]
 [6.  6.6 7.2 7.8 8.4 9. ]]

1.3.3 Indexing, slicing, and boolean masking

Indexing uses [row, col]; slicing selects sub-blocks; negative indices count from the end. A boolean mask is a same-shaped array of True/False that selects the matching elements. Basic slicing (with :) returns a view, not a copy: it shares memory with the original array, so assigning into a slice also changes the array it was sliced from. Call .copy() on the slice when you need an independent array that you can change without touching the original.

5.2
[5.2 4.8 6.1 3.9 2.7 4.4]
[4.4 2.1 1.1 5.8]
[[-0.8 -1.2]
 [-1.9  0.2]]
n freezing cells: 6
freezing values: [-0.8 -1.2 -2.6 -3.1 -1.9 -0.5]
Diagram of four indexing and slicing operations on a 4 by 6 array

Figure 3:Four ways to select from temp_celsius: a single element, a full row, a full column, and a 2x2 sub-block.

Diagram of a boolean mask, the cells it selects, and the resulting 1D array

Figure 4:temp_celsius < 0.0 produces a same-shaped array of True/False; indexing with it, temp_celsius[freezing], pulls out the matching cells as a flat 1D array — not a row or a column, since the selected cells no longer form a rectangle.

1.3.4 Vectorised math and broadcasting

A single expression applies element-wise across an array. Broadcasting lets arrays of different but compatible shapes combine: a length-4 column vector can be stretched across all 6 longitudes.

[[278.35 277.95 279.25 277.05 275.85 277.55]
 [274.45 273.65 272.35 271.95 274.05 275.25]
 [270.55 270.05 271.25 273.35 272.65 274.25]
 [279.55 280.15 278.65 281.25 277.35 278.95]]
Diagram of temp_celsius plus a scalar producing temp_kelvin, with no loop

Figure 5:temp_celsius + 273.15 updates every cell in one expression — no for loop needed.

Broadcasting aligns shapes from the last axis backwards. Our field is (4, 6) and the latitude gradient is (4,) — numpy tries to match the 4 against the 6, fails, and raises an error. We have to say explicitly that the gradient varies along the rows, by giving it a second axis of length 1.

Indexing with None inserts a new axis at that position, turning a (4,) vector into a (4, 1) column. np.newaxis is the same thing spelled out, and you will see both.

(4,)
(4, 1)
(4, 1)
Diagram of temp_celsius (4, 6) and lat_gradient_celsius (4,) failing to broadcast

Figure 6:Broadcasting aligns shapes from the last axis backwards: 6 against 4 does not match, so temp_celsius + lat_gradient_celsius raises a ValueError.

(4, 6)
[[ 5.2  4.8  6.1  3.9  2.7  4.4]
 [-0.2 -1.  -2.3 -2.7 -0.6  0.6]
 [-5.6 -6.1 -4.9 -2.8 -3.5 -1.9]
 [ 1.9  2.5  1.   3.6 -0.3  1.3]]
Diagram of the (4, 1) column broadcasting across the 6 columns of temp_celsius to produce adjusted

Figure 7:The (4, 1) column stretches across all 6 longitudes to combine with the (4, 6) field. (In linear-algebra notation this is A + b·1ᵀ: the column b repeated across every column via an outer product with a row of ones.)

1.3.5 Calling functions on arrays

Most numpy operations are available two ways: as a method on the array (arr.mean()) or as a function in the numpy namespace (np.mean(arr)). They do the same thing — pick whichever reads more clearly.

2.504166666666667 2.504166666666667
60.1 60.1
Diagram of temp_celsius collapsing to a single mean value and a single sum value

Figure 8:.mean() and .sum() with no axis collapse the whole array to one scalar.

1.3.6 Reductions, reshape, and stacking

A reduction collapses an axis: axis=0 aggregates over latitudes (one result per longitude), axis=1 over longitudes. reshape reorganises the same data; stacking combines arrays.

Common reductions all take an optional axis:

FunctionReturns
sum, mean, stdtotal, average, spread
min, maxsmallest / largest value
argmin, argmaxindex of the smallest / largest value
cumsum, cumprodrunning total / product
overall mean: 2.504166666666667
mean per column (over rows): [2.575 2.3   2.225 2.75  1.825 3.35 ]
mean per row (over columns): [ 4.51666667  0.46666667 -1.13333333  6.16666667]
min, max: -3.1 8.1
argmin, argmax (flat index): 13 21
reshaped: (24,)
vstacked: (2, 6)
Diagram of temp_celsius reduced along axis 0 to col_means and along axis 1 to row_means

Figure 9:mean(axis=0) collapses the 4 rows to one value per column; mean(axis=1) collapses the 6 columns to one value per row (shown here as a row too, for comparison).

Diagram of temp_celsius reshaped from (4, 6) into a flat (24,) array

Figure 10:reshape(-1) reorganises the same 24 values into one flat row; nothing is recomputed.

Diagram of index_row and col_means stacked into a (2, 6) array

Figure 11:np.vstack([index_row, col_means]) stacks two (6,) rows into one (2, 6) array.

cumsum and cumprod are the exceptions in the table: they keep every partial result instead of collapsing the axis, so the output has the same shape as the input. For daily rainfall, the cumulative sum is the rain to date:

[ 0.   2.5  2.5  9.5 11. ]
11.0
(4, 6)

1.3.7 Choosing, missing data, and interpolation

np.where picks element-wise between two options. Missing data is represented by NaN; np.isnan flags which elements are missing, NaN-aware reductions (np.nanmean, …) skip them, and ordinary reductions propagate NaN instead. np.interp fills a 1D gap by linear interpolation.

[['above' 'above' 'above' 'above' 'above' 'above']
 ['above' 'above' 'freezing' 'freezing' 'above' 'above']
 ['freezing' 'freezing' 'freezing' 'above' 'freezing' 'above']
 ['above' 'above' 'above' 'above' 'above' 'above']]
nanmean (skips gaps): 2.38695652173913
plain mean is contaminated: nan
n missing: 1
[ 5.2  1.3 -2.6  6.4]
[5.2  1.3  3.85 6.4 ]
Diagram of np.where categorizing cells and a NaN marking a missing cell

Figure 12:np.where(temp_celsius < 0, "freezing", "above") labels every cell; a NaN marks a missing one, propagating through an ordinary .mean() but skipped by np.nanmean().

~ flips a boolean mask, turning every True into False and back. Applied to np.isnan, it flags the values that are present, and indexing with that mask keeps only the real measurements, which are the values np.nanmean averages:

n present: 23 of 24
True

1.3.8 Saving and loading arrays

numpy can write an array to disk in its own binary format, .npy, and read it back exactly as it was — shape, dtype, and all. This is handy for intermediate results.

(4, 6) float64
True

When generated code lies: a silent dtype truncation

AI assistants often preallocate an output array with np.zeros_like, which copies the input’s dtype. If the input is integer, float results are silently truncated on assignment. Here a function computes anomalies (value minus the field mean) for an integer precipitation field.

input dtype: int64
[[-2  0  2]
 [-1 -2  5]
 [ 0  1  0]]
[[-2.77777778 -0.77777778  2.22222222]
 [-1.77777778 -2.77777778  5.22222222]
 [ 0.22222222  1.22222222 -0.77777778]]
output dtype: float64

Summary

ConceptRule to remember
ArraysOne dtype and one shape; build them with np.array, zeros, ones, arange, linspace.
Indexing[row, col] selects, boolean masks filter; a slice is a view, .copy() makes it independent.
VectorisingWrite array expressions instead of element-by-element loops.
BroadcastingShapes align from the last axis backwards; [:, None] adds a length-1 axis.
ReductionsState the axis explicitly (mean, sum, min, max, argmin, argmax).
Missing datanp.where chooses element-wise, np.isnan detects, np.interp fills 1D gaps.
dtypenp.zeros_like(int_array) truncates float results silently — let numpy promote to float.

Resources