02 — Reading GFLOPS & Roofline¶

Understanding the numbers the ladder prints is essential to knowing why an optimization helps.

What's "GFLOPS"?¶

Each program reports:

\[ \text{GFLOPS} = \frac{\text{arithmetic work}}{10^9 \cdot \text{time}} \]

where the work is double-precision FLOPs (floating-point operations):

matmul C = A·B + C: each of the m×n outputs needs k multiplies and k adds, so \(2mnk\) FLOPs.
matvec y = A·x + y: each of the m outputs needs k multiplies and k adds, so \(2mk\) FLOPs.

The code uses 2.0 * m*n*k * 1.0e-09 (matmul) and 2.0 * m*k * 1.0e-09 (matvec), then divides by seconds.

Example

./build/matmat8 at m=n=k=2000 does \(2\times2000^3 = 1.6\times10^{10}\) FLOPs. At 6 GFLOPS it takes about 2.7 s — which matches what you see.

The two regimes: memory-bound vs compute-bound¶

Whether an optimization matters depends on arithmetic intensity — FLOPs per byte moved.

matvec is memory-bound by nature¶

Matvec reads each of the m×k entries of A exactly once. Its intensity is roughly:

\[ I \approx \frac{2mk}{(mk + k + m)\cdot 8\ \text{bytes}} \approx 0.25\ \text{FLOP/byte}. \]

That's tiny: matvec is dominated by memory bandwidth. No amount of register tiling can turn it compute-bound — the wins come from vectorizing the streaming and reading A densely.

matmul can become compute-bound¶

Matmul reuses each block of A and B many times across output tiles. With cache blocking + packing, intensity rises well above the machine's ridge point, and the kernel becomes compute-bound (limited by FMA throughput, not bandwidth).

Roofline in one sentence

If your kernel is below the ridge it's memory-bound (blocking/packing help). If it's above, it's compute-bound (vectorization/FMA help).

A healthy progression¶

On a typical desktop, you might see roughly:

Stage	matmul GFLOPS (approx)	matvec GFLOPS (approx)
naive	0.5–1.0	0.2–0.5
unroll + micro-kernel	2–4	1–2
register + vector	4–8	3–6
blocked + packed	8–20+	4–7

Exact numbers depend on your hardware — run yours and compare.