SIMD and auto-vectorisation
Axle gives you SIMD in two ways. You can drive vector types lane by lane
(f32x8, i32x4), or let auto-vectorisation rewrite an ordinary array
loop into vector operations for you.
What this page covers: what SIMD is, in one minute · the two surfaces and
which to pick · the manual surface: lane shapes, construction, operations,
bitwise and shift operators, a complete example · reading scalars back out · the
errors a vector mistake produces · what the automatic rewrite covers and what
stays scalar · steering the rest with @vectorize and @unroll · what you can
rely on, and what you cannot.
What SIMD is
SIMD stands for Single Instruction, Multiple Data. An ordinary (scalar)
CPU instruction works on one value at a time: add takes two numbers and
produces one sum. A SIMD instruction applies the same operation to a whole
bundle of values at once. One add produces four, eight or sixteen sums in a
single step.
The bundle lives in a vector register, a wide CPU register (128, 256 or 512
bits) divided into equal lanes, one value per lane. A 256-bit register holds
eight 32-bit integers, so it has eight lanes. A SIMD add on two such
registers adds lane 0 to lane 0, lane 1 to lane 1, and so on. All eight
additions happen in the time one scalar addition takes.
Scalar: four separate adds, one per step
a0 + b0 → c0
a1 + b1 → c1 four instructions,
a2 + b2 → c2 four steps
a3 + b3 → c3
Vector (SIMD): one add across four lanes, in one step
┌────┬────┬────┬────┐ ┌────┬────┬────┬────┐
│ a0 │ a1 │ a2 │ a3 │ + │ b0 │ b1 │ b2 │ b3 │
└────┴────┴────┴────┘ └────┴────┴────┴────┘
│ one vector add instruction │
▼ ▼ ▼ ▼
┌────┬────┬────┬────┐
│ c0 │ c1 │ c2 │ c3 │ one instruction, one step
└────┴────┴────┴────┘ That is the whole idea. If a loop does the same arithmetic to every element of an array, four, eight or sixteen elements per step is a speed-up roughly proportional to the lane count. The lanes run in parallel in hardware. No hidden loop is involved.
Two conditions limit where SIMD applies:
- the lanes must be independent. Lane
icannot need a result that lanei - 1produces in the same step. Such a dependence is a loop-carried dependence, and it blocks the rewrite. - the operation must exist as a machine instruction for that lane shape. Every CPU adds integer vectors. Not every CPU fuses a multiply and an add, which is why the runtime picks the best kernel per host. See CPU dispatch.
Axle’s two SIMD surfaces
- Manual SIMD through the
f32x8,i32x4and similar vector types. You name the lane shape and call the lane operations yourself: FMA, a masked blend, a shuffle, a horizontal sum. Use it when you write the hot kernel (a dot product, a matrix block, a filter tap) and need exact control. - Auto-vectorisation. The compiler recognises a plain array loop and
rewrites it into vector form, with no special types. Use it when a normal
forloop should simply run fast. The@vectorizeand@unrollannotations steer loops the rewrite does not cover.
| You want to | Use |
|---|---|
| write a specific kernel (dot product, FFT butterfly, FIR tap) | manual SIMD: f64x4.fma, v.sum() |
run an ordinary for loop faster | write it plainly; the rewrite covers the common shapes |
| ask LLVM to try a loop the rewrite skipped | @vectorize |
| keep a loop scalar | @vectorize(disable) |
| trade code size for instruction-level parallelism on a short loop | @unroll(N) |
The rest of this page covers the manual surface first, then the automatic rewrite, including what it does and does not do.
Manual SIMD
Vector types
Fixed-lane shapes are named <scalar>x<lanes>. Their total width is 64, 128,
256 or 512 bits:
// 64-bit
i8x8 i16x4 i32x2 f32x2
// 128-bit (universal x86-64 baseline, ARM NEON)
i8x16 i16x8 i32x4 i64x2 f32x4 f64x2
// 256-bit (AVX2, SVE)
i8x32 i16x16 i32x8 i64x4 f32x8 f64x4
// 512-bit (AVX-512)
i8x64 i16x32 i32x16 i64x8 f32x16 f64x8 Every signed shape has an unsigned twin at the same width: u8x16, u32x4, u64x2, and so on. Any other total width is rejected at compile time.
Unsigned lanes
An unsigned vector is a distinct type from its signed twin. Where the two
differ, it selects the unsigned operation: max and min lower to umax and umin, / and % to udiv and urem, and >> to a logical shift. Its lanes
are unsigned scalars, so building one takes u32 values:
let all_ones : u32 = -1 as u32;
let v : u32x4 = u32x4(all_ones, 1, 1, 1);
let hi : u32 = v.max(); // 4294967295: a signed vector would answer 1
let s : i32x4 = i32x4(-1, 1, 1, 1);
let lo : i32 = s.max(); // 1 Operations that do not depend on signedness (+, -, *, &, |, ^, ~, <<, eq, movemask, splat, lane access) behave the same on both. An
unsigned lane’s scalar is the unsigned scalar of its width: u8x16.splat(x) takes a u8, and v.lane(k) on a u32x4 returns a u32. Signed and unsigned
shapes never mix implicitly. u32x4 + i32x4 is rejected with E0001.
Construction
Broadcast a single value across every lane:
let zero : f64x4 = f64x4.splat(0.0); Or list every lane explicitly:
let v : f64x4 = f64x4(1.0, 2.0, 3.0, 4.0); The literal-list constructor takes exactly lanes arguments, each of the lane’s
scalar type. A different count is E0717.
Feeding an f32 shape. A bare 0.0 literal is an f64, and there is no f32 literal suffix. In argument position nothing narrows it, so f32x8.splat(0.0) is rejected with E0716 (expects f32, got f64). An
annotated let does narrow, so bind the value first:
let z : f32 = 0.0;
let zero : f32x8 = f32x8.splat(z); Operations
Two call shapes exist, depending on whether the operation produces a vector or acts on one.
Producers are called on the vector type name (Tx<L>):
| Method | Signature | Description |
|---|---|---|
Tx<L>.splat(v) | (elem) → Tx<L> | broadcast a scalar to every lane |
Tx<L>.load(buf, i) | (elem[] \| elem[N], int) → Tx<L> | typed load at &buf[i], alignment 1; the index is any integer width |
Tx<L>(v0, …) | (elem × L) → Tx<L> | literal-list constructor: exactly lanes arguments |
Tx<L>.select(mask, a, b) | (iWxL, Tx<L>, Tx<L>) → Tx<L> | masked blend; the mask is the matching integer shape (f32xN takes i32xN, f64xN takes i64xN); an integer-lane vector is its own mask |
Tx<L>.convert(v) | (UxL) → Tx<L> | each lane converted to T, lane count kept: a wider lane extends by v’s signedness, a narrower one keeps the low bits; integer lanes only |
Tx<L>.saturate(v) | (UxL) → Tx<L> | each lane narrowed to T and clamped to T’s range instead of wrapped: the pack back to 8 or 16 bits; integer lanes only |
Tx<L>.reinterpret(v) | (any shape of the same width) → Tx<L> | the same bits read as Tx<L>, as in u16x8.reinterpret(i16x8) |
Instance methods are called on the vector value (v.method(…)):
| Method | Signature | Description |
|---|---|---|
v.store(buf, i) | (elem[] \| elem[N], int) → void | typed store at &buf[i], alignment 1; the index is any integer width |
a.fma(b, c) | (Tx<L>, Tx<L>) → Tx<L> | fused multiply-add, a * b + c; float lanes only |
v.sum() | () → elem | horizontal sum |
v.max() / v.min() | () → elem | horizontal maximum or minimum; signed lanes use smax and smin, unsigned lanes umax and umin |
v.shuffle(idx) | (i32[N]) → TxN | lanes of v picked by a compile-time index array; each entry is in [0, lanes). N indices give N lanes, so this also splits a vector: v.shuffle([4, 5, 6, 7]) is the high half |
a.shuffle(b, idx) | (Tx<L>, i32[N]) → TxN | the same over two vectors: entries in [0, L) pick from a, entries in [L, 2L) pick from b. Joins, interleaves and reorders across both |
a.eq(b) | (Tx<L>) → iWxL | lane-wise equality: -1 (all bits set) where lanes match, 0 elsewhere, in the matching integer-mask shape |
a.lt(b) / a.gt(b) | (Tx<L>) → iWxL | lane-wise < and >: -1 where the relation holds, 0 elsewhere, same mask shape as eq. Unsigned lanes order unsigned (0xFFFFFFFF is the largest u32). A NaN lane answers 0. |
v.movemask() | () → i32 or i64 | packs each lane’s most significant bit into the low bits of an i32, or of an i64 for a 64-lane vector; integer lanes only (E0718 on float lanes) |
v.storeCompressed(buf, i, mask) | (elem[] \| elem[N], int, iWxL) → void | writes the lanes whose mask lane is non-zero, one after another, from &buf[i], and nothing else. mask is v’s integer-mask shape. |
a.laneMin(b) / a.laneMax(b) | (Tx<L>) → Tx<L> | the smaller or larger of each lane pair, by the lane’s signedness; integer lanes |
v.abs() | () → Tx<L> | absolute value per lane; the minimum stays the minimum; signed integer lanes |
a.addSaturating(b) / a.subSaturating(b) | (Tx<L>) → Tx<L> | lane sum or difference clamped to the lane’s range; integer lanes |
a.mulHigh(b) | (Tx<L>) → Tx<L> | the high half of each lane’s double-width product, the building block of fixed-point and reciprocal multiplies; integer lanes |
a.dotPairs(b) | (Tx<L>) → WxL/2 | lanes 2i and 2i+1 multiplied at double width and summed (the sum wraps). i16x16 gives i32x8. Signed lanes of 8, 16 or 32 bits. |
v.leadingZeros() / v.popcount() | () → Tx<L> | leading zero bits (a zero lane answers its width) and set bits, per lane; integer lanes |
v[k] / v.lane(k) | (const int) → elem | extracts lane k; k is a literal constant in [0, lanes) |
v.withLane(k, x) | (const int, elem) → Tx<L> | non-destructive lane insert: a new vector; k is a constant in [0, lanes); v is unchanged |
The split is deliberate. An operation that acts on an existing vector is only
the value-receiver form. The type-name spelling of an instance operation, such
as f32x8.sum(v) or f32x8.store(buf, i, v), is rejected with E0714. Call
it on the value instead.
shuffletakes its mask inline. LLVM’sshufflevectorneeds the permutation as a constant operand, so the index array is read straight off the call site:v.shuffle([3, 2, 1, 0]). A mask held in a named binding is rejected with E0716, even when it is ani32[4]local initialised from literals, because the value is a runtime array by the time the call is checked. Write the literal list at the call. When the permutation really varies at run time,storeto a buffer, reorder it there, and reload.
fn reversed(v : i32x4) : i32x4 {
return v.shuffle([3, 2, 1, 0]);
} lt and gt produce the same mask as eq. That mask feeds select for a
branch-free lane-wise choice, and movemask for a bit set:
fn lanewiseMin(a : i32x8, b : i32x8) : i32x8 {
return i32x8.select(a.lt(b), a, b); // the smaller lane of each pair
}
fn lanesAbove(v : u32x8, limit : u32) : i32 {
// bit k is set when lane k is greater than `limit`
return v.gt(u32x8.splat(limit)).movemask();
} The conversions and lane-wise integer operations are what a fixed-point kernel is written in. Here a 16-bit multiply-add is packed back to bytes, the step a colour transform takes for each pixel pair:
fn weighted(samples : i16x16, weights : i16x16) : u8x8 {
let sums : i32x8 = samples.dotPairs(weights); // one multiply-add per pair
let rounded : i32x8 = (sums + i32x8.splat(1 << 13)) >> 14;
return u8x8.saturate(rounded); // clamp to 0..255, no wrap
} On x86 each of these is one instruction. dotPairs on i16 lanes is pmaddwd, mulHigh is pmulhw or pmulhuw, and saturate is a saturating pack. A target
without the instruction computes the same value.
The eq and movemask pair is the canonical probe of a hash table group:
// Locate any matching key in a 4-lane group with one branch.
fn probe(keys : i32[], base : i32, target : i32) : i32 {
let group : i32x4 = i32x4.load(keys, base);
let bits : i32 = group.eq(i32x4.splat(target)).movemask();
// bits is a 4-bit set of matching lanes; a trailing-zero scan finds the slot.
return bits;
} Arithmetic operators +, -, *, / and % work directly between two
vectors of the same shape:
fn ops(a : f32x8, b : f32x8, u : i32x4, v : i32x4, m : u32x4, n : u32x4) {
let c : f32x8 = a + b; // elementwise add
let d : f32x8 = a * b; // elementwise multiply
let r : f32x8 = a % b; // elementwise remainder (`fmod` semantics)
let q : i32x4 = u % v; // signed integer remainder per lane
let w : u32x4 = m % n; // unsigned integer remainder per lane
} Both operands must have identical (elem, lanes) and signedness. There is no
implicit broadcasting and no signed and unsigned mixing. Use .splat(...) to
lift a scalar.
Lane-wise integer / and % carry the same divide-by-zero protection as their
scalar counterparts. If any lane’s divisor is zero, or (for signed integer
lanes) a lane computes INT_MIN / -1, the program aborts with a runtime error
before the divide, exactly as a scalar zero divisor would. Unsigned lanes are
guarded only for a zero divisor, because INT_MIN / -1 is not an overflow for udiv and urem. A bad lane never silently yields a poison value.
Float lanes follow IEEE-754, like scalar floats: a zero lane divisor gives ±Inf or NaN in that lane and does not abort.
let a : f64x4 = f64x4(1.0, 2.0, 3.0, 4.0);
let b : f64x4 = f64x4(2.0, 0.0, 2.0, 2.0);
let q : f64x4 = a / b; // (0.5, +Inf, 1.5, 2.0) Bitwise and shift operators
&, |, ^, ~, <<, >> and >>> work lane by lane on integer-lane
vectors. Their compound forms (&=, |=, ^=, <<=, >>=, >>>=) do too:
fn mix(a : i32x8, b : i32x8, u : u32x8) {
let m : i32x8 = (a & b) ^ ~a; // lane-wise and, xor, complement
let s : i32x8 = a >> 31; // arithmetic: each lane keeps its sign
let z : i32x8 = a >>> 31; // logical: zeros shifted in
let h : u32x8 = u >> 28; // unsigned lanes: logical
let p : i32x8 = a << b; // one count per lane
u >>= 1;
} - The two operands of
&,|and^must have the same shape and signedness, as for arithmetic.a & 1is rejected, so writea & i32x8.splat(1). - A shift count is either a vector of the same shape (one count per lane) or
a single scalar applied to every lane. A scalar count has the lane’s own type.
A literal adopts it, as in
u >> 28on au32x8. A variable of another type needs a cast, as inu >> (n as u32). >>follows the lane type, as its scalar counterpart does. It keeps the sign on signed lanes and shifts in zeros on unsigned ones.>>>always shifts in zeros.- A literal count outside
0..widthis a compile-time error (i32x8 << 32). A count computed at run time is taken modulo the lane’s width, as with a scalar shift:n & 31on 32-bit lanes. - Float lanes have no bitwise operators (E0001).
A complete example
A dot product over f64[8] arrays, using two chunks of four lanes with FMA,
followed by one horizontal sum:
fn dot(a : f64[8], b : f64[8]) : f64 {
let acc : f64x4 = f64x4.splat(0.0);
let va0 : f64x4 = f64x4.load(a, 0);
let vb0 : f64x4 = f64x4.load(b, 0);
let acc1 : f64x4 = va0.fma(vb0, acc);
let va1 : f64x4 = f64x4.load(a, 4);
let vb1 : f64x4 = f64x4.load(b, 4);
let acc2 : f64x4 = va1.fma(vb1, acc1);
return acc2.sum();
} a: [a0 a1 a2 a3 | a4 a5 a6 a7] b: [b0 b1 b2 b3 | b4 b5 b6 b7]
step 1: acc1 = [a0..a3] * [b0..b3] + 0 (fma)
step 2: acc2 = [a4..a7] * [b4..b7] + acc1 (fma)
step 3: acc2.sum() (horizontal reduction) The fma calls on the dispatched shapes (f32x4, f32x8, f64x2, f64x4)
go through a runtime-selected kernel, so the same binary uses the AVX2-and-FMA
variant on a capable host and the baseline on an older one. Other shapes compile
to LLVM’s inline vector intrinsic. You do not recompile or ship a different
binary per CPU level. See CPU dispatch for the model.
Reading scalars back out
A vector is opaque. There are four ways to bring values into locals, depending on what you need:
| You want | Approach | Cost |
|---|---|---|
| an aggregate (sum, maximum, minimum) | v.sum() | one horizontal reduction |
| every lane in locals | v.store(buf, 0), then buf[k] | one store and N loads, often partly removed |
| one lane known at compile time | v[k] or v.lane(k) | one extractelement |
| the pattern of a comparison | m.movemask() | one pack operation |
let v : f64x4 = f64x4(10.0, 20.0, 30.0, 40.0);
let total : f64 = v.sum(); // 100.0: aggregate
let third : f64 = v[2]; // 30.0: one lane, no buffer
let w : f64x4 = v.withLane(2, 99.0); // <10, 20, 99, 40>; v unchanged Errors you might see
| Code | Trigger |
|---|---|
| E0001 | An operator on mismatched vector shapes (u32x4 + i32x4, a & b across lane counts), a vector beside a scalar where no broadcast is allowed (a & 1), a bitwise operator on float lanes, a shift count of the wrong scalar type, or a literal shift count outside 0..width. |
| E0714 | An unknown method on a vector type (f32x8.frobnicate()), or the type-name spelling of an instance operation (f32x8.sum(v) instead of v.sum(), i32x4.lt(a, b) instead of a.lt(b)). |
| E0715 | Wrong number of arguments, as in a.fma(b): fma takes two operands. |
| E0716 | An argument of the wrong type. This includes a.lt(b) or a.eq(b) on two different shapes, such as i32x8.gt(u32x8). Also a lane index (v[k], lane, withLane) that is not a constant or is outside [0, lanes), a shuffle index that is out of range or whose count makes no vector type (i32x4 with three indices), a shuffle mask held in a binding, and a conversion that breaks its width rule (i32x4.convert(i16x8), i32x8.saturate(i16x8), i32x8.reinterpret(i16x8)). |
| E0717 | A literal-list constructor with the wrong lane count, as in f32x8(1.0, 2.0). |
| E0718 | A float-only operation (fma) on integer lanes, an integer-only operation (movemask, convert, laneMin, mulHigh, …) on float lanes, abs on unsigned lanes, or dotPairs on lanes with no signed pair sum. |
| E0292 | @vectorize(width: N) asks for more lanes than the build’s target CPU can execute. See annotations. |
Constraints
- No implicit broadcasting.
f64x4 + 1.0is rejected. Writef64x4.splat(1.0)explicitly. The one exception is a shift count, which may be a scalar of the lane type. - No implicit lane-shape conversion.
f64x4 + f64x2is rejected. Change a lane type withconvert,saturateorreinterpret, and a lane count withshuffle. f32literals are awkward. Axle rejects an implicitf64tof32narrowing, and there is no0.0f32literal. Float SIMD work is easiest in thef64xNshapes.
Auto-vectorisation
Some ordinary array loops vectorise with no annotation and no vector types.
You write a plain for loop, and the compiler rewrites it into vector form:
fn axpy(out : f64[], a : f64[], b : f64[], x : f64, n : i32) {
for (i of 0..n) {
out[i] = a[i] * x + b[i];
}
} The compiler turns such a loop into a vector main loop, which processes one
full register of elements per step. A scalar tail handles the n % lanes leftover elements that the vector loop could not fill:
n = 10, lanes = 4
vector main loop scalar tail
┌───────────┬───────────┐ ┌────┬────┐
│ 0 1 2 3 │ 4 5 6 7 │ │ 8 │ 9 │
└───────────┴───────────┘ └────┴────┘
step 1 step 2 one at a time The rewrite happens in the compiler front end (sema), before LLVM, so the
vector form is guaranteed. It does not depend on an optimiser’s cost model
deciding that it is worthwhile. That is the difference from the @vectorize hint described below, which only asks LLVM to try.
Note: the rewrite is deliberately partial It fires only on the narrow shapes listed below, which are provably safe. Any other loop stays scalar. LLVM’s own vectoriser may still vectorise it, but without the guarantee. This is not a general vectoriser.
What the rewrite covers
The rewrite recognises two loop forms. Both use a for (i of lo..hi) loop over
a range whose bounds are pure: a local, a parameter or a literal.
The element-wise store. The body is one statement, d[i] = <expression>.
Every array read in the expression is s[i] at the same index i. The index
is used only to address arrays, never as a value. Same-index access means each
lane touches only its own element, so the lanes are independent. That makes the
rewrite sound even when d aliases a source, as in d[i] = d[i] + s[i], and no
alias analysis is needed.
Inside that statement, these expressions vectorise:
| Shape | Example | Notes |
|---|---|---|
| map with a constant or invariant | d[i] = s[i] * 2 + c | loop-invariant scalars are broadcast to every lane |
| zip-map of two arrays | d[i] = a[i] + b[i] | |
| AXPY (scale and add) | d[i] = k * a[i] + b[i] | on float lanes, with an FMA-capable host, contracts into one fma |
+, -, * | d[i] = a[i] - b[i] | the core arithmetic |
| integer negation | d[i] = -a[i] | lowered to 0 - a[i], which is exact for integers |
/ by a non-trapping literal | d[i] = a[i] / 4 | integer: the divisor is neither 0 nor -1; float: any literal divisor |
The accumulator reduction. The body is acc = acc + <expression>, where the
expression is an element-wise addend over s[i] as above, and acc is a scalar
local. The addend is vectorised into per-lane partial sums, which are combined
with a horizontal sum() after the loop.
- An integer accumulator is rewritten unconditionally, because integer addition is associative, so the lane split changes nothing.
- A float accumulator is rewritten only under
@float(fast). Splitting a floating-point sum into lane partials changes the rounding, so the compiler will not do it unless the function allows reassociation.
fn sumInts(a : i32[], n : i32) : i32 {
let mut acc : i32 = 0;
for (i of 0..n) {
acc = acc + a[i];
}
return acc;
}
@float(fast)
fn sumFloats(a : f64[], n : i32) : f64 {
let mut acc : f64 = 0.0;
for (i of 0..n) {
acc = acc + a[i];
}
return acc;
} The destination must be a buffer addressed through a pointer. The store
target d is a dynamic T[], or an owned or borrowed pointer. A fixed-size T[N] array is the storage itself, and its loops stay scalar.
Lane width. The rewrite uses the widest register the target executes that
the trip count can fill. That is 512 bits on AVX-512F, 256 on AVX2, and the
128-bit baseline otherwise. When a constant trip count cannot fill the widest
shape, the width halves until it fits. The lane count therefore depends on the
machine you compile for, and a loop of twelve i32 iterations fills eight lanes
but not sixteen.
Trip counts.
- A constant trip count below 8, or below the lane count, is left scalar. The vector loop plus its tail would cost more than the scalar loop.
- A runtime trip count is always rewritten. The tail absorbs any remainder,
and the main loop runs zero times when
nis small. - A counting
while (i <= n)orwhile (i < n)whose limit the compiler can prove stays small (a parameter that every call passesx % 10, for instance, so the loop runs at most eight times) is told not to vectorise at all. That is a hint to LLVM’s own vectoriser, which would otherwise spend a vector body, a reduction and an epilogue on a handful of iterations. A loop the compiler cannot bound this way, or one you annotated with@vectorize, is left as it was.
Bounds. The rewrite never removes a bounds check that the scalar loop would have run. Vector loads and stores are unchecked, so the compiler adds a precheck before the vector loop that traps exactly when the scalar loop would have trapped. That precheck bounds the end of the range only, so when a check is owed (the buffer’s length is not known statically, as with a parameter) the loop must start at a non-negative literal. Otherwise the loop stays scalar.
What the rewrite does NOT do
| Not covered | Why |
|---|---|
Reductions with another operator: max, min, *, - | the pass recognises one accumulator shape, acc = acc + …. Every other operator needs its own identity value and its own horizontal reduction. |
Float reductions without @float(fast) | splitting a float sum into lane partials changes its rounding |
Stencils and shifted index: d[i] = a[i-1] + a[i+1] | lane i would need a neighbour’s input, so the lanes are not independent |
% (modulo) | no safe vector form is emitted, so it stays scalar with its per-element divisor check |
Runtime or trapping /: d[i] = a[i] / x | a vector divide guard would abort a whole chunk, changing which stores had already happened |
Float negation: d[i] = -f[i] on floats | 0.0 - x differs from -x at x = +0.0, because IEEE keeps +0.0 and -0.0 distinct |
Fixed-size T[N] destinations | the array is the storage, and the loop stays scalar |
| Multi-statement bodies, conditional stores, computed indices | outside the single-statement, same-index shape |
| A loop whose start is a runtime value, over a buffer of unknown length | the bounds precheck covers the end of the range only, so it needs a literal start |
A reduction like for (i of 0..n) { sum = sum + a[i]; } is common, and it is
covered for integers. For a max or min reduction, write the manual form
(v.max() over f64x4 accumulators) when the loop is hot.
Steering the rest with annotations
For each loop the rewrite leaves scalar, the annotations below attach to for, while or do-while and steer LLVM’s own vectoriser. Both are honoured at -O1 and above:
@vectorize
fn addInto(out : i32[], a : i32[], b : i32[], n : i32) {
@vectorize
for (i of 0..n) {
out[i] = a[i] + b[i];
}
} Asks LLVM to vectorise this loop. The width is chosen by LLVM, from the host’s SIMD level and the cost of the body.
@vectorize(width: N)
fn mulInto(out : i32[], a : i32[], b : i32[], n : i32) {
@vectorize(width: 8)
for (i of 0..n) {
out[i] = a[i] * b[i];
}
} Pins the vector width to N lanes. The compiler refuses a width that the
machine running the compiler cannot honour. A width above that machine’s maximum
is rejected up front with E0292, and is never quietly downgraded. --target names the output and does not raise the cap.
| Host SIMD level | Maximum lanes (32-bit elements) |
|---|---|
| Baseline (SSE2 on x86-64, NEON on AArch64) | 4 |
| AVX2 | 8 |
| AVX-512F | 16 |
| No-SIMD embedded target | 1 |
@vectorize(disable)
fn fold(table : i32[], n : i32) : i32 {
let mut state : i32 = 0;
@vectorize(disable)
for (i of 0..n) {
// A tight scalar dependency: vectorising would scatter through
// memory more than it saves.
state = state * 31 + table[i];
}
return state;
} Keeps the loop scalar. Use it when the vectoriser would produce strided scatter or gather code that runs slower than the plain form.
@unroll(N)
fn total(values : i32[], count : i32) : i32 {
let mut accum : i32 = 0;
@unroll(4)
for (i of 0..count) {
accum = accum + values[i];
}
return accum;
} Sets the unroll factor to N. N must be a positive integer literal.
What you can rely on
- Every vector method exists on every supported lane shape, except where its
table row names a restriction.
fmais float-only.movemask, the conversions and the lane-wise integer operations are integer-only.absanddotPairswant signed lanes. Each violation is E0718.shuffletakes its index list inline at the call site (see the note under Operations). - The operators
+ - * / %and, on integer lanes,& | ^ ~ << >> >>>compute on each lane exactly what the scalar operator computes on that lane’s value, compound forms included. a.eq(b),a.lt(b)anda.gt(b)set a lane to-1exactly when the scalar==,<or>holds for that lane’s values. The comparison is ordered by the lane’s signedness, and a NaN lane is false. Otherwise the lane is0.- One binary adapts to the host CPU at startup. The same executable runs on a 2010 Westmere CPU, a 2015 Haswell and a 2023 Zen 4.
- Compile errors surface wrong-shape problems at the call site, as the typed codes E0714 to E0718, never as a runtime crash.
- The automatic rewrite covers same-index maps, zip-maps, AXPY, integer
reductions, and float reductions under
@float(fast). For these shapes the vector form is guaranteed whenever the transform passes run, which is-O1and above. At-O0the loop stays as written. Other reductions, stencils,%and float negation are not auto-vectorised. Use the manual types for them. @vectorizeand@unrollare honoured at every optimisation level above-O0.
Limits / Uncabled
| Limit | Why it holds |
|---|---|
No implicit broadcasting: f64x4 + 1.0 is rejected | use f64x4.splat(1.0). The one exception is a shift count, which may be a scalar of the lane type. |
| No implicit lane-shape conversion | change a lane type with convert, saturate or reinterpret, and a lane count with shuffle |
A f32 lane literal cannot be written directly | a bare 0.0 is an f64, and there is no f32 suffix. Bind an annotated let first. |
A shuffle index array must sit inline at the call | LLVM’s shufflevector needs the permutation as a constant operand |
| The automatic rewrite is deliberately partial | reductions with another operator, stencils, %, float negation, fixed-size destinations, runtime loop starts over unknown-length buffers and multi-statement bodies stay scalar. Use the manual surface there. |
Float reductions need @float(fast) | splitting a float sum into lane partials changes its rounding |
| A signed and an unsigned lane shape never mix | u32x4 + i32x4 is rejected with E0001 |
dotPairs and abs are signed-only | each violation is E0718 |
| The lane cap follows the build’s target CPU | @vectorize(width: N) above the cap is E0292; the default target allows 128-bit vectors, raised by --target-cpu, --target-features or native |
The per-construct tables above (Constraints and What the rewrite does NOT do) carry the detail behind each row.
When to pick which
| You want to | Use |
|---|---|
| Express a specific SIMD kernel (dot product, FFT butterfly, FIR tap) | manual SIMD (f64x4.fma and friends) |
Run a plain for faster, and trust the compiler | write the loop; the rewrite covers its common shapes |
| Ask LLVM to vectorise a loop the rewrite skipped | @vectorize |
| Keep a loop scalar that you proved is scalar | @vectorize(disable) |
| Trade code size for instruction-level parallelism on a short loop | @unroll(N) |
The two surfaces combine. You can call a manual SIMD helper from an
auto-vectorised loop. You can also put @vectorize(disable) on the outer loop of
a manually vectorised kernel, so the outer loop does not disturb the inner SIMD
path.
See also
- Optimisations overview: where auto-vectorisation sits among the compiler’s optimisations, and what the compiler tells LLVM about aliasing and ranges that unlocks vector loops
- Annotations reference: every
@…annotation the compiler recognises - Memory model: the storage tiers that vector buffers sit in
- SIMD CPU dispatch: how the runtime picks the AVX2 and FMA variant or the baseline at startup
- Concept index: every SIMD operation on one page