tileir

fn tileiras_args(tileiras: String, gpu_name: String) -> List[String]

The arguments to put before -o <cubin> <bytecode> when tileiras version tileiras (as tileiras --version prints it after V, e.g. "13.4.92") assembles for gpu_name (e.g. "sm_90"): --gpu-name and, for a combination assembler_defect knows to be broken, the arguments that avoid the defect. For tileiras 13.4.92 on sm_90, sm_100, sm_103, sm_107 and sm_110 that is --opt-level 0.

fn assembler_defect(tileiras: String, gpu_name: String) -> Option[String]

Why tileiras_args adds arguments for this combination, or None when it does not: one English sentence naming the assembler, the target, the defect and the issues that track it.

const BYTECODE_MAJOR: Int

The bytecode version written into the header (tile-backend-design 6.3). The three numbers that must move together are this one, the tileiras pin in scripts/tile-golden/toolchain.txt, and the driver in the ledger.

Knife T8 moved the minor from 2 to 3 and knife U2 from 3 to 4. 13.4 is what tileiras 13.4.92 calls its current version; it still accepts 13.1, 13.2 and 13.3, and 13.2 is what this tree pinned from knife 3 to knife T15. Neither move changed a device answer: the cubin tileiras --gpu-name sm_86 produces from the 13.3 bytes is byte for byte the cubin it produced from the 13.2 bytes (tile-backend-design 6.11), and the cubin 13.4.92 produces from the 13.4 bytes is byte for byte the one it produces from the 13.3 bytes (6.18), each measured over every golden rather than assumed.

const BYTECODE_MINOR: Int

The minor half of the bytecode version whose major is BYTECODE_MAJOR; the note there on what must move together covers this number too.

fn bytecode_version() -> String

The version as tileiras --list-versions spells it.

fn ieee_bits(x: Float, p: Int, emin: Int, emax: Int) -> Int

The IEEE pattern of x in the binary format with p significand bits (the hidden bit included) and minimum normal exponent emin, as an Int of p + 1 + the exponent width bits: the sign, the biased exponent, the significand. x must already be a value of the format (the callers round it first); a value above the format's range answers the infinity of its sign and NaN answers the canonical quiet NaN.

fn put_varint_u64(b: Buf, x: Int) -> Buf

x, a 64-bit PATTERN whose top bit may be set, as an LEB128 varint. put_varint refuses a negative Int because a length, an index and a value number never are; a zigzagged float pattern can be.

fn zigzag(x: Int) -> Int

The reader's writeSignedVarInt encoding: zigzag, so a pattern with its sign bit set (every negative float, -inf among them) is a short varint and not a ten-byte one. The shift is arithmetic, which is what makes the mask all ones for a negative value.

fn put_varint(b: Buf, x: Int) -> Buf

x as an unsigned LEB128 varint appended to b.

fn global_payload(dtype: String, values: List[Float]) -> Bytes

The initializer blob of a global: every element's bytes in order, which is the layout get_global's pointer then walks (the dialect: "the initial values are stored in memory in linear order").

fn encode_kernel(k: Kernel) -> Bytes

k as a bytecode file holding one kernel entry.

fn encode(prog: TileProg) -> Bytes

prog lowered and encoded.

type Tile[D]

A tile value inside a kernel: the SSA handle the handler issued for it, of any rank, rank 0 (tile<f64>) included. D is a phantom (spec 2.7): Tile[Float] and Tile[F32] are two types over one representation, so add of an f64 tile and an f32 one is a type error.

The SHAPE is not in the type. It belongs to the value: the recording handler holds every handle's format and shape, a public function reads them off its operands (t_shape_of) rather than taking them as arguments, and a place that needs a particular rank (an if's condition, a loop bound, a reduction's yield) is held to it when the kernel records. Until the batch's PR-3 a rank-0 tile had a type of its own, Scalar[D]; it was folded in here because every reason it existed (an if wants a rank-0 i1, a store wants a tile as wide as its pointers) is a fact the recording sees, and the rank is one dimension of the shape like any other (docs/tile-backend-design.md 6.26).

type Idx

A scalar index tile inside a kernel: a block id, or later what is computed from one. Distinct from Tile[D] (it is i32 and has no format marker) and from a host Int (a host number is not an SSA name; load wants the latter and the checker enforces it). idx_of and idx_as_tile cross between this and a rank-0 Tile[I32] explicitly.

type Param[D]

A kernel parameter: its position in the entry signature and its dtype name. The phantom D is fixed at param() and read back as the string, so load can tell the handler which format it is reading without a dictionary the effect could not carry.

type RedId = IdF(dtype: String, value: Float) | IdI(dtype: String, value: Int)

The identity of one operand of a reduction: what a fold that has seen nothing starts from, in that operand's tile format. reduce carries one per operand.

Its FORMAT is checked outside this repository (tileiras refuses an f32 identity for an f64 operand, measured) and its VALUE is not: a sum whose identity is 1.0 assembles into a cubin. So the value is a layer-2 claim and nothing else, which is what scripts/tile-gpu-diff's reduce-identity-wrong mutant is for.

type AssumePred = DivBy(divisor: Int, every: Option[Int], along: Option[Int]) | SameElements(groups: List[Int]) | Bounded(lb: Option[Int], ub: Option[Int])

What an assume attaches to a value: one of the dialect's three predicate attributes (knife T6). Each is a separate bytecode attribute tag, and the payloads are three different shapes, which is why this is a sum type rather than three fields.

DivBy every element of the tile is a multiple of divisor, which must be a positive power of two. With every and along given, only the FIRST element of each group of every along dimension along is claimed to be a multiple, and the rest of the group is claimed to increase by one from it (by the pointee's width, for a tile of pointers). The two are used together or not at all, and neither may be given for a rank-0 tile. SameElements one value per dimension: dividing dimension k into groups of groups[k], every group holds one repeated value. 1 everywhere is true of any tile. Bounded every element, read as a SIGNED integer, is at least lb and at most ub; either bound may be absent, and both are inclusive.

NOTHING CHECKS THESE. The dialect says so in as many words (Ops.td's AssumeOp: "assume does not check the correctness of the predicate. Incorrect predicates may inject incorrect static information and cause miscompilation"), so a false predicate is undefined behaviour and not a wrong answer. That is why the coverage ledger carries a named layer-3 exemption for assume and this comment is where the reason lives: a mutant that made a predicate false would be measuring nothing.

type Hint

One optimization hint: the dialect's own spelling of the key, and the integer under it. OptimizationHintsAttr (AttrDefs.td) names five keys and every one this backend can carry takes an integer, so a hint is a pair and not a sum type: allow_tma is the one boolean, and it belongs to the two view operations that ruling 2 defers.

Which key is legal is decided by the OPERATION, not by this type (OptimizationHintsAttr::verifyWithOp): an entry takes num_cta_in_cga, num_worker_warps_per_cta and occupancy, a load or a store takes latency. Knife T15's four constructors below are the legal ones, and nothing here refuses the others -- see Hints.

type Hints

The optimization_hints of one entry or one memory operation: the dialect's NESTED dictionary, one entry per architecture. The outer key is an sm_xx name (or default, the fallback the dialect reads when the compilation target has no entry of its own) and the value is that architecture's hints.

NOTHING CHECKS THESE, and that is the point of them. The dialect says a hint is a suggestion: tileiras computes the same answer with them and without them, and its verifier for the keys and the ranges is behind -Wunsupported-hints, off by default, so an unknown architecture, an unknown key and an out-of-range value are all accepted in silence (measured, knife T15; docs/tile-backend-design.md 6.10). What is NOT silent is the SHAPE: every value of the outer dictionary must be a dictionary, keys may not repeat, and the counts on the wire must match what follows them. That is the whole of what layer 1 can say here, and the ledger's exemption for this family says so.

type Ptrs[D]

A tile of POINTERS inside a kernel. D is a phantom as Tile[D]'s is: the element format the pointers point AT, so a Ptrs[Float] where a Ptrs[I32] is wanted is a type error.

Every load and store of the strided surface builds one of these inside the lowering and never shows it. This type is what the three pointer conversions of knife T2 convert, and it exists because they have nothing else to convert: ptr_to_int, int_to_ptr and ptr_to_ptr say nothing about tiles of numbers.

type TensorView[D]

A tensor_view value inside a kernel: a reference to a region of global memory with a static extent and a static stride along each dimension. D is a phantom, as Tile[D]'s is.

type GridView[D]

A GRID view value inside a kernel: a tensor_view cut into a grid of equally sized tiles, which is a partition_view when the grid steps by the tile and a strided_view when it steps by something else. load_view and store_view index the GRID, one index a tile dimension, and the tile they move is tile_shape wide. Knife T11 called this type PartitionView[D]; knife T13 renamed it when strided_view became the second constructor answering it, because the two are ONE surface: they differ in what the type says and in nothing a kernel does with the handle.

type GatherScatterView[D]

A gather_scatter_view value inside a kernel (knife T13). It is a grid view in every dimension but sparse_dim, where the index is a 1D tile of ROW NUMBERS rather than one tile index: a load collects those rows and stacks them into the tile. It is a type of its own rather than a GridView[D] because its indices are a different shape, which is what GatherIdx spells, and because atomic_red_view_tko refuses it.

type GatherIdx = At(i: Idx) | Rows(t: Tile[I32])

One index of a gather/scatter load or store: At is the tile index every other view takes at every dimension, and Rows is the 1D tile the sparse dimension takes. Its length must be the tile's own extent there, which the lowering holds it to.

type Dim = Fixed(n: Int) | Dyn(v: Idx)

One dimension of a tensor_view_dyn: a constant extent or stride, or a value the kernel computed. Fixed is what tensor_view takes for every dimension, so the two spellings say the same thing when nothing is dynamic; Dyn is the ? one.

type Padding = NoPadding | PadZero | PadNegZero | PadNan | PadPosInf | PadNegInf

Which value an out-of-bounds lane of a load_view reads. NoPadding is the dialect's absent padding_value: out-of-bounds lanes then hold unspecified values, which is why no kernel here uses it for a lane it then stores.

The four special values are only legal over a floating point element format; PadZero is legal over any (Types.cpp's verifyPartitionViewLike).

type Overflow = NoAssumption | NoSignedWrap | NoUnsignedWrap | NoWrap

What an integer operation lets the compiler assume about overflow: the dialect's overflow attribute. NoAssumption is none, the default; the other three tell it to assume the operation does not wrap, as a signed operation (no_signed_wrap), as an unsigned one (no_unsigned_wrap) or as either (no_wrap). They are assumptions and not arithmetic: a program that wraps under one of them is undefined, so a caller uses them only where it can prove the range, and no corpus can distinguish them from NoAssumption on a program that keeps its promise.

Written today: add_i with NoSignedWrap, sub_i with NoUnsignedWrap, mul_i with NoWrap (knife T4's three, one per value). Any other pairing is refused when the kernel is recorded; the dialect allows all of them, and teaching the writer and the renderer the rest is a change with a layer-1 answer to show.

type Carry[D]

A device variable of format D: a tile a kernel body replaces as it goes, which a region carries when its body replaces it. Made by carry, read by get, replaced by set; its format and shape are its first tile's for good.

type Step[B] = One | By(by: B)

How a loop steps: by one, or by a bound's own kind of value. One is a constructor and not a literal because a parameter default must be pure (spec, default values): the constant 1 of an Idx loop is an operation the kernel records, so it is made when the loop starts, not when the range is written. A literal step is By(2); a device step is By(nprog) (a grid-stride loop, ruling-drange-step-20261006).

type DRange[B] = { lower: B, upper: B, step: Step[B], unsigned: Bool }

What d_range answers: the range, not yet recorded.

type Visibility = Public | Private

A global's symbol visibility: the dialect's symbol_visibility, a field the Global section's record grew at 13.3. Public is the default.

trait FloatDtype[D] {
  fn float_dtype[D: FloatDtype]() -> Dtype[D]
  fn host_float[D: FloatDtype](v: D) -> Float
}

The float formats a constant can be made in when nothing but the TYPE names the format (tileir 0.10.0): full and mma's accumulator read theirs off this rather than off a witness argument, so let m: Tile[F32] = full([32, 1], 0.0) is an f32 tile and full([32, 1], 0.0) with no expected type does not check.

float_dtype answers the witness of D from the type alone. A function generic in D and given no D value has no other way to learn which format it is making (a type parameter that appears in no argument position cannot be inferred, and a Tile[D] is erased at run time), so D is in the return position and the dictionary this bound passes is that way. std/dtype's HasDtype is the same shape for every format; this one is kept apart because the seven formats here are the dialect's arithmetic floats, the operands mma takes, and HasDtype would let the exponent-only f8E8M0FNU, the sub-byte f4E2M1FN and the integers into full's bound, turning a type error into a recording refusal.

trait RangeBound[B] {
  fn range_idx[B: RangeBound](b: B) -> Idx !Dev
}

What a loop's bounds may be: a host number, fixed when the kernel is recorded (Int), or a value the device computes (Idx). Both come to an Idx when the loop starts; an Int records an idx_const for it.

const DYN_DIM: Int

The extent (or the stride) of a tensor view dimension that is not known until the kernel runs: the dialect prints it ? and carries its value in an OPERAND of make_tensor_view (knife T12). It is a MARKER and not a number: what goes on the wire is ShapedType::kDynamic, and that lives in bytecode.dawn with every other value this package writes. Any non-positive marker would do, because a static extent and a static stride are both strictly positive; this one is minus one so that a shape printed in a diagnostic reads as one.

effect Dev {
  fn t_block_id(axis: Int) -> Int !Dev
  fn t_load(param: Int, dtype: String, base: Int, shape: List[Int], strides: List[Int], mask: Option[Int], pad: Option[Int], hints: List[(String, List[(String, Int)])]) -> Int !Dev
  fn t_store(param: Int, dtype: String, base: Int, shape: List[Int], strides: List[Int], v: Int, mask: Option[Int], hints: List[(String, List[(String, Int)])]) -> Unit !Dev
  fn t_gather(param: Int, dtype: String, index: Int, shape: List[Int], mask: Option[Int], pad: Option[Int]) -> Int !Dev
  fn t_scatter(param: Int, dtype: String, index: Int, shape: List[Int], v: Int, mask: Option[Int]) -> Unit !Dev
  fn t_atomic_rmw(param: Int, dtype: String, mode: String, index: Int, shape: List[Int], v: Int, mask: Option[Int]) -> Int !Dev
  fn t_atomic_cas(param: Int, dtype: String, index: Int, shape: List[Int], cmp: Int, v: Int, mask: Option[Int]) -> Int !Dev
  fn t_scalar(param: Int, dtype: String) -> Int !Dev
  fn t_constf(dtype: String, shape: List[Int], value: Float) -> Int !Dev
  fn t_consti(dtype: String, shape: List[Int], value: Int) -> Int !Dev
  fn t_iota(n: Int) -> Int !Dev
  fn t_lanes(base: Int, shape: List[Int], strides: List[Int]) -> Int !Dev
  fn t_unaryf(op: String, dtype: String, shape: List[Int], a: Int) -> Int !Dev
  fn t_binaryf(op: String, dtype: String, shape: List[Int], a: Int, b: Int) -> Int !Dev
  fn t_fma(dtype: String, shape: List[Int], a: Int, b: Int, c: Int) -> Int !Dev
  fn t_mmaf(dtype: String, acc: String, m: Int, k: Int, n: Int, a: Int, b: Int, c: Int) -> Int !Dev
  fn t_mmaf_scaled(dtype: String, scale: String, dims: List[Int], a: Int, b: Int, c: Int, sa: Int, sb: Int) -> Int !Dev
  fn t_mmai(m: Int, k: Int, n: Int, a: Int, b: Int, c: Int) -> Int !Dev
  fn t_cmpf(pred: String, dtype: String, shape: List[Int], a: Int, b: Int) -> Int !Dev
  fn t_cmpi(pred: String, shape: List[Int], a: Int, b: Int) -> Int !Dev
  fn t_unaryi(op: String, dtype: String, shape: List[Int], a: Int) -> Int !Dev
  fn t_binaryi(op: String, dtype: String, shape: List[Int], a: Int, b: Int) -> Int !Dev
  fn t_convert(op: String, from: String, to: String, shape: List[Int], a: Int) -> Int !Dev
  fn t_repack(op: String, from: String, to: String, shape: List[Int], a: Int) -> Int !Dev
  fn t_select(dtype: String, shape: List[Int], cond: Int, a: Int, b: Int) -> Int !Dev
  fn t_idx_const(value: Int) -> Int !Dev
  fn t_idx_add(a: Int, b: Int) -> Int !Dev
  fn t_idx_mul(a: Int, b: Int) -> Int !Dev
  fn t_loop_begin(lower: Int, upper: Int, step: Int, unsigned: Bool, inits: List[Int]) -> (Int, List[Int]) !Dev
  fn t_loop_end(outs: List[Int]) -> List[Int] !Dev
  fn t_while_begin(inits: List[Int]) -> List[Int] !Dev
  fn t_while_end(cond: Int, outs: List[Int]) -> List[Int] !Dev
  fn t_return_if(cond: Int) -> Unit !Dev
  fn t_spread(dtype: String, shape: List[Int], src: Int) -> Int !Dev
  fn t_reduce_begin(dim: Int, shape: List[Int], identities: List[RedId], operands: List[Int]) -> List[Int] !Dev
  fn t_reduce_end(yields: List[Int]) -> List[Int] !Dev
  fn t_scan_begin(dim: Int, reverse: Bool, shape: List[Int], identities: List[RedId], operands: List[Int]) -> List[Int] !Dev
  fn t_scan_end(yields: List[Int]) -> List[Int] !Dev
  fn t_if_begin(cond: Int) -> Unit !Dev
  fn t_if_else(yields: List[Int]) -> Unit !Dev
  fn t_if_end(yields: List[Int]) -> List[Int] !Dev
  fn t_extract(dtype: String, from: List[Int], to: List[Int], src: Int, indices: List[Int]) -> Int !Dev
  fn t_cat(dtype: String, dim: Int, lhs_shape: List[Int], rhs_shape: List[Int], lhs: Int, rhs: Int) -> Int !Dev
  fn t_permute(dtype: String, shape: List[Int], perm: List[Int], src: Int) -> Int !Dev
  fn t_insert(dtype: String, sub: List[Int], whole: List[Int], src: Int, dst: Int, indices: List[Int]) -> Int !Dev
  fn t_powi(dtype: String, exp_dtype: String, shape: List[Int], a: Int, n: Int) -> Int !Dev
  fn t_num_blocks(axis: Int) -> Int !Dev
  fn t_extent_of(param: Int, dim: Int, blocks: Bool) -> Int !Dev
  fn t_carry_new(init: Int) -> Int !Dev
  fn t_carry_get(c: Int) -> Int !Dev
  fn t_carry_set(c: Int, h: Int) -> Unit !Dev
  fn t_trial_begin() -> Unit !Dev
  fn t_trial_end() -> List[Int] !Dev
  fn t_tok_get() -> Int !Dev
  fn t_tok_set(tok: Int) -> Unit !Dev
  fn t_tok_join(toks: List[Int]) -> Int !Dev
  fn t_shape_of(h: Int) -> (String, List[Int]) !Dev
  fn t_ptrs(param: Int, dtype: String, shape: List[Int]) -> Int !Dev
  fn t_ptr_offset(dtype: String, shape: List[Int], ptrs: Int, offs: Int) -> Int !Dev
  fn t_ptr_to_int(dtype: String, shape: List[Int], ptrs: Int) -> Int !Dev
  fn t_int_to_ptr(dtype: String, shape: List[Int], ints: Int) -> Int !Dev
  fn t_ptr_to_ptr(from: String, to: String, shape: List[Int], ptrs: Int) -> Int !Dev
  fn t_load_ptrs(dtype: String, shape: List[Int], ptrs: Int, mask: Option[Int], pad: Option[Int]) -> Int !Dev
  fn t_assert(shape: List[Int], cond: Int, message: String) -> Unit !Dev
  fn t_assume(pred: AssumePred, dtype: String, shape: List[Int], src: Int) -> Int !Dev
  fn t_print(fmt: String, dtype: String, shape: List[Int], args: List[Int]) -> Unit !Dev
  fn t_token_only(op: String) -> Unit !Dev
  fn t_store_ptrs(dtype: String, shape: List[Int], ptrs: Int, value: Int, mask: Option[Int]) -> Unit !Dev
  fn t_global(sym: String, dtype: String, values: List[Float], align: Int, is_private: Bool, constant: Bool) -> Unit !Dev
  fn t_get_global(sym: String, dtype: String, shape: List[Int]) -> Int !Dev
  fn t_alloca(dtype: String, num_elem: Int, align: Int, shared: Bool, shape: List[Int]) -> Int !Dev
  fn t_tensor_view(param: Int, dtype: String, base: Int, shape: List[Int], strides: List[Int], dyn_shape: List[Int], dyn_strides: List[Int]) -> Int !Dev
  fn t_partition_view(dtype: String, view: Int, tile_shape: List[Int], dim_map: List[Int], padding: Padding) -> Int !Dev
  fn t_strided_view(dtype: String, view: Int, tile_shape: List[Int], traversal: List[Int], dim_map: List[Int], padding: Padding) -> Int !Dev
  fn t_gather_view(dtype: String, view: Int, tile_shape: List[Int], sparse_dim: Int, padding: Padding) -> Int !Dev
  fn t_atomic_red_view(dtype: String, view: Int, indices: List[Int], mode: String, scope: String, value: Int) -> Unit !Dev
  fn t_load_view(dtype: String, view: Int, indices: List[Int], inbounds: List[Bool]) -> Int !Dev
  fn t_store_view(dtype: String, view: Int, indices: List[Int], value: Int, inbounds: List[Bool]) -> Unit !Dev
  fn t_tensor_shape(view: Int, dim: Int) -> Int !Dev
  fn t_index_space_shape(view: Int, dim: Int) -> Int !Dev
  fn t_cell_view(param: Int, dtype: String, free: List[Int]) -> (Int, List[Int]) !Dev
  fn t_cell_fill(param: Int, value: Float) -> Int !Dev
  fn t_sub_view(param: Int, dtype: String, sub: List[Int], at: List[Int]) -> (Int, List[Int]) !Dev
  fn t_retile(param: Int, dtype: String, extent: List[Int], tile: List[Int], pad: Padding) -> Int !Dev
  fn t_reshape(dtype: String, from: List[Int], to: List[Int], src: Int) -> Int !Dev
  fn t_broadcast(dtype: String, from: List[Int], to: List[Int], src: Int) -> Int !Dev
  fn t_call_enter(name: String) -> Unit !Dev
  fn t_call_exit() -> Unit !Dev
  fn t_body_enter() -> Unit !Dev
  fn t_body_exit() -> Unit !Dev
}

The device, as operations a recording handler answers. Names carry a t_ prefix for the reason std/gpu's carry gpu_: use tileir/dev.{Dev} brings every operation into the importing module's namespace (spec 6.5).

t_block_id answers the handle of the block index along axis (0, 1, 2). t_load answers the handle of a tile of dtype and shape whose lane [i, j, ..] is element base + i * strides[0] + j * strides[1] + .. of parameter param; t_store writes one there. base is an index handle and strides are host constants in elements, so a tile of a larger array is a stride rather than an operation of its own. Both take an optional mask (an i1 tile of shape: a lane whose mask bit is 0 is not read and not written), and t_load an optional pad (a tile of dtype and shape: what a masked-off lane reads instead). None for both is the unmasked memory operation of the earlier releases.

t_gather / t_scatter are those same two operations with the element offsets given as an i32 TILE rather than built from base and strides: lane [i, j, ..] reads or writes element index[i, j, ..] of param. There is NO new dialect operation behind them (knife 12). offset, which every load already goes through, takes any tile<Nxi32> as its offset operand, so an index tile that came out of memory is the same instruction as one the stride ladder computed, and all the two spellings differ in is where the numbers come from. mask and pad mean what they mean on t_load: a lane the mask blocks is neither read nor written, so an index outside the array is legal there.

A t_scatter whose index names one element twice has NO defined answer: the dialect does not order the lanes of a single store against each other. Every kernel and every corpus here writes a permutation, and scripts/tile-gpu-diff/gath_diff.dawn says so beside its data.

t_atomic_rmw / t_atomic_cas are the two atomic operations (knife 14), addressed the way t_gather and t_scatter are: an i32 index tile rather than a base and a stride ladder, because the reason to want an atomic is that the destination is data dependent. Both answer the tile of values memory held BEFORE the update, and both take the same optional mask (a lane it blocks is not touched at all).

t_atomic_rmw(param, dtype, mode, index, shape, v, mask) applies mode to the element and the matching lane of v in one transaction: mode is one of and / or / xor / add / addf / max / min / umax / umin / xchg. The integer modes want an integer buffer and addf a float one; add is the integer sum and addf the float one, two modes rather than one because the dialect spells them apart. t_atomic_cas(param, dtype, index, shape, cmp, v, mask) writes the lane of v where the element equals the matching lane of cmp and leaves it alone otherwise, answering what was there either way. Its comparison is on the BITS, so a float cmp that is a NaN never matches and the two zeroes are two values.

Unlike a scatter, an atomic whose lanes name ONE element twice is defined: that is what an atomic is for. What is not defined is the order they arrive in, so the fold has to be one whose answer does not depend on it -- integer addition is, and histogram is the kernel that rests on it.

t_constf / t_consti answer a tile of shape every lane of which holds the given host constant; t_iota the n-wide i32 tile [0, 1, .., n-1] (Tile IR's iota is one-dimensional and this is it); t_lanes the i32 tile of the ELEMENT OFFSETS a load at base with shape and strides reads, which is the same tile the pointer arithmetic of that load is built from.

t_unaryf(op, ..) / t_binaryf(op, ..) / t_fma are the element-wise floating-point arithmetic: op is one of negf / absf for the unary form and addf / subf / mulf / divf / maxf / minf for the binary one. t_cmpf(pred, ..) / t_cmpi(pred, ..) answer an i1 tile of the operands' shape, pred being one of eq / ne / lt / le / gt / ge; a float comparison is ORDERED, so every predicate answers 0 on a NaN operand. t_select takes an i1 tile and two dtype tiles and answers the second where the condition is 1 and the third where it is 0. t_mmaf is the matrix multiply-accumulate: a is [m, k], b is [k, n], c and the answer are [m, n]. dtype is the operands' format and acc the accumulator's and the answer's, which may differ (tileir 0.10.0): the dialect's table pairs each operand format with the accumulators it allows, and the recording holds a kernel to it. t_mmai is the INTEGER one (knife 11): its a and b are i8 tiles, its c and its answer are i32 ones, and both operands are read SIGNED. It takes no dtype argument for that reason -- the three formats are fixed by the dialect (Ops.td's MmaIOp: lhs and rhs are i8, acc and result are i32), and there is nothing for a caller to choose.

t_unaryi(op, ..) / t_binaryi(op, ..) are the element-wise INTEGER arithmetic (knife 10): op is negi / absi for the unary form and one of addi / subi / muli / divi / remi / maxi / mini / mulhii / andi / ori / xori / shli / shri / shru for the binary one. Every one of them is EXACT on the device: an i32 result has one value and no rounding mode, which is why the layer-2 corpus for these has no tolerance branch.

shri and shru are one dialect opcode under two signedness attributes: shri is arithmetic (the sign bit is copied down) and shru is logical (zeros are shifted in), and the two differ on every negative operand. divi / remi / maxi / mini are signed, as t_cmpi is; mulhii takes no signedness at all (the dialect defines it for unsigned operands, and bytecode.dawn says so where it is encoded). Overflow is none everywhere: the device wraps, and so must a host reference (std/gpu's wrap_i32).

t_convert(op, from, to, shape, a) is the width and format change: ftof (a float tile to another float format, nearest_even; knife T17 adds ftof_zero, ftof_neg_inf, ftof_pos_inf and ftof_away, the same conversion under the other four IEEE modes, which bytecode 13.4 allows on some pairs of formats and not on others), itof (integer tile to float, signed, nearest_even), ftoi (float tile to integer, signed, nearest_int_to_zero, the one mode the dialect accepts here; ftoi_sat is the same with 13.4's saturating modifier), exti (a narrower integer tile to a wider one, UNSIGNED, so an i1 mask widens to 0 or 1 rather than to 0 or -1), trunci (a wider integer tile to a narrower one, which for i32 to i1 is the low bit) and bitcast (the same bits read as another format of the same width, i32 and f32 here). extis is exti under the OTHER signedness, spelled as a second name for the reason shri and shru are: one dialect opcode, two names, and the attribute is the whole difference.

t_repack(op, from, to, shape, a) is the other two conversions, pack and unpack, and what separates them from the six above is that the two tiles have different SHAPES. Both reinterpret a rank-1 tile as the byte array it occupies, so shape is the SOURCE's and the answer's lane count is scaled by the ratio of the two element widths; pack answers an i8 tile and unpack takes one. They are the only way to reach a sub-byte format (i4, f4E2M1FN), which no pointer type can hold.

t_idx_const / t_idx_add / t_idx_mul answer scalar i32 handles: a host constant, and the sum and product of two index handles.

t_loop_begin / t_loop_end bracket a region (tile-backend-design 5.2). t_loop_begin(lower, upper, step, unsigned, inits) opens a for over the index handles [lower, upper) by step, with inits as the loop-carried values entering it, and answers the induction variable's handle and one handle per carried value as the body sees them; the operations issued until the matching t_loop_end fall inside the region. t_loop_end(outs) closes it with outs as the values the body hands to the next iteration (one per carried value, in order) and answers one handle per carried value as they stand after the loop. unsigned is the loop's unsignedCmp attribute: false compares the induction variable against the bound as a SIGNED i32 and true as an unsigned one, which is a different loop whenever the range straddles 2^31. The handler carries the memory ordering token through the loop on its own; a body never sees it. Only the d_range family calls the pair; a body calls those.

t_while_begin / t_while_end bracket the OTHER loop region, the one whose trip count is not known when it opens (knife T5). t_while_begin(inits) opens a loop with inits as the loop-carried values entering it and answers one handle per carried value as the body sees them; t_while_end(cond, outs) closes it with the rank-0 i1 handle cond -- true when this iteration is the last -- and outs, the values the NEXT iteration receives, and answers one handle per carried value as they stand after the loop. What the loop answers when it stops is the values that ENTERED the iteration that stopped it, not outs: the handler writes if cond { break <entering> } else { yield } before the continue, so a body reads "run me while cond is false, and give me back the value I was called with". The memory ordering token rides through as the last carried value, exactly as for's does, and it is carried by BOTH terminators (a break yields the loop's final values, and the token is one of them). Only the d_loop family calls the pair.

t_return_if(cond) ends the WHOLE KERNEL for this tile block when the rank-0 i1 handle cond is true and does nothing when it is false: the handler records if cond { return } else { yield } where the call is. Bytecode 13.4 is what lets that if sit inside a loop (knife T17: ReturnOp's parents became entry, if and loop, and its verifier walks through if and loop to the entry); inside a for, a reduction or a scan it is refused at recording time, because the dialect refuses it. Nothing is carried and nothing is answered: what the block stored before the return stays stored, and nothing after it runs.

A shape is a list of tile dimensions, each a power of two, EXCEPT that [] is the rank-0 tile (tile<f64>): the shape a reduction answers and an if branches on. The typed surface reads every shape off its operands, so a kernel body never writes one for a value it already has.

t_constf / t_consti of shape [] record NOTHING when they are issued: the recording handler keeps the value aside and answers a handle for it, and the constant is materialised where it is used, at the shape the use wants (one constant of tile<1024xf64> where a 1024-wide operation takes it, a rank-0 one where a region's body does). That is what "a constant takes no shape" (the batch's C3') costs the recording, and it is why a handle a body holds for a constant may never appear in the recorded program at all.

t_spread(dtype, shape, src) widens a rank-0 tile to shape, every lane the same value: a reshape to one lane and a broadcast to shape. The typed surface never calls it directly: an element-wise operation whose operand is rank 0 has the recording spread it (the rank-0 rule), and broadcast is the explicit spelling.

t_reduce_begin / t_reduce_end and t_if_begin / t_if_else / t_if_end bracket a region the way the loop pair does. t_reduce_begin(dim, shape, identities, operands) opens a reduction of the operands, tiles of shape, along dim, one identity per operand in the same order, and answers TWO handles per operand as the body sees them: for operand i, the ACCUMULATOR at 2 * i and the ELEMENT at 2 * i + 1. The result of a reduction of a rank-r tile is rank r - 1: the dimension dim is dropped.

That order is MEASURED, not read: Ops.td's prose says the region arguments are [op_0_current_iter, op_0_prev_iter, ...] (element first) and its own mlirExample names them (%acc, %elem) (accumulator first), and the two cannot both be true. A kernel whose body answers its first region argument and nothing else answers the FIRST element of the dimension on the device, and one that answers its second answers the LAST, for reduce and for scan, forward and reversed (09-04, sm_86, tileiras 13.3.36). So argument 2 * i is the fold of everything before this point and 2 * i + 1 is the value arriving now. It could never show under reduce alone -- every reduction body in this tree is commutative -- and it decides the answer under scan, whose bodies are not.

The same probe says the IDENTITY is not folded in at the first position: the fold of one element is that element, not f(identity, element). Every identity here is a real identity of its body, so the two readings agree, and this is written down rather than relied on. t_reduce_end(yields) closes it with the new accumulator of each operand and answers one result per operand. t_scan_begin / t_scan_end are the same bracket for the prefix scan: the same dim, shape, identities and operands, the same two handles per operand, and one attribute more, reverse. What differs is the SHAPE OF THE RESULT: a scan keeps every dimension (it answers the running accumulator at each position along dim) where a reduction drops the one it folded. t_if_begin(cond) opens the then-region of an if over the rank-0 i1 handle cond; t_if_else(yields) closes it with what it yields and opens the else-region; t_if_end(yields) closes that one and answers one handle per yielded value. The plan's sketch put no arguments on t_if_else; the then-region's yields have to reach the record somewhere, and this is the only operation of the bracket that can carry them.

t_extract / t_cat / t_permute are the shape operations (knife T2). t_extract(dtype, from, to, src, indices) answers the subtile of src whose SLICE number along dimension k is indices[k]: to must divide from evenly, there is one index per dimension, and the indices are slice numbers rather than element offsets (the dialect's own wording). t_cat(dtype, dim, lhs_shape, rhs_shape, lhs, rhs) joins two tiles that agree in every dimension but dim. t_permute(dtype, shape, perm, src) reorders the dimensions: dimension k of the answer is dimension perm[k] of the source, and the source's rank is at least two. t_insert(dtype, sub, whole, src, dst, indices) is t_extract's other direction (knife T16, bytecode 13.4): it answers dst (of shape whole) with the subtile whose slice number along dimension k is indices[k] replaced by src (of shape sub). The same three rules hold with the two shapes' roles exchanged: sub divides whole, one index per dimension, and the indices are slice numbers.

t_powi(dtype, exp_dtype, shape, a, n) is the one float operation here whose two operands have DIFFERENT formats (knife T16, bytecode 13.4): a is a float tile of dtype and n an integer tile of exp_dtype (i1, i8, i16 or i32, which the dialect reads as SIGNED) of the same shape, so it cannot travel through t_binaryf, whose one dtype is both operands'.

t_num_blocks(axis) answers the extent of the grid along axis, which is t_block_id's companion: t_block_id is where this block is and this is how many there are. A grid-stride loop needs both.

t_tok_get / t_tok_set / t_tok_join are the handler's memory ordering cell, exposed. t_tok_get answers the token every memory operation issued here would take; t_tok_set puts one back, so that two chains can start from the same place; t_tok_join merges two or more of them into one and makes it the current token. They are the only operations here through which a body sees the token at all, and the typed surface has one client for them, d_fork2. Everything else keeps the chain a line.

t_shape_of(h) is the other state query (knife C1'): the element format and the shape the handler holds for handle h, issuing nothing. For a tile it is the tile's type, [] for rank 0, a tile read through a view included. For a tensor_view it is the tensor's extent along each dimension (DYN_DIM where an operand carries it); for a partition, strided or gather/scatter view, the tile one load_view or store_view moves. A token has no shape, and the recording handler refuses the question under the kernel's name and the number of the operation about to be recorded. It is how a public function reads the shape off an operand rather than taking it as an argument, which since the batch's PR-3 is every public function that has an operand.

t_ptrs / t_ptr_offset / t_ptr_to_int / t_int_to_ptr / t_ptr_to_ptr / t_load_ptrs are the pointer surface. t_ptrs(param, dtype, shape) answers a tile of shape every lane of which is the parameter's own address (the reshape and broadcast every ladder opens with); t_ptr_offset advances such a tile by an i32 tile of ELEMENT offsets, the elements being the pointee's; the three conversions are the dialect's own, address-preserving and pure type changes; and t_load_ptrs reads through a pointer tile the body built, sitting in the token chain exactly as t_load does. Between them they are the second way to address memory here, the first being the base and strides of t_load -- and they are the only way to say "the same bytes read as another format", which is what t_ptr_to_ptr is for.

t_assert / t_assume / t_print are knife T6's three, and no two of them are the same kind of thing. t_assert issues an operation with no result that can STOP the launch: the host is told when a lane's condition is false. t_assume issues one whose result is its operand, carrying a claim the compiler may use and nobody checks. t_print is in the token chain (print_tko takes the current token and answers the next), and what it does is not visible in any buffer: it writes to the process's standard output when the context is synchronised.

t_store_ptrs is t_load_ptrs's other direction, and t_global / t_get_global are the static globals (knife T7). t_global(sym, dtype, values, align, is_private, constant) DECLARES one: it adds a record to the module beside the entry rather than an instruction to the body, so it answers nothing and may not be issued inside a region. is_private and constant are the two fields the Global section's record grew at bytecode 13.3 (knife T8); they are booleans here and the writer turns the first into a SymbolVisibility enum value, the way every other enum in this package lives in bytecode.dawn and not on this surface. t_get_global(sym, dtype, shape) answers a tile of shape every lane of which holds the global's own address, the same reshape and broadcast t_ptrs performs on a parameter; from there the pointer surface above reads and writes it. A t_get_global naming a symbol no t_global declared, or declaring the same symbol twice, is refused when the kernel is recorded.

t_mmaf_scaled is the block-scaled matrix multiply (knife T10). It is t_mmaf with two more operands and three formats instead of one: the two matrices are dtype, the two scale tiles are scale, and the accumulator and the result are f32, which is the only output type the dialect allows. dims is [m, k, n, block], where block is the scale vector size V: a scale tile is [m, k / block] on the left and [k / block, n] on the right, and one of its elements scales a run of block elements along K.

The four numbers travel as ONE LIST rather than as four parameters because a Dawn function value takes at most eight parameters (the compiler emits dawn/rt/Fn0 to Fn9 and the evidence pack is one of the nine), and an effect operation is a function value. Eleven parameters compile and then fail at run time with NoClassDefFoundError: dawn/rt/Fn12; this shape has eight.

t_alloca(dtype, num_elem, align, shared, shape) is the automatic allocation (knife T10). It answers a tile of shape every lane of which holds the SAME address, the way t_get_global does, because alloca answers a rank-0 tile<ptr<D>> too. num_elem elements of dtype are reserved, aligned to align bytes, and the allocation lives as long as the block the operation is in: it is scratch within one launch and nothing reads it afterwards. shared is the dialect's global unit attribute, which says the address may be handed to other tile threads; without it the address is private to this one.

t_tensor_view's shape and strides may each hold DYN_DIM where the extent is not known until the kernel runs (knife T12). Those positions print as ? in the type and take an OPERAND instead: dyn_shape and dyn_strides are the handles for them, in the order the markers appear, and the recording handler refuses a count that does not match the markers. Every one of them is a rank-0 i32 tile, which is what an Idx already is.

t_tensor_shape(view, dim) and t_index_space_shape(view, dim) are the two SHAPE QUERIES (knife T12). Each answers the extent along one dimension, and each is t_block_id's shape: the dialect's operation has one result per dimension and the lowering issues the whole of it, binding the handle to the dim-th. t_tensor_shape asks the TENSOR (so its dimensions are the tensor view's own and a ? one is the point of it); t_index_space_shape asks the GRID of tiles a partition view cuts the tensor into, so its rank is the tile's rank and its extent along k is ceil(shape[dim_map[k]] / tile_shape[k]).

t_strided_view and t_gather_view are the other two views (knife T13). A strided view is a partition view whose TRAVERSAL is parametric: index k advances traversal[k] elements rather than tile_shape[k], so consecutive tiles may overlap or skip. A gather/scatter view has no dim_map: its index at sparse_dim is a 1D tile of ROW NUMBERS rather than a tile index, and t_load_view takes that handle in that position. t_atomic_red_view reduces a whole tile into one tile of a partition or a strided view's grid and answers nothing, so it is a Unit operation that still moves the token chain on.

A load or a store inside a reduction or an if is REFUSED: the dialect requires a reduction's body to be pure, and neither region carries the memory ordering token, so a memory operation inside one would have no order at all. t_print is refused there for the same reason, and t_assert and t_assume are not: neither is ordered against memory.

t_cell_view(param, dtype, free) and t_cell_fill(param, value) read the CELLS a parameter's marker gives it (prog.trace1 to trace5; the batch's PR-1, docs/tile-backend-design.md 6.25). t_cell_view answers a partition view over the parameter's whole tensor, cut into its cells and padded as the marker says, and the index of THIS block's cell in it: a dimension that follows grid axis k takes the block id along k, and the dimensions that follow no axis take the handles free names, in order. The view is the handler's to build and to reuse: one per parameter while it is in scope. For an In parameter it is the one view here whose base carries an alignment claim (assume div_by<16> on the parameter itself), because a buffer std/gpu allocates starts on a 256-byte boundary and an In's cells start at its first element; for an Out it carries none. t_cell_fill answers a tile of the parameter's cell shape and format with every lane value, which for an integer format must be a whole number. A parameter with no cells (a Whole or Shared marker, or any parameter under trace_kernel) is refused by both, under the kernel's name and the call that asked.

t_sub_view(param, dtype, sub, at) is t_cell_view's finer cut, for an Out written in pieces (store_sub): a partition view over the same tensor cut into tiles of sub, which must divide the cell's tile in every dimension, and the index of the piece at of THIS block's cell in it (block * (tile / sub) + at along each dimension). at is host numbers, each below tile / sub, so a piece can never leave the block's own cell; the recording refuses one that would.

t_retile(param, dtype, extent, tile, pad) is the escape hatch for a parameter read two ways (the batch's C4): a partition view over the parameter's whole row-major tensor of extent, cut into tile, padded with pad, which load_view then reads at any index. It answers the view alone; an In's carries the alignment claim its cells do.

t_reshape(dtype, from, to, src) regroups the lanes of src into to, which holds as many: what a reduction that keeps its dimension issues after the fold. t_broadcast(dtype, from, to, src) widens the dimensions of length 1 in from to their length in to, every other dimension unchanged and the rank the same; a rank-0 tile is t_spread's.

t_call_enter(name) and t_call_exit() record nothing a kernel computes. Every public function below wraps its body in called(name, ..), which brackets it with the two, so a handler can tell which public call issued which operations: prog.trace_calls keeps the kernel's own calls as a side table beside the program, and every other handler answers (). They are the only way that is known exactly, because the mapping is not one to one (tile_at issues t_idx_const and t_idx_mul, add issues t_binaryf, d_range a whole region).

t_body_enter() and t_body_exit() record nothing either. Every public function that takes a closure (d_range, d_range, d_loop, the reductions, the scans, d_if, d_fork2) runs that closure between the two, and runs nothing else between them; a region's trial (Carry) runs it between them too, and the trial's calls are thrown away with the rest. A call marked inside a body is one the KERNEL wrote, in the region the closure fills; a call marked anywhere else inside another call is one the library made on its own behalf (idx_mul inside tile_at). That difference is what lets prog.trace_calls keep a tree of the kernel's calls rather than only the outermost ones.

fn for_arch(arch: String, hints: List[(String, Int)]) -> (String, List[(String, Int)])

The hints of one architecture, for the list Hints is. arch is an sm_xx name or default.

fn hint_num_cta_in_cga(n: Int) -> (String, Int)

num_cta_in_cga: how many CTAs to put in a CGA. An ENTRY hint. The dialect wants a power of two no greater than 16, and 1 on sm_80 to sm_89, which is every architecture this machine has.

fn hint_num_worker_warps_per_cta(n: Int) -> (String, Int)

num_worker_warps_per_cta: how many worker warps to give a CTA. An ENTRY hint. The dialect wants a power of two no greater than 32 and clamps anything but 4 or 8 into that range.

fn hint_occupancy(n: Int) -> (String, Int)

occupancy: how many blocks to aim to keep resident. An ENTRY hint, and the one key AttrDefs.td's table leaves out while verifyWithOp accepts it. The dialect wants 1 to 32.

fn hint_latency(n: Int) -> (String, Int)

latency: the latency to assume for a memory operation. A LOAD or STORE hint, and the only key either of them takes (allow_tma is the view operations'). The dialect wants 1 to 10.

fn is_float_dtype(name: String) -> Bool

Whether name is one of the dialect's floating point formats. What the special padding values are confined to.

fn dim_extent(d: Dim) -> Int

The extent of a Dim as the type states it: the constant, or DYN_DIM for the ones an operand carries.

fn dim_operands(ds: List[Dim]) -> List[Int]

The handles of the dynamic entries of ds, in order. What make_tensor_view takes as its dynamicShape or dynamicStrides group, and its length is how many DYN_DIM markers dim_extent produced.

fn inbounds_note(inbounds: List[Bool]) -> String

The suffix a recorded load_view or store_view carries in a handler's log when it makes an inbounds promise, and nothing when it makes none, so the logs of every kernel that makes none stay as they were.

fn padding_name(p: Padding) -> String

The keyword the dialect spells a padding value with, for a diagnostic and for the renderer. The NUMBER each one carries on the wire is not here: it lives in bytecode.dawn with every other enum this package writes, and this surface only ever names the value.

fn padding_is_special(p: Padding) -> Bool

Whether p is one of the four SPECIAL padding values, which verifyPartitionViewLike allows only over a floating point element type. zero is legal over any format and the absent parameter is legal everywhere.

fn padding_present(p: Padding) -> Bool

Whether the view carries a padding_value at all. An atomic_red_view_tko refuses one (AtomicRedViewTkoOp::verify) and a load past the end of the tensor reads it.

fn param[D](d: Dtype[D], pos: Int) -> Param[D]

Parameter pos of the entry, of format d. Positions count from zero and must agree with the params list given to trace_kernel; the handler refuses a load or store that disagrees.

fn position[D](p: Param[D]) -> Int

The position p was declared at.

fn param_dtype[D](p: Param[D]) -> String

The dtype name p was declared with ("f64", "f32").

fn block_id(axis: Int) -> Idx !Dev

The block index along axis: 0, 1 or 2.

fn tile_at(idx: Idx, n: Int) -> Idx !Dev

The element offset of tile block idx in a run of n-element tiles: idx * n. What load and store want as their base, and the one place a block index becomes an element offset; a kernel over a multi-dimensional grid builds its base from several of these (idx_add / idx_mul), which is why the memory operations take the offset rather than the block index.

fn row_major(shape: List[Int]) -> List[Int]

The row-major strides of shape: the suffix products, [1] for a one-dimensional tile. What load and store use when the tile is a contiguous run of memory (strides: None); strides: Some(..) is the one to use when it is a block of a larger array.

fn permute(strides: List[Int], perm: List[Int]) -> List[Int]

The strides of a PERMUTED view of an array laid out by strides: dimension k of the view is dimension perm[k] of the array, so permute(row_major([r, c]), [1, 0]) is the layout that reads a row-major r by c array transposed. A permutation is a rearrangement of the strides and nothing else: no operation is issued, no element moves, and the tile a load answers is already in the order the permuted view has. This is not the dialect's permute (opcode 0x53), which knife T2 added as permute_tile below: a transpose of an array being READ is a layout and issues nothing, where a transpose of a tile already in registers is an instruction. Both are here because both are real; which one a kernel wants is decided by where the elements are.

Why the layout is two lists rather than one Layout record: knife 8 fixed the pointer path's load as a shape and a strides list, and since the batch's PR-3 the shape is the one geometry load still takes (the arithmetic reads every shape off its operands). A record would be a second spelling of the same pair, not a shorter one.

fn axis_strides(rank: Int, axis: Int, step: Int) -> List[Int]

The strides that walk ONE axis of a tile of rank dimensions and hold every other still: step at axis and zero elsewhere. A load with these reads the same element down a whole axis, and lanes_strided with them answers that axis's coordinate.

fn coord(base: Idx, shape: List[Int], axis: Int, step: Int) -> Tile[I32] !Dev

The i32 tile of shape whose lane [i, j, ..] holds the coordinate of that lane along axis, counted from base in steps of step: lane [i, j] of coord(base, shape, 0, 1) is base + i. The one thing a multi-dimensional boundary needs and a flat lanes cannot say, because a tile at the edge of an array is outside it along one axis and inside it along the other.

fn and_mask(a: Tile[Bool], b: Tile[Bool]) -> Tile[Bool] !Dev

The lanes both masks select: a select, a where it is 1 choosing b and 0 elsewhere. Knife 10 brought andi, which computes the same tile over i1 operands; this is still a select because switching it would re-record six kernels that were measured against a device under the spelling they have, and buy no claim the andi clients below do not already make. The shape is the masks' own.

fn or_mask(a: Tile[Bool], b: Tile[Bool]) -> Tile[Bool] !Dev

The lanes either mask selects: a select, a where it is 1 choosing the all-true tile and b elsewhere. The or to and_mask's and, written the same way and for the same reason.

fn in_range(c: Tile[I32], lo: Int, hi: Int) -> Tile[Bool] !Dev

The lanes whose coordinate c satisfies lo <= c < hi, as a mask of c's shape. Both bounds are host constants; lo is not always zero, because a stencil reads a coordinate one before the tile's own.

fn axis_mask(base: Idx, shape: List[Int], axis: Int, limit: Int) -> Tile[Bool] !Dev

The lanes whose axis coordinate, counted from base, is below limit: tail_mask for one axis of a multi-dimensional tile. A tile over an array whose extent is not a multiple of the tile's covers the whole of the last block along each axis, and this is the mask that keeps it from reading or writing what is past the end along that one. Like tail_mask it tests the upper bound alone, because the coordinate it is given starts at a block's own base and counts up; a stencil, whose taps reach BELOW that base, wants in_range. shape is the tile's: a mask has no operand to read it from.

fn load[D](p: Param[D], base: Idx, shape: List[Int], strides: Option[List[Int]] = None, mask: Option[Tile[Bool]] = None, pad: Option[Tile[D]] = None, hints: List[(String, List[(String, Int)])] = []) -> Tile[D] !Dev

The tile of p's format and shape whose element [i, j, ..] is at element base + i * strides[0] + j * strides[1] + .. of p: THE load of the pointer path, the escape hatch beside load_cell for every read a parameter's cells do not describe (a stride of zero, a stencil's neighbour, a tile that starts between two cells). The five names 0.6.0 spelled it with were folded into it in 0.7.0 and removed in 0.8.0.

shape is the one shape this takes, because it is the one shape no operand carries: it is the block's geometry, what the load reads.

• strides: None is row_major(shape), a contiguous tile. Some(s) is every other layout: s is in ELEMENTS and is a host constant, so a tile of a larger array is a stride of that array's row length rather than a new operation (the ladder is one iota per dimension, reshaped and broadcast to shape, scaled by that dimension's stride and summed).
• mask: None reads every lane, so every element named must be inside p. Some(m), an i1 tile of shape, reads only the lanes whose bit is 1; a lane whose bit is 0 is not read at all.
• pad: what a lane the mask blocks answers, a tile of p's format: a rank-0 one (f_const(F64, 0.0)) is widened to shape. With a mask and no pad such a lane answers an UNSPECIFIED value (the dialect's load with no paddingValue), which is right for a lane that is only ever stored back under the same mask. A pad with no mask is refused when the kernel is recorded: every lane is read, so it would name a value nothing can answer.
• hints: optimization_hints on load_ptr_tko (knife T15); the one key a load takes is latency. A hint does not change the answer, only the bytes.

Why strides is an Option rather than defaulting to row_major(shape): the shape-once ruling makes strides the exception a kernel states, not a value every call recomputes and shows.

fn store[D](p: Param[D], base: Idx, v: Tile[D], strides: Option[List[Int]] = None, mask: Option[Tile[Bool]] = None, hints: List[(String, List[(String, Int)])] = []) -> Unit !Dev

Write the tile v at base: THE store of the pointer path, load's other direction, with the options load has and no pad. The shape is v's own, so it cannot disagree with the value; a rank-0 v is refused (a store writes a tile with a dimension: broadcast(v, [1]) is one). strides lays the tile out as load's does; mask writes only the lanes whose bit is 1 and leaves memory as it was under the rest; hints is optimization_hints on store_ptr_tko, whose one key is latency. Without a mask every lane is written, so every element named must be inside p.

An Out parameter is written through its cells (store_cell, store_sub) and nothing else: a store into one is refused when the kernel records, and so is one into an In. A Shared parameter, or any parameter of trace_kernel, takes this.

fn gather[D](p: Param[D], index: Tile[I32], mask: Option[Tile[Bool]] = None, pad: Option[Tile[D]] = None) -> Tile[D] !Dev

The tile of p's format whose lane [i, j, ..] is element index[i, j, ..] of p, shaped like index: a load whose element offsets come from a tile instead of from base and strides. Without a mask every lane is read, so every index must be inside p; with one, a lane whose bit is 0 is not read at all and answers the matching lane of pad (a rank-0 pad is widened). That is how an index OUTSIDE the array is handled, and the reason the corpora carry such indices: the mask, not the arithmetic, is what keeps the read inside the buffer.

Nothing new reaches the device here (knife 12). A strided load already builds an i32 tile of element offsets and hands it to offset; this hands offset a tile somebody else computed. Which is why an index tile LOADED from a buffer is the whole of data-dependent addressing: load it, then gather with it. 0.8.0 folded gather_masked into the two options, as 0.7.0 did the pointer path's loads.

fn scatter[D](p: Param[D], index: Tile[I32], v: Tile[D], mask: Option[Tile[Bool]] = None) -> Unit !Dev

Write lane [i, j, ..] of v to element index[i, j, ..] of p: a store whose element offsets come from a tile. Without a mask every lane is written, so every index must be inside p; with one, a lane whose bit is 0 leaves memory as it was, which is what an index outside the array gets. NO TWO LANES WRITTEN MAY NAME THE SAME ELEMENT: the dialect gives the lanes of one store no order against each other, so a duplicated index has no answer at all rather than one of two. A permutation is the shape every caller here writes. A scatter is not a cell write, so its parameter is a Shared one.

fn atomic_rmw[D](p: Param[D], mode: String, index: Tile[I32], v: Tile[D], mask: Option[Tile[Bool]] = None) -> Tile[D] !Dev

Apply mode to element index[i, j, ..] of p and lane [i, j, ..] of v, one indivisible transaction per lane, and answer what each element held BEFORE. Only the lanes mask selects take part (every lane without one); the others are not read and not written, which is how an index outside the array is handled, exactly as on scatter.

Two lanes MAY name one element here -- that is the whole point of an atomic, and the difference between this and scatter. The order they arrive in is still unspecified, so a caller may only use a mode whose answer does not depend on it. Integer add is such a mode and is the one histogram uses; xchg is not, and a kernel that used it on colliding lanes would have no answer to compare against.

fn atomic_add(p: Param[I32], index: Tile[I32], v: Tile[I32], mask: Option[Tile[Bool]] = None) -> Unit !Dev

Add lane [i, j, ..] of v to element index[i, j, ..] of the integer buffer p where mask selects it, and throw the old value away: the scatter-with-collisions a histogram is. Named because the returned tile is what a caller forgets to use, and a function that never answers it cannot be misread as a plain store.

fn atomic_cas[D](p: Param[D], index: Tile[I32], cmp: Tile[D], v: Tile[D], mask: Option[Tile[Bool]] = None) -> Tile[D] !Dev

Compare element index[i, j, ..] of p against lane [i, j, ..] of cmp, write the matching lane of v where the two are equal, leave the element alone where they are not, and answer what was there either way -- one indivisible transaction per lane, on the lanes mask selects (every lane without one). The comparison is on the BITS and not on the format's own equality, so two NaNs with different payloads are two values.

fn add[D](a: Tile[D], b: Tile[D], rounding: Rounding = NearestEven, ftz: Bool = false) -> Tile[D] !Dev

The element-wise sum of two tiles of one format.

rounding is the dialect's rounding attribute and ftz its flush_to_zero unit attribute, both at the dialect's own defaults (nearest_even, not set). NearestEven is the one mode tile-backend-design 3.2's theorem covers and what every kernel here writes unless it says otherwise. The other values this package writes are listed at "the attributes of float arithmetic" below.

fn sub[D](a: Tile[D], b: Tile[D], rounding: Rounding = NearestEven, ftz: Bool = false) -> Tile[D] !Dev

The element-wise difference a - b. rounding and ftz as for add.

fn mul[D](a: Tile[D], b: Tile[D], rounding: Rounding = NearestEven, ftz: Bool = false) -> Tile[D] !Dev

The element-wise product a * b. rounding and ftz as for add.

fn div[D](a: Tile[D], b: Tile[D], rounding: Rounding = NearestEven, ftz: Bool = false) -> Tile[D] !Dev

The element-wise quotient a / b. rounding and ftz as for add.

fn max[D](a: Tile[D], b: Tile[D], propagate_nan: Bool = false) -> Tile[D] !Dev

The element-wise larger of a and b. Without propagate_nan (the dialect's default), a NaN operand answers the other operand; with it, a NaN operand answers a canonical NaN.

fn min[D](a: Tile[D], b: Tile[D], propagate_nan: Bool = false) -> Tile[D] !Dev

The element-wise smaller of a and b. propagate_nan as for max.

fn rounding_name(r: Rounding) -> String

The dialect's spelling of a rounding direction: the keyword rounding<..> is written with, and the one std/narrow's file header pairs with each variant. The renderer spells the modes from the operation names, and this module's tests hold the two to each other.

fn sqrt_approx[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise square root under the APPROXIMATE rounding mode: the architecture's fast reciprocal-square-root path rather than the correctly rounded root. f32 only. A tolerance-tier operation by construction, and the tolerance is the hardware's, not a series'.

fn neg[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise negation -a: a sign flip, exact for every value.

fn abs[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise absolute value |a|: a sign clear, exact for every value.

fn exp[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise exponential e^a. The device computes it with the CUDA Math API (the dialect's full rounding mode, its default and the only one an f64 tile accepts here), which is an APPROXIMATION: a kernel built on it is a tolerance-tier kernel and no corpus makes it exact (docs/tile-backend-design.md 6.5).

At bytecode 13.2 exp carries no attribute at all. Its rounding_mode field was added in 13.3, and a 13.2 reader that met one would take the next byte of the stream for something else (measured: failed to get result type 0 for CmpIOp).

fn exp2[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise 2^a, without flush_to_zero. Tolerance tier.

fn log[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise natural logarithm. Tolerance tier.

fn log2[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise base-2 logarithm. Tolerance tier.

fn sqrt[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise square root, rounding<nearest_even> and no flush_to_zero. Unlike its neighbours here this one is a CORRECTLY ROUNDED operation in IEEE 754, so a kernel built on it can be held to the bit-exact tier; the tier is a measurement, not an assumption, and scripts/tile-gpu-diff records which one each kernel passed under.

fn rsqrt[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise reciprocal square root, without flush_to_zero: a hardware approximation, tolerance tier.

fn tanh[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise hyperbolic tangent, rounding<full> (the only mode an f64 tile accepts: approx is f32-only and nearest_even is refused, both measured). Tolerance tier.

fn floor[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise largest integer not above a. Exact for every value it is defined on, so a kernel built on it stays in the bit-exact tier.

fn ceil[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise smallest integer not below a. Exact.

fn pow[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise a raised to the power b. Tolerance tier.

fn powi[D, E](a: Tile[D], n: Tile[E]) -> Tile[D] !Dev

The element-wise a raised to the INTEGER power n: Tile IR's fpowi (knife T16, bytecode 13.4), pow's sibling with an exponent tile of an integer format (i1, i8, i16 or i32) instead of a float one. The dialect reads n as SIGNED whatever its width, so an i8 lane holding 0xFF is the power -1 and not 255.

EXACT tier, but not against the power a mathematician means. Over f64 the device answers right-to-left binary exponentiation of a to |n| (one correctly rounded product a step) and, for a negative n, the reciprocal of that: bit for bit on every lane measured on an RTX 3080 (tileiras 13.4.92, sm_86), and up to 34 units in the last place away from the correctly rounded power on the same corpus (docs/tile-backend-design.md 6.19). a^0 is 1.0 for every a, a NaN included. tileref's ref_fpowi is that sequence; pow above, with a float exponent, is a different operation and stays tolerance tier.

fn sin[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise sine. The device runs the CUDA Math API's, an APPROXIMATION, and the host second opinion is tileref/ref.ref_sin, an independent range reduction and Taylor series. Tolerance tier.

fn cos[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise cosine. Tolerance tier.

fn tan[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise tangent. Tolerance tier, and the one of the five with poles: near an odd multiple of pi / 2 the answer is enormous and the relative part of the tolerance is what carries the comparison, so a corpus must not sit on a pole.

fn sinh[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise hyperbolic sine. Tolerance tier. It grows like e^|x|, so a corpus reaching |x| = 700 in f64 is at the edge of the format.

fn cosh[D](a: Tile[D]) -> Tile[D] !Dev

The element-wise hyperbolic cosine. Tolerance tier.

fn atan2[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise atan2 of a over b, with the quadrant taken from the signs of both.

THE ARGUMENT ORDER IS THE NUMERATOR FIRST, which is C's atan2(y, x) with the names swapped: the dialect calls its first operand x and its second y and defines the operation as "the arc tangent of the ratio of first and second input arguments x / y" (Ops.td, Atan2Op). So the dialect's x is C's y. Reading the operand names instead of the sentence gets the two backwards on every lane off the positive axis, which is what the layer-2 mutant atan2-operands-swapped is. The quadrant behaviour is measured rather than assumed, in scripts/tile-gpu-diff.

Tolerance tier.

fn remf[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise floating-point remainder of a by b.

TRUNCATED division, not IEEE remainder: the dialect states it as a - trunc(a / b) * b, so the answer takes the sign of the DIVIDEND and its magnitude is below |b| (Ops.td, RemFOp). That is C's fmod and not C's remainder, and the two differ on every lane where the true quotient is more than half a step from its truncation, so a corpus of negative dividends and of |a| < |b| is what tells them apart. The device is what settles it and scripts/tile-gpu-diff is where it was settled.

The value is EXACT wherever it is defined: the remainder of two representable numbers is itself representable. So a kernel whose only operation is this one could be held to the bit-exact tier; the sweep that covers it here is a tolerance-tier kernel because its neighbours are.

fn fma[D](a: Tile[D], b: Tile[D], c: Tile[D]) -> Tile[D] !Dev

The element-wise fused multiply-add a * b + c, rounded ONCE (the product is not rounded before the sum). A host reference that multiplies and then adds agrees with this only where the product is exact.

fn erf[D](a: Tile[D]) -> Tile[D] !Dev

The error function of a, element-wise, BY COMPOSITION.

Abramowitz and Stegun, Handbook of Mathematical Functions, formula 7.1.26: for x >= 0, with t = 1 / (1 + 0.3275911 x),

erf(x) = 1 - (a1 t + a2 t^2 + a3 t^3 + a4 t^4 + a5 t^5) exp(-x^2) + e

with |e| <= 1.5e-7 for every x >= 0 (the handbook's own bound; the largest this tree has measured is 1.385e-7, in a std/gpu test block). The negative half comes from the odd symmetry erf(-x) = -erf(x), which is what the select below is: 7.1.26 holds for x >= 0 only, and dropping the sign flip answers erf(|x|).

1.5e-7 is inside the atol = 1e-5 leetgpu compares under (docs/tile-backend-design.md 6.5), so every kernel built on this is a TOLERANCE tier kernel and cannot be anything else. The host reference it is compared against is tileref/ref.ref_erf, a series accurate to 4e-15, so the difference a device run reports IS this error and not a disagreement between two approximations.

Operations used, all of them already here: absf, mulf, addf, divf, negf, exp, subf, a float compare and a select. The constants take no shape: each is materialised where it is used, at the operand's.

fn mma[A, C: FloatDtype](a: Tile[A], b: Tile[A], acc: Tile[C]) -> Tile[C] !Dev

The matrix multiply-accumulate a * b + acc: a is [m, k], b is [k, n] and acc and the answer are [m, n]. The two operands share one format and the accumulator may have another (tileir 0.10.0; 0.9's mmaf took three of one format): bf16 operands into an f32 accumulator is mma(a, b, acc) with a: Tile[BF16] and acc: Tile[F32]. The pairs the dialect allows (Ops.td's MmaFOp table) are the only ones the recording takes:

and it refuses any other pair with this table in the message. The accumulator's format is C, read from the type (FloatDtype), so a format-less lit(0.0) accumulator is made in it: C comes from the accumulator's own type or from what the answer is checked against, as in let s: Tile[F32] = mma(a, b, lit(0.0)).

The three numbers are the operands' own, and the recording refuses a b whose K is not a's; a rank-0 accumulator is widened to [m, n]. This is the ONE operation here whose summation over k has no specified order and no promised instruction: tileiras --remarks=tensorcore says whether the device used a tensor core or plain FMAs, and either way a kernel built on it is a tolerance-tier kernel (docs/tile-backend-design.md 6.5).

fast_acc is a 13.3 flag and is never written here. The Tile IR operation is still mmaf; the integer one is mmai below.

fn mma_scaled[D, S](a: Tile[D], b: Tile[D], c: Tile[F32], sa: Tile[S], sb: Tile[S]) -> Tile[F32] !Dev

The BLOCK-SCALED floating-point matrix multiply-accumulate (knife T10; mmaf_scaled before tileir 0.10.0): mma for the low-precision formats that carry a separate exponent per run of K. a is [m, k], b is [k, n], c and the answer are f32 tiles of [m, n], and sa and sb are scale tiles of [m, k / block] and [k / block, n]: each scale element multiplies a run of block elements along K in the matrix beside it, and block is read off sa.

The dialect allows two combinations and nothing else (Ops.td's table): f8E4M3FN, f8E5M2 or f4E2M1FN operands with f8E8M0FNU scales, and f4E2M1FN operands with f8E4M3FN scales. The result is f32 in both. There is no combination with a wide operand format, which is why this operation cannot be assembled for this machine's sm_86 at all: every operand type it takes is one tileiras refuses below sm_100.

fn mmai(a: Tile[I8], b: Tile[I8], c: Tile[I32]) -> Tile[I32] !Dev

The INTEGER matrix multiply-accumulate a * b + c (knife 11): a is an i8 tile of [m, k], b an i8 tile of [k, n], and c and the answer are i32 tiles of [m, n]. Both operands are read SIGNED (the dialect carries signedness_lhs and signedness_rhs separately; this surface fixes both, because a kernel that wanted the unsigned reading would want a u8 tile type the dialect does not have).

Unlike mma, this is an EXACT operation: every partial product of two i8 values fits in 15 bits and the accumulator is i32, so the sum over k is the same integer whatever order the device folds it in. A kernel built on it is exact tier, and that is a theorem rather than a corpus.

It keeps its i where the float mma dropped its f (tileir 0.10.0), as the integer arithmetic keeps addi beside the float add: its formats are the dialect's and not the caller's, its tier is exact where mma's is tolerance, and one name over both would check an I8 operand against the float table and refuse it only when the kernel records.

fn f_const[D: FloatDtype](d: Dtype[D], value: D) -> Tile[D] !Dev

The rank-0 tile of format d holding value, fixed when the kernel is recorded, which is what a host value in a kernel body means. value is a D (tileir 0.13.0), so the rounding to the format happens where the host value is made, in plain sight: f_const(BF16, 0.1) types the literal as a BF16 and records the bfloat16 nearest to 0.1, the 0.10009765625 the device holds, and a Float computed on the host is f_const(BF16, bf16(x)).

fn lit[D](value: Float) -> Tile[D] !Dev

A float constant with no format of its own: mul(s, lit(0.5)) is mul(s, f_const(F64, 0.5)) when s is a Tile[Float] and mul(s, f_const(F32, 0.5)) when it is a Tile[F32] (tileir 0.8.2). As a constant's shape is the use's, so is its format: the recording makes it where it meets an element-wise operation, of that operation's format (JAX calls this a weakly typed constant). D is found from what the call is checked against, so lit is written after a typed operand (mul(s, lit(0.5)), not mul(lit(0.5), s)) or under an annotation. A place that needs a format of its own -- a broadcast, a reduction, a loop's starting value, a store, an integer operation -- refuses a lit by name and says to write f_const there.

fn full[D: FloatDtype](shape: List[Int], value: D) -> Tile[D] !Dev

The tile of shape every lane of which holds value, in the format the call is checked against (tileir 0.10.0): let m: Tile[F32] = full([32, 1], 0.0), or a Carry[F32] annotation on the carry it starts. Like lit, its format comes from where it is used; unlike lit, it has one of its own once made, so it can be what a broadcast or a loop's starting value cannot take a lit for. It records exactly what broadcast(f_const(F32, 0.0), [32, 1]) records (one constant of that shape, made where it is used), so swapping the two changes no byte.

zeros(p) and fill(p, v) are the same tile shaped like a cell of p in p's format; they stay, because there the parameter already says both, and they also make integer tiles, which this does not.

fn i_const(value: Int) -> Tile[I32] !Dev

The rank-0 i32 tile holding value.

fn i8_const(value: Int) -> Tile[I8] !Dev

The rank-0 i8 tile holding the low octet of value, read as a two's-complement signed byte. The padding value of a masked load over an 8-bit buffer, and the only way to name an i8 constant here: there is no i8 ARITHMETIC in this surface (exti widens to i32 and trunci comes back), so this and the memory operations are the whole of what an i8 tile does.

fn bool_const(value: Bool) -> Tile[Bool] !Dev

The rank-0 i1 tile holding value.

fn arange(n: Int) -> Tile[I32] !Dev

The n-wide i32 tile [0, 1, .., n-1].

fn lanes_strided(base: Idx, shape: List[Int], strides: List[Int]) -> Tile[I32] !Dev

The i32 tile of shape holding the ELEMENT OFFSET each lane of a tile at base laid out by strides reads: base + i * strides[0] + ... The same ladder a load at that base builds, memoised, so a masked load costs no instruction more than an unmasked one. shape and strides are the block's geometry, which no operand carries.

fn lanes(base: Idx, shape: List[Int]) -> Tile[I32] !Dev

lanes_strided over a contiguous tile, which is the run of element indices [base, base + 1, .., base + prod(shape) - 1] in row-major order.

fn eq[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a == b over two tiles of one float format, as a mask of their shape. Ordered: a NaN operand answers 0 for every one of the six comparisons this heads, ne included.

fn ne[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a != b: one of the six ordered predicates eq heads, so a NaN operand answers 0 here too.

fn lt[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a < b: one of the six ordered predicates eq heads.

fn le[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a <= b: one of the six ordered predicates eq heads.

fn gt[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a > b: one of the six ordered predicates eq heads.

fn ge[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a >= b: one of the six ordered predicates eq heads.

fn eq_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a == b or either operand a NaN.

fn ne_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a != b or either operand a NaN.

fn lt_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a < b or either operand a NaN.

fn le_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a <= b or either operand a NaN.

fn gt_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a > b or either operand a NaN.

fn ge_u[D](a: Tile[D], b: Tile[D]) -> Tile[Bool] !Dev

a >= b or either operand a NaN.

fn eq_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

The same six over two i32 tiles, signed: a == b as a mask of their shape.

fn ne_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

a != b over two i32 tiles, signed: one of the six eq_i heads.

fn lt_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

a < b over two i32 tiles, signed: one of the six eq_i heads.

fn le_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

a <= b over two i32 tiles, signed: one of the six eq_i heads.

fn gt_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

a > b over two i32 tiles, signed: one of the six eq_i heads.

fn ge_i(a: Tile[I32], b: Tile[I32]) -> Tile[Bool] !Dev

a >= b over two i32 tiles, signed: one of the six eq_i heads.

fn add_i(a: Tile[I32], b: Tile[I32], overflow: Overflow = NoAssumption) -> Tile[I32] !Dev

The element-wise sum a + b, wrapping modulo 2^32 under the default overflow; see Overflow for the other three.

fn sub_i(a: Tile[I32], b: Tile[I32], overflow: Overflow = NoAssumption) -> Tile[I32] !Dev

The element-wise difference a - b, wrapping. overflow as for add_i.

fn mul_i(a: Tile[I32], b: Tile[I32], overflow: Overflow = NoAssumption) -> Tile[I32] !Dev

The element-wise product a * b, wrapping: the LOW 32 bits of the 64-bit product. mulhi_i is the other half. overflow as for add_i.

fn div_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise signed quotient a / b, rounded toward zero (the dialect's default rounding for divi, and to_int's). Division by zero is undefined on the device, as it is in CUDA.

fn rem_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise signed remainder a % b, whose sign follows the dividend (the remainder of a quotient rounded toward zero).

fn max_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise signed larger of a and b.

fn min_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise signed smaller of a and b.

fn mulhi_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The HIGH 32 bits of the 64-bit product of a and b, read as UNSIGNED operands: the dialect defines mulhii for unsigned integers and gives it no signedness attribute (Ops.td, MulhiIOp). A hash that needs the whole product takes this beside mul_i.

fn and_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise bitwise a & b.

fn or_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise bitwise a | b.

fn xor_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise bitwise a ^ b.

fn shl_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise left shift a << b, wrapping: the bits shifted past the top are dropped. A shift count outside [0, 32) is undefined on the device.

fn shr_i(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise ARITHMETIC right shift a >> b: the sign bit is copied into the vacated positions, so a negative operand stays negative. This is shri with signedness<signed>.

fn shr_u(a: Tile[I32], b: Tile[I32]) -> Tile[I32] !Dev

The element-wise LOGICAL right shift a >>> b: zeros are shifted in, so a negative operand becomes a large positive one. This is the SAME opcode as shr_i with signedness<unsigned>, and the two differ on every negative operand and nowhere else -- which is what scripts/tile-gpu-diff's shri-always-logical mutant measures.

fn neg_i(a: Tile[I32]) -> Tile[I32] !Dev

The element-wise negation -a, wrapping (so -(-2^31) is -2^31).

fn abs_i(a: Tile[I32]) -> Tile[I32] !Dev

The element-wise absolute value |a|, wrapping (so |-2^31| is -2^31).

fn overflow_name(o: Overflow) -> String

The dialect's spelling of an overflow assumption, as overflow<..> writes it.

fn addi[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise sum a + b over two integer tiles of one format, wrapping.

fn subi[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise difference a - b, wrapping.

fn muli[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise product a * b, wrapping: the low bits of the exact product, as many as the format holds.

fn shli[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise left shift a << b, wrapping. A shift count outside [0, width) is undefined on the device.

fn shri[D](a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise ARITHMETIC right shift a >> b: the sign bit is copied into the vacated positions.

fn consti[D](d: Dtype[D], value: Int) -> Tile[D] !Dev

The rank-0 tile of the integer format d holding value, truncated to the format's width. i_const is the i32 spelling.

fn int_to_float[A, B](a: Tile[A], to_d: Dtype[B]) -> Tile[B] !Dev

The float tile of format to_d holding each lane of the signed integer tile a, rounded to nearest even. Exact for every i32 into an f64 tile (a binary64 significand covers the whole range) and not always exact into an f32 one, nor from an i64 tile.

fn float_to_int[A, B](a: Tile[A], to_d: Dtype[B], saturating: Bool = false) -> Tile[B] !Dev

The integer tile of format to_d holding each lane of the float tile a, TRUNCATED TOWARD ZERO.

Without saturating (the default) a lane that is infinite, NaN or outside the target's range is undefined on the device (the dialect says so), so a kernel that uses this owes a bound on its input. With it, bytecode 13.4's saturating modifier (knife T17), every lane has an answer: a lane above the target's largest value is that value, one below its smallest is that one, an infinity is the end of its own sign and a NaN is 0 (Ops.td's FToIOp says all four in as many words); the rest truncate toward zero either way. saturating is legal on every ftoi the dialect allows (FToIOp has no per-format rule for it), so the recording handler checks it by the same rule as the plain conversion: the operation name ftoi_sat is on its list.

fn mask_to_int(a: Tile[Bool]) -> Tile[I32] !Dev

The i32 tile holding 1 where the mask is set and 0 where it is not: an exti from i1, ZERO-extending. Signedness is the whole of the claim here -- a signed widening of a one-bit 1 is -1, not 1 -- and it is why a count of the lanes a comparison selects is this and not a select.

fn int_to_mask(a: Tile[I32]) -> Tile[Bool] !Dev

The mask holding the LOW BIT of each lane of a: a trunci from i32 to i1. The parity test, and the one direction back from an integer to a mask that costs no comparison.

fn bits_to_float(a: Tile[I32]) -> Tile[F32] !Dev

The f32 tile whose lanes hold the BITS of a unchanged: a bitcast, which moves no bits and reads them as another format of the same width. Not int_to_float, which preserves the value rather than the pattern.

fn bits_to_int(a: Tile[F32]) -> Tile[I32] !Dev

The i32 tile whose lanes hold the bits of the f32 tile a unchanged: the inverse of bits_to_float, and its round trip is the identity on every bit pattern.

fn float_to_float[A, B](a: Tile[A], to_d: Dtype[B], rounding: Rounding = NearestEven) -> Tile[B] !Dev

The float tile of format to_d holding each lane of the float tile a, rounded by rounding: the dialect's ftof. Widening (f16 to f64) is exact and narrowing rounds.

Which modes a pair of formats may take is the dialect's table (Ops.td's FToFOp at bytecode 13.4, knife T17; prog.ftof_modes is it), and the recording handler refuses a kernel that asks for another, with the pair and the modes it allows. On an exact widening every mode is allowed; otherwise NearestAway only from f32 to tf32, Down only from f64 to f32, Up from f64 to f32 and to f8E8M0FNU, and TowardZero to every format but the 8-bit and 4-bit floats other than f8E8M0FNU. The table is the dialect's to the letter, so it is checked here rather than left to tileiras.

The default is the one exception to "the mode it names": into f8E8M0FNU, which has no nearest_even, the default conversion writes zero, the dialect's own default there, as it did before rounding was a parameter. TowardZero writes the same mode under its own name.

The two formats must DIFFER (Ops.td's FToFOp says so and its verifier enforces it); ftof from a format to itself is refused by tileiras, not here.

fn to[A, B](a: Tile[A], fmt: Dtype[B]) -> Tile[B] !Dev

a in the float format fmt, rounded to nearest even: p.to(BF16) (tileir 0.10.0). float_to_float with its default rounding, except that a tile already in fmt is answered as it is and records nothing, where float_to_float would ask tileiras for an ftof between equal formats, which it refuses. A kernel generic in its format can then write p.to(D) and get no instruction where D is the format p has. Another rounding is float_to_float's third argument.

fn ext_u8(a: Tile[I8]) -> Tile[I32] !Dev

The i32 tile holding each lane of the i8 tile a widened UNSIGNED, so an octet reads as a value in [0, 255]. This is how a kernel over a DeviceBuffer[U8] gets at its pixels: the dialect has no u8 tile type, and the signedness of the widening is where the format's reading lives (std/gpu's U8 says the same thing from the host's side).

fn trunc_u8(a: Tile[I32]) -> Tile[I8] !Dev

The i8 tile holding the LOW OCTET of each lane of the i32 tile a: a trunci, which keeps the bits and drops the rest. What a kernel stores back into an 8-bit buffer, and the reason a value outside [0, 255] is the caller's problem rather than an error: the octet is what the buffer holds, and std/gpu's wrap_u8 is the host reading of the same byte.

fn ext_i4(a: Tile[I4]) -> Tile[I32] !Dev

The i32 tile holding each lane of the i4 tile a widened with its SIGN, so a nibble reads as a value in [-8, 7]. The dialect has no arithmetic at four bits at all (Types.td leaves i4 out of CudaTile_IntTileType), so this is how a kernel computes over one: widen, compute at 32 bits, trunc_i4 back.

ext_u4 is the same widening the other way and answers [0, 15]. Which of the two a kernel takes is a device answer on every nibble whose top bit is set, and scripts/tile-gpu-diff's exti-i4-zero-extends is that sentence as a mutant.

fn ext_u4(a: Tile[I4]) -> Tile[I32] !Dev

The i32 tile holding each lane of the i4 tile a widened with zeros, so a nibble reads as a value in [0, 15].

fn trunc_i4(a: Tile[I32]) -> Tile[I4] !Dev

The i4 tile holding the LOW NIBBLE of each lane of the i32 tile a: a trunci, which keeps the bits and drops the rest. The wrap at four bits, and std/gpu's wrap_i4 is the host reading of the same nibble.

fn pack_bytes[A](a: Tile[A]) -> Tile[I8] !Dev

The rank-1 i8 tile holding the BYTES of the rank-1 tile a: the dialect's pack. No bit moves; the tile is reinterpreted as the byte array it occupies, so the answer has lanes * bits / 8 lanes.

Both tiles must be RANK 1 (PackOp's verifier says so, to leave no ambiguity about which lane is which byte) and a's format may not itself be an 8-bit format (bitcast is that one). A sub-byte format packs two lanes to a byte, the EVEN lane in the low nibble; Types.td writes that order out and std/gpu.i4_lo is the host reading of it.

fn unpack_bytes[B](a: Tile[I8], to_d: Dtype[B]) -> Tile[B] !Dev

The rank-1 tile of format to_d the rank-1 i8 tile a holds: the dialect's unpack, and pack_bytes's inverse. The answer has lanes * 8 / bits(to_d) lanes, so unpacking a byte tile into a sub-byte format DOUBLES the lane count.

fn tail_mask(base: Idx, shape: List[Int], limit: Int) -> Tile[Bool] !Dev

The mask of the lanes at base that fall inside a buffer of limit elements: lanes(base, shape) < limit. The boundary of a pointer-path kernel over a vector whose length is not a multiple of the tile width; a parameter's cells (load_cell, store_cell) carry their own and need none. shape is the block's geometry.

fn select[D](cond: Tile[Bool], a: Tile[D], b: Tile[D]) -> Tile[D] !Dev

The element-wise choice: lane i of a where lane i of cond is 1, lane i of b where it is 0. A rank-0 condition or value is widened to the shape of the others.

fn idx_const(value: Int) -> Idx !Dev

The scalar index tile holding the host constant value: a loop bound, a step, a stride. The constant is fixed when the kernel is recorded, which is what a host Int in a kernel body means (tile-backend-design 5.2).

fn idx_add(a: Idx, b: Idx) -> Idx !Dev

The sum of two index tiles.

fn idx_mul(a: Idx, b: Idx) -> Idx !Dev

The product of two index tiles.

fn idx_div(a: Idx, b: Idx) -> Idx !Dev

The quotient a / b of two index tiles, rounded toward zero.

fn idx_rem(a: Idx, b: Idx) -> Idx !Dev

The remainder of a / b, of a's sign.

fn idx_sub(a: Idx, b: Idx) -> Idx !Dev

The difference a - b of two index tiles.

fn carry[D](init: Tile[D]) -> Carry[D] !Dev

A device variable starting at init. init must have a format of its own (f_const, not lit) and a shape: a loop's accumulator starts from zeros(p) or a broadcast, as before.

fn get[D](c: Carry[D]) -> Tile[D] !Dev

The tile c holds where the body stands: inside a region that carries it, the region's block argument; after the region, its result.

fn set[D](c: Carry[D], t: Tile[D]) -> Unit !Dev

Replace the tile c holds with t, of the same format and shape (the recording refuses any other). Not inside a reduction's or a scan's body, whose regions carry nothing.

fn d_range[B: RangeBound](lower: B, upper: B, step: Step[B] = One, unsigned_cmp: Bool = false) -> DRange[B]

The index range [lower, upper) by step, to loop over with for:

for j in d_range(0, K / TK) { acc = mma(load_at(a, [j]), load_at(b, [j]), acc) }

The body runs once, on the host, with the induction variable (an Idx whichever kind the bounds are), and what it issues falls inside the loop's region. A var the body assigns is carried through the loop and holds what the last iteration assigned after it. A body may nest a for; the handler bounds the depth. The memory ordering token is carried by the handler, so a store after the loop is ordered after every store inside it. break, continue and return are not allowed in the body (staged-for-design); a data-dependent exit is d_loop.

With Int bounds the range records idx_const for lower, upper and step, in that order; with Idx bounds it records only what the step needs.

unsigned_cmp sets the loop's unsignedCmp unit attribute (knife T4): the induction variable is compared against the upper bound as an UNSIGNED 32-bit integer, so a range whose bounds straddle 2^31 runs forward instead of not running at all. Its own kernel is scripts/tile-golden's attr_ucmp.

fn d_loop(body: fn() -> Tile[Bool] !Dev) -> Unit !Dev

The loop whose trip count the DATA decides: body runs on the host and answers a rank-0 mask, true when the loop is to stop here; the carries it sets are carried (Carry). When the mask is true the loop stops with the carries as they ENTERED that iteration, so the loop reads "while the mask is false, keep what the body set"; what body set on the iteration that stopped it is discarded. Compute the mask from what the iteration started with, before the sets, unless that is not what you mean.

This is d_range's sibling and the only other region that iterates. The difference is where the trip count comes from: d_range's bounds are index handles the host or the grid supplies, and this one's exit is a value the kernel computes from the tiles it is carrying. A body that never answers a true mask never terminates on the device, exactly as a while (true) would; nothing here bounds it.

Do not store inside one unless the store is meant to happen on the discarded iteration too: body runs to the end every time, including the last.

fn d_return_if(cond: Tile[Bool]) -> Unit !Dev

End the kernel for this tile block where cond is true, from wherever the call is: the top of the kernel, an if, or -- what bytecode 13.4 added (knife T17) -- the body of a d_loop, where it leaves the loop AND the kernel at once. Stores issued before it are kept; nothing recorded after it runs for a block whose cond was true, including the code after the loop. Where cond is false it does nothing and the body carries on.

Not inside d_range, a reduction or a scan: the dialect forbids a return under a for (use the loop for an early exit, Ops.td says) and the other two are pure regions. The recording refuses all three.

fn idx_eq(a: Idx, b: Idx) -> Tile[Bool] !Dev

a == b over two index tiles, as the rank-0 mask d_if branches on. Signed, as every integer comparison here is.

fn idx_ne(a: Idx, b: Idx) -> Tile[Bool] !Dev

a != b over two index tiles, signed, as a rank-0 mask: one of the six index predicates idx_eq heads.

fn idx_lt(a: Idx, b: Idx) -> Tile[Bool] !Dev

a < b over two index tiles, signed, as a rank-0 mask: one of the six index predicates idx_eq heads.

fn idx_le(a: Idx, b: Idx) -> Tile[Bool] !Dev

a <= b over two index tiles, signed, as a rank-0 mask: one of the six index predicates idx_eq heads.

fn idx_gt(a: Idx, b: Idx) -> Tile[Bool] !Dev

a > b over two index tiles, signed, as a rank-0 mask: one of the six index predicates idx_eq heads.

fn idx_ge(a: Idx, b: Idx) -> Tile[Bool] !Dev

a >= b over two index tiles, signed, as a rank-0 mask: one of the six index predicates idx_eq heads.

fn d_reduce[D](t: Tile[D], identity: Float, body: fn(Tile[D], Tile[D]) -> Tile[D] !Dev, dim: Int = -1) -> Tile[D] !Dev

The reduction of the tile t along dimension dim (the last by default; -k is the k-th from the end), which answers t with that dimension DROPPED: a [64, 32] tile along dimension 1 answers a [64] one, and a [1024] tile answers a rank-0 one. body runs ONCE, on the host, with the accumulator and an element of the tile as the region sees them, IN THAT ORDER, both rank 0, and answers the new accumulator; what it issues in between is the region. identity is what a fold that has seen nothing starts from (0.0 for a sum, -inf for a max), written in t's format (a whole number for an integer one), and must be the identity of body or the answer is wrong on the device and nowhere else.

THE ORDER IS NOT SPECIFIED. Tile IR requires the function to be associative and commutative and says nothing about the tree, so a host reference that folds left is not an oracle for this in general (docs/tile-backend-design.md 6.5). A corpus on which every order is exact, or a tolerance, is. Integer addition modulo 2^32 is exactly associative and commutative, so an integer sum's answer does not depend on the order at all.

fn d_reduce2[A](values: Tile[A], idxs: Tile[I32], value_identity: Float, index_identity: Int, body: fn(Tile[A], Tile[I32], Tile[A], Tile[I32]) -> (Tile[A], Tile[I32]) !Dev, dim: Int = -1) -> (Tile[A], Tile[I32]) !Dev

d_reduce over TWO tiles at once, a tile of values and the i32 index tile beside it: the shape argmax and argmin have, where the winning value and the lane it came from must travel together through one fold. body takes the accumulator of each and then the element of each, and answers the new accumulator of each.

fn d_scan[D](t: Tile[D], identity: Float, body: fn(Tile[D], Tile[D]) -> Tile[D] !Dev, dim: Int = -1, reverse: Bool = false) -> Tile[D] !Dev

The inclusive prefix scan of the tile t along dimension dim (the last by default): an answer of the SAME shape whose position j is the fold of everything at or before j along that dimension. body runs ONCE, on the host, with the ACCUMULATOR (everything folded so far) and the ELEMENT arriving now, in that order, and answers the new accumulator; what it issues in between is the region. With reverse the prefix runs the other way: position j folds everything at or after it, and the accumulator is still the first argument.

identity is what a fold that has seen nothing starts from, as a reduction's is. The device does not fold it in at the first position (measured, see t_reduce_begin above), so it must be a real identity of body or nothing at all depends on it.

THE EVALUATION ORDER IS NOT SPECIFIED, the same way d_reduce's is not: the dialect asks the function to be associative and says the order within each prefix is implementation-defined. Measured on sm_86, the device's float prefix sum is NOT the sequential left fold -- it differs in the last bits from lane 2 onwards -- so a host reference that folds left is not a bit oracle for a float scan, and a float scan kernel is tolerance tier (docs/tile-backend-design.md 6.6). What a scan does NOT need is COMMUTATIVITY: every element keeps its side of the accumulator, which is why a non-commutative body (an affine composition, "the last value that carried a flag") is legal here and is not under reduce.

fn d_scan2[D](a: Tile[D], b: Tile[D], a_identity: Float, b_identity: Float, body: fn(Tile[D], Tile[D], Tile[D], Tile[D]) -> (Tile[D], Tile[D]) !Dev, dim: Int = -1, reverse: Bool = false) -> (Tile[D], Tile[D]) !Dev

d_scan over TWO tiles at once, both of one format: the shape a first-order linear recurrence has, where a step's coefficient and its addend must travel through one fold together. body takes the accumulator of each and then the element of each, and answers the new accumulator of each.

The dialect's own description says scan "is restricted to only support single tile input". Its verifier does not say so, tileiras assembles a two-operand scan, and the DEVICE answers one correctly: measured on sm_86 with the affine composition (c, v) . (c', v') = (c * c', c' * v + v'), which is h[t] = a[t] * h[t-1] + x[t] (docs/tile-backend-design.md 6.6). The sentence in the dialect's prose is the only thing that says otherwise and nothing enforces it, so this is recorded as a measurement and the kernels that need it name it.

fn d_if(cond: Tile[Bool], on_true: fn() -> Unit !Dev, on_false: fn() -> Unit !Dev) -> Unit !Dev

The if over the rank-0 mask cond: on_true and on_false each run on the host, and what each issues is one of the operation's two regions. A carry either branch sets is a result of the if (Carry): each branch starts from the tile the carry held before the if, a branch that does not set it answers that tile, and after the if the carry holds the branch's answer the mask chose. Both must keep the carry's format and shape. A wider cond is refused when the kernel records.

Both regions are always written, as the dialect's reader requires (an if with one region is refused, measured). Neither may load or store.

fn extract[D](t: Tile[D], to: List[Int], indices: List[Idx]) -> Tile[D] !Dev

The subtile of t of shape to whose slice number along dimension k is indices[k]. to must divide t's shape evenly in every dimension, and there are shape[k] / to[k] slices along dimension k; the indices are SLICE numbers and not element offsets, which is the dialect's own wording and the one thing about extract that is easy to read backwards. An index outside its range is undefined behaviour, not an error. to is the one shape this takes: no operand carries it.

fn insert[D](s: Tile[D], t: Tile[D], indices: List[Idx]) -> Tile[D] !Dev

t with its subtile whose slice number along dimension k is indices[k] replaced by s: Tile IR's insert (knife T16, bytecode 13.4), extract's other direction. s's shape must divide t's evenly in every dimension and the indices are SLICE numbers, exactly as extract's are, so extract(insert(s, t, i), shape(s), i) is s and insert(extract(t, shape(s), i), t, i) is t. Every lane outside the named subtile is t's own. An index outside its range is undefined behaviour, not an error.

fn cat[D](a: Tile[D], b: Tile[D], dim: Int) -> Tile[D] !Dev

a and b joined along dimension dim: the two agree in every other dimension, and the answer's dim is the sum of theirs.

fn permute_tile[D](t: Tile[D], perm: List[Int]) -> Tile[D] !Dev

t with its dimensions reordered: dimension k of the answer is dimension perm[k] of t, so permute_tile(t, [1, 0, 2]) exchanges the first two axes of a rank-3 tile. perm is a permutation of 0 .. rank - 1 and t has rank two or more.

NOT permute above, which takes strides and answers strides: that one is a LAYOUT for a load to read an array through, and no instruction comes of it. This one is the dialect's own permute (opcode 0x53) and moves the elements of a tile that is already in registers. Before knife T2 the layout was the only thing here that could be called a transpose, and the comment on permute said the opcode had no client; it has one now, and the two remain different operations at different times.

fn transpose[D](t: Tile[D]) -> Tile[D] !Dev

The rank-2 t with its two dimensions exchanged: permute_tile(t, [1, 0]), the same instruction (tileir 0.10.0). A tile of another rank is refused when the kernel records, by the check every permutation passes ("a permutation of a 3-dimensional tile has that many entries"); permute_tile with the whole permutation is what a higher rank writes.

fn num_blocks(axis: Int) -> Idx !Dev

The number of tile blocks along axis (0, 1 or 2): the extent of the grid this kernel was launched on, which is block_id's companion. A grid smaller than the work is covered by stepping through it, and this is the step.

fn scalar[D: ScalarDtype](p: Param[D]) -> Tile[D] !Dev

Parameter p of the entry, read as a rank-0 tile: the value the launch passed by value (a ByValue parameter of trace_kernel). Nothing is loaded: the dialect's entry parameter IS the tile, so this issues no instruction and a kernel that reads it twice reads one SSA value. Arithmetic with it follows the rank-0 rule, so mul(t, scalar(scale)) scales every lane of t, and idx_of(scalar(n)) is a runtime extent or loop bound. A parameter that is a buffer, or one declared in another format, is refused when the kernel records (tileir 0.12.0).

fn extent_of[D](p: Param[D], dim: Int) -> Idx !Dev

How many elements p's tensor has along dimension dim, as an index: the number its cells declare, or for a DYN_DIM dimension the grid's blocks along the axis it follows times its tile, which is what such an extent is (prog.cells). p must have cells (an In or an Out of trace1 to trace5; a dynamic dimension that follows no grid axis is refused by the markers already) (tileir 0.9.0).

fn blocks_of[D](p: Param[D], dim: Int) -> Idx !Dev

How many cells p is cut into along dimension dim: the extent over the tile, rounded up, or for a DYN_DIM dimension the grid's blocks along the axis it follows. The upper bound of a loop over a dimension of an In that follows no grid axis: d_range(idx_const(0), blocks_of(k, 0), idx_const(1)) { j => .. load_at(k, [j]) .. } (tileir 0.9.0).

fn d_fork2(a: fn() -> Unit !Dev, b: fn() -> Unit !Dev) -> Unit !Dev

a and b issued as two INDEPENDENT chains of memory operations, joined afterwards: both start from the ordering token as it stands here, and everything after this call is ordered after both. Neither is ordered against the other, so the two must not touch the same element, and the recording refuses two chains it cannot show to be disjoint.

This is the only client of t_tok_get / t_tok_set / t_tok_join, and the only thing in this package that makes the token graph a graph. A body that just issues its stores in order gets the line it already had.

fn ptrs[D](p: Param[D], shape: List[Int]) -> Ptrs[D] !Dev

The tile of shape every lane of which holds p's own address: the reshape and broadcast every address ladder opens with, as a value a body can name. ptr_offset is what moves it. shape is the geometry of the pointer tile, which nothing else carries.

fn ptr_offset[D](ps: Ptrs[D], offs: Tile[I32]) -> Ptrs[D] !Dev

ps advanced by offs, in ELEMENTS of the pointee format: lane i of the answer is lane i of ps plus offs[i] elements. The same offset instruction a strided load builds, with the body's own tile where its ladder would be.

fn ptr_to_int[D](ps: Ptrs[D]) -> Tile[Int] !Dev

ps as a tile of i64 addresses. The values are to be read UNSIGNED; the inverse is int_to_ptr.

fn int_to_ptr[D](v: Tile[Int], d: Dtype[D]) -> Ptrs[D] !Dev

A tile of i64 addresses as pointers to d. The inverse of ptr_to_int, and the point of the pair is that an address can be COMPUTED: a kernel that wants a byte offset no offset can express takes the address apart here and puts it back. A write through what this answers cannot be followed back to a parameter, so it is refused in a kernel with an In or an Out parameter (Shared and trace_kernel ones take it).

fn ptr_to_ptr[A, B](ps: Ptrs[A], to_d: Dtype[B]) -> Ptrs[B] !Dev

ps reinterpreted as pointers to to_d: the same addresses, another element format. Nothing is converted and nothing is read; what changes is what a later ptr_offset counts in and what a later load_ptrs reads. The dialect keeps this apart from int_to_ptr and ptr_to_int so that a compiler can still follow pointer provenance across it.

fn d_global[D](sym: String, d: Dtype[D], values: List[Float], align: Int = 0, visibility: Visibility = Public, constant: Bool = false) -> Unit !Dev

Declare a statically allocated global named sym holding values, one element of d each. The allocation lives as long as the loaded module does and is initialized at module load time, so what one launch writes the next launch of the same module reads; scripts/tile-gpu-diff/global_diff.dawn is the measurement of that sentence.

This is a MODULE-LEVEL declaration and not an instruction: it answers nothing, and issuing it inside a d_range, a d_loop, a d_if or a reduction is refused. Declare each global once, before the get_global that names it.

The three remaining parameters are the three fields of the Global section's record past the values, at the dialect's own defaults (public, writable, alignment 0), and any combination of them may be given:

• align: the alignment in bytes. The record has carried the number since 13.1, so this is the one field a corpus can give two different values in one module.
• visibility: Private says the symbol is not part of the module's interface, so a host that has the module cannot look it up. That sentence is the dialect's and not this machine's, and knife TG measured the difference. What the value DOES reach is the cubin: tileiras gives a private global a LOCAL ELF binding and a public one a GLOBAL binding, which scripts/tile-golden holds on global_syms. What it does not reach is any answer: the CUDA driver's cuModuleGetGlobal (std/gpu's module_global) answers a LOCAL global as readily as a GLOBAL one, the host reads and writes both, and the next launch reads the write (scripts/tile-gpu-diff/sym_diff.dawn). So this buys the byte, layer 1 and a binding, and scripts/tileir-features/attrs.txt carries the exemption visibility-not-in-the-lookup on the row.
• constant: the global is read-only, which is a promise to the compiler and not a guard. The dialect does not say a store through a pointer got from a constant global is diagnosed, so this tree does not store through one. The flag reaches one byte of the bytecode and nothing further: with the writer's flag forced to 0 the sm_86 cubin tileiras writes is byte for byte the same one (measured, knife TG; scripts/tile-golden's constant-flag-as-mutable is the gate). The exemption on the row is constant-not-in-the-cubin.

fn visibility_name(v: Visibility) -> String

The dialect's spelling of a symbol visibility.

fn global_ptrs[D](sym: String, d: Dtype[D], shape: List[Int]) -> Ptrs[D] !Dev

The tile of shape every lane of which holds the address of the global named sym: get_global answers a rank-0 tile<ptr<D>>, and this is the same reshape and broadcast ptrs performs on a parameter. The element at position x of the global is ptr_offset's x from here, which is what the dialect means by "the initial values are stored in memory in linear order".

fn alloca_ptrs[D](d: Dtype[D], num_elem: Int, align: Int, shape: List[Int], shared: Bool = false) -> Ptrs[D] !Dev

The tile of shape every lane of which holds the address of a fresh automatic allocation of num_elem elements of d, aligned to align bytes (knife T10). alloca answers a rank-0 tile<ptr<D>>, so this is the same reshape and broadcast global_ptrs performs, and ptr_offset walks the elements from here.

The allocation is SCRATCH: the dialect limits its lifetime to the block the operation is in, so what one launch writes is not there for the next, and each alloca in a block answers a region of its own. By default the address is private to this tile thread; shared sets the dialect's global unit attribute, and then the address may be read and written by other tile threads as well as this one. The lifetime is the same and so is everything else; what changes is one bit of the operation's flags word.

align must be a non-zero power of two and at least the natural size of d, which the dialect's verifier requires and the recording handler refuses ahead of it.

fn tensor_view[D](p: Param[D], base: Idx, shape: List[Int], strides: List[Int]) -> TensorView[D] !Dev

The tensor_view over p's buffer starting base ELEMENTS in, with shape elements along each dimension and strides elements between two neighbours along it (knife T11), in p's format. This is the whole of the addressing: a view says where a tensor is and how it is laid out, and nothing about the tiles it will be read in. A kernel whose parameter's cells describe the tile it reads wants load_cell, which builds the view itself; this is the escape hatch for every other view.

base is the same rank-0 index a strided load takes, and it becomes an offset on the parameter's own pointer, so a view may start anywhere in the buffer. Every stride is STRICTLY POSITIVE, which is where the two addressing schemes part: a strided load reads a broadcast with a stride of zero and make_tensor_view refuses one (docs/tile-backend-design.md 6.14).

fn tensor_view_dyn[D](p: Param[D], base: Idx, shape: List[Dim], strides: List[Dim]) -> TensorView[D] !Dev

tensor_view with dimensions the kernel computes (knife T12). A Fixed entry is an extent (or a stride) the TYPE states; a Dyn entry prints ? there and hands make_tensor_view the value as an operand, so one cubin serves every tensor of that rank rather than one shape.

Everything else is tensor_view's: the base is an element offset on the parameter's own pointer, and a dynamic extent or stride is held to the same rule a static one is (strictly positive), except that the device rather than the recorder is what holds it.

fn idx_of(t: Tile[I32]) -> Idx

A rank-0 Tile[I32] as an index. Idx and a rank-0 Tile[I32] are ONE lowered type (tile<i32>) with two surface names, and this and idx_as_tile are the only places a kernel crosses between them.

It is here for knife T12: a dynamic extent has to come from somewhere, and the general somewhere is a buffer. The rank is not checkable here (a handle carries no shape at this layer), so the recording holds it instead: an index of rank one or more is refused where it is used (a loop bound, a cell index), and every dynamic operand of a make_tensor_view must be a rank-0 i32 tile and is refused by name in the lowering if it is not.

fn idx_as_tile(i: Idx) -> Tile[I32]

idx_of the other way: an index as the rank-0 i32 tile every element-wise operation takes, so a shape the device answered can be written out, or compared against a tile (the rank-0 rule widens it). Until the batch's PR-3 this was idx_as_scalar.

fn partition_view[D](tv: TensorView[D], tile_shape: List[Int], dim_map: List[Int], padding: Padding) -> GridView[D] !Dev

tv cut into a grid of tile_shape tiles, in tv's format. dim_map[k] says which dimension of the tensor view tile dimension k runs along, so the identity map is the plain partition and a permutation of it transposes the tile; padding says what an out-of-bounds lane of a load_view reads.

The INDEX SPACE of the answer has one dimension per tile dimension, and its extent along k is the tensor view's extent along dim_map[k] divided by tile_shape[k], rounded up: a tile may run past the end of the tensor and a tile index may not. view_grid computes it.

fn strided_view[D](tv: TensorView[D], tile_shape: List[Int], traversal: List[Int], dim_map: List[Int], padding: Padding) -> GridView[D] !Dev

tv cut into a grid of tile_shape tiles that ADVANCES by traversal (knife T13): index k of the grid covers traversal[k] elements along the tensor dimension dim_map[k] and reads tile_shape[k] of them, so the tiles overlap when the traversal is the shorter and skip when it is the longer. traversal == tile_shape is partition_view written the long way, and the dialect's own documentation says so.

Every traversal stride is STRICTLY POSITIVE and the tile dimensions are powers of two, which is StridedViewType::verify and is what the recording handler holds. The index space along k is ceil(shape[dim_map[k]] / traversal[k]): strided_grid computes it, and partial tiles at the far end are IN it (the dialect includes them).

fn gather_scatter_view[D](tv: TensorView[D], tile_shape: List[Int], sparse_dim: Int, padding: Padding) -> GatherScatterView[D] !Dev

tv read a row at a time along sparse_dim (knife T13). The index at that dimension is a 1D tile of row numbers rather than a tile index, so one load collects tile_shape[sparse_dim] rows chosen while the kernel runs and stacks them into the tile; every other dimension is a grid view's tile index.

There is no dim_map: what the dialect holds against the tensor view instead is that sparse_dim is one of its dimensions.

fn load_view[D](v: GridView[D], indices: List[Idx], inbounds: List[Bool] = []) -> Tile[D] !Dev

The tile of v's grid at indices, one index a tile dimension. In the token chain where every other load is; lanes that fall outside the underlying tensor read the view's padding value.

inbounds (knife T18) is the dialect's inbounds array: true at a dimension PROMISES that no tile this load reads is cut by the tensor's edge along it, so the assembler may drop the bounds handling there. [], the default, promises nothing at any dimension; otherwise it holds one entry per index. The promise is the caller's: the dialect makes a broken one undefined behaviour, and tileiras answers a tile that does reach the edge with an unnamed failed to compile Tile IR program.

fn store_view[D](v: GridView[D], indices: List[Idx], t: Tile[D], inbounds: List[Bool] = []) -> Unit !Dev

t written into the tile of v's grid at indices. In the token chain where every other store is; lanes that fall outside the underlying tensor are MASKED, which is the store's half of the same rule the padding value is the load's half of. A view of an Out parameter is not one of its cells, so the recording refuses this into one (store_cell and store_sub are an Out's writes); a Shared parameter takes it. inbounds is load_view's promise, for the lanes this store would mask.

fn load_gather[D](v: GatherScatterView[D], indices: List[GatherIdx]) -> Tile[D] !Dev

The tile v gathers at indices: the rows named by the Rows entry at sparse_dim, each cut to the tile at the At indices of the other dimensions. In the token chain where every other load is.

fn store_gather[D](v: GatherScatterView[D], indices: List[GatherIdx], t: Tile[D]) -> Unit !Dev

t scattered back the way load_gather collects: row k of the tile goes to the row the Rows entry names at k. In the token chain where every other store is.

fn atomic_red_view[D](v: GridView[D], indices: List[Idx], mode: String, scope: String, t: Tile[D]) -> Unit !Dev

t reduced into the tile of v's grid at indices, one element at a time and ATOMICALLY: atomic_red_view_tko (knife T13). Unlike atomic_rmw, it answers nothing at all, so two tile blocks may reduce into one tile of the grid and the result does not depend on the order they arrive in.

mode is one of the nine the operation takes (and, or, xor, add, addf, max, min, umax, umin; xchg is refused because a reduction that returns nothing cannot exchange). scope is tl_blk or device; the ordering is always relaxed, which is the only one the dialect allows here. A view carrying a padding_value is refused.

fn tensor_dim[D](tv: TensorView[D], dim: Int) -> Idx !Dev

The extent of tv's tensor along dimension dim, asked of the DEVICE: get_tensor_shape (knife T12). The dialect's operation answers every dimension at once, the way get_tile_block_id answers every axis, and this names one of them; the lowering issues the whole operation and binds this handle to the dim-th result.

For a static dimension the answer is the number already in the type, and for a ? one it is the operand make_tensor_view was handed. That is what makes it worth asking: a kernel over a dynamic tensor has no other way to know where the tensor ends.

fn index_space_dim[D](v: GridView[D], dim: Int) -> Idx !Dev

The extent of v's INDEX SPACE along tile dimension dim, asked of the device: get_index_space_shape (knife T12). The index space is the grid of tiles the partition view cuts its tensor into, so this is ceil(shape[dim_map[dim]] / tile_shape[dim]) computed where the tensor's extent is known, which for a dynamic tensor is only on the device. view_grid is the same number computed on the host from a static shape.

fn view_grid(shape: List[Int], tile_shape: List[Int], dim_map: List[Int]) -> List[Int]

The extent of a partition view's index space, as the grid a kernel over it is launched on: ceil(shape[dim_map[k]] / tile_shape[k]) along tile dimension k. The host needs the same number, so this is the one place it is computed.

fn strided_grid(shape: List[Int], traversal: List[Int], dim_map: List[Int]) -> List[Int]

The same for a STRIDED view, where the grid steps by the traversal and not by the tile: ceil(shape[dim_map[k]] / traversal[k]). Partial tiles at the far end of the tensor are in the index space, which is why this rounds up exactly as view_grid does.

fn load_ptrs[D](ps: Ptrs[D]) -> Tile[D] !Dev

The tile the pointers in ps point at, unmasked, shaped like ps. In the token chain where load is, and ordered against every other memory operation the same way.

fn store_ptrs[D](ps: Ptrs[D], v: Tile[D]) -> Unit !Dev

Write v through the pointers in ps, unmasked: load_ptrs's other direction, and the same store_ptr_tko a store through a parameter emits. In the token chain where store is. A write through pointers into an Out is refused (an Out's writes are its cells'), as is one the recording cannot follow to a parameter while any parameter is an In or an Out.

fn i64_const(value: Int) -> Tile[Int] !Dev

The rank-0 i64 tile holding value. What an address is offset by, in bytes; the i64 spelling of consti.

fn add_i64(a: Tile[Int], b: Tile[Int]) -> Tile[Int] !Dev

The element-wise i64 sum a + b, the spelling address arithmetic uses. addi over i64 tiles is the same instruction and the same bytes.

fn d_assert(cond: Tile[Bool], message: String) -> Unit !Dev

Stop the kernel with message when any lane of cond is false (knife T6). Every zero lane prints the message and its index, and the host is told that an assertion fired; the dialect leaves the tile block free to keep running, so what a failing kernel wrote before the assert is not defined away.

The failure reaches the HOST, not this program: a launch whose assertion fired makes the CUDA context report an error, and that error is sticky -- every later launch in the same process fails too. That is why scripts/tile-gpu-diff runs the failing corpus in a process of its own, and it is the reason "this launch should fail" is a judgement shape and not just another comparison.

fn d_assume[D](t: Tile[D], pred: AssumePred) -> Tile[D] !Dev

t with pred attached: the same value, and a claim about it the compiler may use. The claim is NOT checked, here or on the device, so a false one is undefined behaviour (AssumePred says where the dialect says so).

fn assume_div_by[D](t: Tile[D], divisor: Int) -> Tile[D] !Dev

t, assumed to hold multiples of divisor in every lane.

fn assume_same_elements[D](t: Tile[D], groups: List[Int]) -> Tile[D] !Dev

t, assumed to repeat one value in every group of groups[k] along dimension k.

fn assume_in_range[D](t: Tile[D], lb: Option[Int], ub: Option[Int]) -> Tile[D] !Dev

t, assumed to hold values between lb and ub inclusive, read signed. Either bound may be None.

fn d_print[D](fmt: String, args: List[Tile[D]]) -> Unit !Dev

Print fmt with args interleaved, C printf style: one format expression (a % and its conversion) per argument, and the dialect refuses a count that disagrees (PrintTkoOp's verifier counts them the way format_args in prog.dawn does). Every argument is of one format, the first's.

RANK-0 ARGUMENTS ONLY, and that is a decision about what can be JUDGED rather than a limit of the operation. The dialect takes a tile of any shape and says in as many words that prints "are not guaranteed to be atomic" and that simultaneous ones "may be interleaved", so the bytes a wide tile puts on standard output are not a value a test can compare. One rank-0 tile printed by one tile block is one line, and scripts/tile-gpu-diff compares it byte for byte. The effect operation underneath carries a shape, so nothing here has to change if a later knife finds a judgement for the wide case.

TOKEN ORDERED, which is what the _tko in the opcode's name means: this takes the ordering token as it stands and answers the next one, so a print sits in the same chain as the loads and stores around it. Without that the compiler is free to move it (Ops.td says so).

fn memory_fence_alias() -> Unit !Dev

The alias proxy fence (memory_fence_alias_tko, bytecode 13.4). Along the token chain it makes every earlier access that reaches the same physical memory through a different virtual alias complete before anything after it. It answers nothing: like d_print it takes the ordering token as it stands and puts the next in its place, so it sits in the chain with the loads and stores around it. Refused in a reduce, scan or if region for the reason d_print is.

fn gdc_launch_dependents() -> Unit !Dev

Tell the grid's dependent kernel it may be scheduled (gdc_launch_dependents_tko, Programmatic Dependent Launch, bytecode 13.4). Grid-wide: every block must execute it, or finish, before the dependent is scheduled. It controls scheduling only and orders no memory. On a target below sm_90 it is a no-op, and so it is on any target when the kernel was not launched with the programmatic stream serialization attribute, which std/gpu's launch does not set.

fn gdc_wait() -> Unit !Dev

Wait for the predecessor kernel to complete (gdc_wait_tko): every store it made is visible after this. The pair of gdc_launch_dependents, with the same two no-op conditions.

fn load_cell[D](p: Param[D]) -> Tile[D] !Dev

This block's cell of p, read through a view of p's cells: the tile of p's marker, at the block ids of the grid axes its dimensions follow. Lanes past the extent read the marker's padding value, so cells([1000], [1024], pad: PadNegInf) reads a whole row with no mask. Every dimension must follow a grid axis; load_at names the cells of the ones that do not. p must be an In or an Out cut into cells.

fn load_at[D](p: Param[D], free: List[Idx]) -> Tile[D] !Dev

load_cell for a parameter some of whose dimensions follow no grid axis (FREE_AXIS in its along): free names the cell along each of those, in dimension order, and the others are the block's own, as for load_cell. The left operand of a matrix product cut cells([M, K], [TM, TK], along: [0, FREE_AXIS]) is read load_at(a, [k]) inside the loop over k.

fn store_cell[D](p: Param[D], t: Tile[D]) -> Unit !Dev

t written into this block's cell of p, an Out cut into cells: the same view load_cell reads, at the block's own index. Lanes past the extent are not written. No format marker and no shape: t must be the tile of p's marker, which the recording holds it to, and a rank-0 t is widened to it (every lane of the cell the same value).

This and store_sub are the only writes an Out takes (the batch's D-7): a store, a scatter, an atomic or a store_view into an Out is refused while the kernel records, so the claim "each block writes its own cell and nothing else" is not a convention but the only thing a kernel can say.

fn store_sub[D](p: Param[D], at: List[Int], t: Tile[D]) -> Unit !Dev

t written into piece at of this block's cell of p, an Out: the cell is cut into pieces shaped like t (which must divide the cell's tile in every dimension), and at names one by its number along each dimension, host numbers each below tile / shape(t). What a block that writes its cell in several parts writes with: a slab per segment, the two halves of a rotated row. The piece can never leave the block's cell, which is the recording's claim and the reason at is a host number.

fn zeros[D](p: Param[D]) -> Tile[D] !Dev

A tile shaped like one cell of p, in p's format, every lane zero: the accumulator a block starts from before it stores its cell.

fn fill[D](p: Param[D], v: Float) -> Tile[D] !Dev

A tile shaped like one cell of p, in p's format, every lane v. For an integer format v must be a whole number.

fn retile[D](p: Param[D], extent: List[Int], tile: List[Int], pad: Padding = PadZero) -> GridView[D] !Dev

p's tensor, of extent elements row-major, cut into tiles of tile and padded with pad, as a grid view load_view reads at any index: the escape hatch for a parameter a kernel reads two ways (the batch's C4), when one marker can say only one of them. Read only: an Out's writes are its cells. An In's view carries the alignment claim its cells carry.

fn reduce_sum[D](t: Tile[D], dim: Int = -1, keepdims: Bool = false) -> Tile[D] !Dev

The sum of t along dimension dim (the last by default, -k the k-th from the end): the tile with that dimension dropped, or kept with length 1 when keepdims is true, so a [BQ, BK] tile's row sums are [BQ] or [BQ, 1]. The identity is the format's own (0), so a fold seeded wrongly cannot be written. The order of the fold is not specified, as d_reduce's is not.

A rank-1 tile reduces to a rank-0 one, which every element-wise operation of a wider shape widens on its own (the recording's rank-0 rule, 6.25): sub(t, reduce_max(t)) needs no broadcast.

fn reduce_max[D](t: Tile[D], dim: Int = -1, keepdims: Bool = false) -> Tile[D] !Dev

reduce_sum for the largest element; the identity is -inf for a float format and the least value for an integer one.

fn reduce_min[D](t: Tile[D], dim: Int = -1, keepdims: Bool = false) -> Tile[D] !Dev

reduce_sum for the smallest element; the identity is +inf for a float format and the greatest value for an integer one.

fn scan_sum[D](t: Tile[D], dim: Int = -1, reverse: Bool = false) -> Tile[D] !Dev

The running sum of t along dim (the last by default): the tile of t's shape whose element i along dim is the sum of elements 0 to i, or i to the end when reverse is true.

fn broadcast[D](t: Tile[D], shape: List[Int]) -> Tile[D] !Dev

t widened to shape: every dimension of length 1 in t takes its length in shape, and every other dimension must already have it. The rank does not change; a rank-0 t widens to any shape (a constant becomes one constant of that shape). This is the explicit half of the rank-0 rule: a [BQ, 1] row statistic is broadcast(m, [BQ, BK]) before it meets a [BQ, BK] tile, and a loop's starting value is broadcast(f_const(F64, 0.0), [64, 64]), which full writes for you.

fn reshape[D](t: Tile[D], shape: List[Int]) -> Tile[D] !Dev

t with its lanes regrouped into shape, in row-major order: the lanes are read off t row by row and laid into shape row by row, which is Tile IR's reshape. shape holds as many lanes as t and every dimension is a power of two; the recording refuses anything else. This is how a tile meets a cell of another rank: a [T, T] product is stored into a [1, T, T] cell as store_cell(o, reshape(acc, [1, T, T])). Adding or dropping a dimension of length 1 moves no lane; regrouping the others may cost data movement the compiler chooses.

type Elem = Num(name: String) | PtrTo(name: String)

The element type of a tile: a number format by its Tile IR name (i32, f32, f64), or a pointer to one.

type Ty = Token | Tile(shape: List[Int], elem: Elem) | TensorView(elem: Elem, shape: List[Int], strides: List[Int]) | PartitionView(tile_shape: List[Int], elem: Elem, shape: List[Int], strides: List[Int], dim_map: List[Int], padding: Padding) | StridedView(tile_shape: List[Int], traversal: List[Int], elem: Elem, shape: List[Int], strides: List[Int], dim_map: List[Int], padding: Padding) | GatherScatterView(tile_shape: List[Int], elem: Elem, shape: List[Int], strides: List[Int], sparse_dim: Int, padding: Padding)

A Tile IR value type: the ordering token, a tile of elem with the given shape ([] is the scalar tile<elem>, [n] is tile<n x elem>), or one of the two view types knife T11 added.

TensorView is !cuda_tile.tensor_view<shape x elem, strides=[..]>: a reference to a region of global memory, its extent along each dimension and the number of ELEMENTS one step along that dimension costs. An entry of either list may be DYN_DIM, which is the dialect's ?: the extent (or the stride) is not in the type at all and MakeTensorView carries it in an operand instead (knife T12). The MARKER is what travels through this module; the value the writer puts on the wire for it (ShapedType::kDynamic) lives in bytecode.dawn.

PartitionView is !cuda_tile.partition_view<tile=(..), tensor_view<..>, dim_map=[..]>, and it is FLAT rather than holding a Ty: its elem, shape and strides are the tensor view's own, and tensor_view_of builds that type back when the writer needs to intern it. A nested Ty would say the same thing and make every match arm carry a case that cannot occur (the dialect's parameter is a TensorViewType and nothing else).

StridedView and GatherScatterView are knife T13's two, and both are flat for the same reason. A strided view is a partition view whose TRAVERSAL is parametric: index k covers traversal[k] elements rather than tile_shape[k], so consecutive tiles may overlap or skip. A gather/scatter view has no dim_map and a sparse_dim instead: the index at that dimension is a 1D tile of ROW NUMBERS rather than a tile index, so one load collects rows chosen at run time.

padding is the one OPTIONAL parameter any of the three has: NoPadding is the dialect's absent padding_value, and out-of-bounds loads then read unspecified values.

type Ref = Arg(pos: Int) | Val(id: Int)

An operand: an entry parameter by position, or the value an earlier instruction defined, by its number.

type Instr = MakeTok(dst: Int) | BlockIds(x: Int, y: Int, z: Int) | ConstInt(dst: Int, ty: Ty, value: Int) | ConstFloat(dst: Int, ty: Ty, value: Float) | MulInt(dst: Int, ty: Ty, lhs: Ref, rhs: Ref) | AddInt(dst: Int, ty: Ty, lhs: Ref, rhs: Ref) | Reshape(dst: Int, src: Ref, from: Ty, to: Ty) | Broadcast(dst: Int, src: Ref, from: Ty, to: Ty) | IotaTile(dst: Int, ty: Ty) | Offset(dst: Int, ptrs: Ref, offs: Ref, ptr_ty: Ty, off_ty: Ty) | LoadPtr(dst: Int, tok_out: Int, ptrs: Ref, mask: Option[Ref], pad: Option[Ref], tok_in: Ref, ptr_ty: Ty, mask_ty: Ty, val_ty: Ty, hints: List[(String, List[(String, Int)])]) | StorePtr(tok_out: Int, ptrs: Ref, value: Ref, mask: Option[Ref], tok_in: Ref, ptr_ty: Ty, mask_ty: Ty, val_ty: Ty, hints: List[(String, List[(String, Int)])]) | AtomicRMWPtr(dst: Int, tok_out: Int, ptrs: Ref, mode: String, arg: Ref, mask: Option[Ref], tok_in: Ref, ptr_ty: Ty, mask_ty: Ty, val_ty: Ty) | AtomicCASPtr(dst: Int, tok_out: Int, ptrs: Ref, cmp: Ref, val: Ref, mask: Option[Ref], tok_in: Ref, ptr_ty: Ty, mask_ty: Ty, val_ty: Ty) | FloatUn(dst: Int, op: String, ty: Ty, src: Ref) | FloatBin(dst: Int, op: String, ty: Ty, lhs: Ref, rhs: Ref) | FloatFma(dst: Int, ty: Ty, lhs: Ref, rhs: Ref, acc: Ref) | FloatMma(dst: Int, ty: Ty, lhs_ty: Ty, rhs_ty: Ty, lhs: Ref, rhs: Ref, acc: Ref) | FloatMmaScaled(dst: Int, ty: Ty, lhs_ty: Ty, rhs_ty: Ty, scale_ty: Ty, rhs_scale_ty: Ty, lhs: Ref, rhs: Ref, acc: Ref, lhs_scale: Ref, rhs_scale: Ref) | IntMma(dst: Int, ty: Ty, lhs_ty: Ty, rhs_ty: Ty, lhs: Ref, rhs: Ref, acc: Ref) | CmpFloat(dst: Int, pred: String, ty: Ty, res_ty: Ty, lhs: Ref, rhs: Ref) | CmpInt(dst: Int, pred: String, ty: Ty, res_ty: Ty, lhs: Ref, rhs: Ref) | IntUn(dst: Int, op: String, ty: Ty, src: Ref) | IntBin(dst: Int, op: String, ty: Ty, lhs: Ref, rhs: Ref) | Cast(dst: Int, op: String, from: Ty, to: Ty, src: Ref) | SelectTile(dst: Int, cond_ty: Ty, ty: Ty, cond: Ref, lhs: Ref, rhs: Ref) | ForLoop(results: List[Int], tys: List[Ty], iv: Int, lower: Ref, upper: Ref, step: Ref, unsigned: Bool, inits: List[Ref], carried: List[Int], body: List[Instr]) | WhileLoop(results: List[Int], tys: List[Ty], inits: List[Ref], carried: List[Int], body: List[Instr]) | BreakVals(values: List[Ref], tys: List[Ty]) | Cont(values: List[Ref], tys: List[Ty]) | ReduceTile(results: List[Int], tys: List[Ty], dim: Int, identities: List[RedId], operands: List[Ref], operand_tys: List[Ty], args: List[Int], body: List[Instr]) | ScanTile(results: List[Int], tys: List[Ty], dim: Int, reverse: Bool, identities: List[RedId], operands: List[Ref], operand_tys: List[Ty], args: List[Int], body: List[Instr]) | IfElse(results: List[Int], tys: List[Ty], cond: Ref, cond_ty: Ty, on_true: List[Instr], on_false: List[Instr]) | YieldVals(values: List[Ref], tys: List[Ty]) | ExtractTile(dst: Int, src: Ref, from: Ty, to: Ty, indices: List[Ref]) | InsertTile(dst: Int, src: Ref, dest: Ref, sub: Ty, to: Ty, indices: List[Ref]) | FloatPowI(dst: Int, ty: Ty, exp_ty: Ty, base: Ref, exp: Ref) | CatTile(dst: Int, dim: Int, lhs: Ref, rhs: Ref, lhs_ty: Ty, rhs_ty: Ty, to: Ty) | PermuteTile(dst: Int, src: Ref, perm: List[Int], from: Ty, to: Ty) | BlockCounts(x: Int, y: Int, z: Int) | JoinTokens(dst: Int, toks: List[Ref]) | PtrToIntTile(dst: Int, src: Ref, from: Ty, to: Ty) | IntToPtrTile(dst: Int, src: Ref, from: Ty, to: Ty) | PtrToPtrTile(dst: Int, src: Ref, from: Ty, to: Ty) | AssertTile(cond: Ref, cond_ty: Ty, message: String) | AssumeTile(dst: Int, src: Ref, pred: AssumePred, ty: Ty) | PrintTile(tok_out: Int, fmt: String, args: List[Ref], arg_ty: Ty, tok_in: Ref) | TokenOnlyTile(tok_out: Int, op: String, tok_in: Ref) | GetGlobalPtr(dst: Int, sym: String, ty: Ty) | AllocaPtr(dst: Int, num_elem: Int, align: Int, shared: Bool, ty: Ty) | MakeTensorView(dst: Int, base: Ref, base_ty: Ty, ty: Ty, dyn_shape: List[Ref], dyn_strides: List[Ref], dyn_ty: Ty) | MakePartitionView(dst: Int, src: Ref, src_ty: Ty, ty: Ty) | MakeStridedView(dst: Int, src: Ref, src_ty: Ty, ty: Ty) | MakeGatherScatterView(dst: Int, src: Ref, src_ty: Ty, ty: Ty) | GetTensorShape(dsts: List[Int], src: Ref, src_ty: Ty, res_ty: Ty) | GetIndexSpaceShape(dsts: List[Int], src: Ref, src_ty: Ty, res_ty: Ty) | LoadViewTile(dst: Int, tok_out: Int, view: Ref, view_ty: Ty, indices: List[Ref], idx_tys: List[Ty], ty: Ty, tok_in: Ref, inbounds: List[Bool]) | StoreViewTile(tok_out: Int, view: Ref, view_ty: Ty, value: Ref, val_ty: Ty, indices: List[Ref], idx_tys: List[Ty], tok_in: Ref, inbounds: List[Bool]) | AtomicRedViewTile(tok_out: Int, view: Ref, view_ty: Ty, value: Ref, val_ty: Ty, indices: List[Ref], idx_tys: List[Ty], mode: String, scope: String, tok_in: Ref) | Ret

One Tile IR instruction. Every dst / tok_out / x / y / z is a value number this instruction defines; every Ref refers to a parameter or an earlier definition. The types are the ones the operation's assembly and bytecode both state: a result type on every value-producing instruction, plus the operand types the assembly prints (from / to on the shape operations, ptr_ty / off_ty / val_ty on the memory ones).

A memory instruction carries the mask and the padding value it was given, None standing for the operand the assembly and the bytecode both leave out; mask_ty is the i1 tile type both spell when there is one. A comparison carries the type of its operands and the i1 type of its result, as its assembly prints both.

ScanTile is the prefix scan, ReduceTile with one attribute more (reverse) and results that keep every dimension: its tys are its operand_tys, where a reduction's drop dim.

ReduceTile is the reduction: results are its rank-0 results, typed tys; operands the tiles it folds, typed operand_tys; identities one per operand; args the 2 * k rank-0 block arguments of its one region (accumulator then element, per operand) and body that region, YieldVals last. IfElse is the two-region conditional: cond is a rank-0 i1, on_true and on_false are its regions and each ends in its own YieldVals, and results are the values it defines after both.

ExtractTile, CatTile and PermuteTile are the shape operations (knife T2). ExtractTile's indices are the SLICE numbers along each dimension of from, one per dimension and each a rank-0 i32 operand, not element offsets; CatTile's dim is the dimension the two operands are joined along; PermuteTile's perm says that dimension k of the answer is dimension perm[k] of the source. InsertTile is ExtractTile's other direction (knife T16): src, typed sub, replaces the subtile of dest its indices name, and the answer has dest's type to; the indices are slice numbers along each dimension of to. FloatPowI is fpowi (knife T16): a float base typed ty (which the answer shares) and an integer exponent typed exp_ty, the one float instruction whose two operands differ in format. BlockCounts is the grid's extent, get_tile_block_id's companion and its exact shape. JoinTokens merges two or more ordering tokens into one.

GetGlobalPtr is get_global (knife T7): the address of the module's global named sym, as the rank-0 tile<ptr<D>> the dialect's verifier demands. The spread to a full tile is Reshape and Broadcast after it, the same two the parameter ladder emits, so this instruction has one result and no operands.

FloatMmaScaled is mmaf_scaled (knife T10): FloatMma with two more operands and three formats. The two matrices share lhs_ty /rhs_ty element format, the two scale tiles share scale_ty / rhs_scale_ty's, and ty is the f32 accumulator and result, which is the only output format the dialect allows. It carries no attribute and no optional field, so unlike mmaf it writes no flags word.

AllocaPtr is alloca (knife T10) and has the same shape: one result, no operands, a rank-0 tile<ptr<D>>, spread by the same two instructions. What it carries instead of a symbol is the request -- num_elem elements and an alignment, both inline I64Attrs -- and shared, the global unit attribute, which is a bit of the flags word and not an attribute in the stream.

PtrToIntTile, IntToPtrTile and PtrToPtrTile are the three pointer conversions: a pointer tile as i64, an i64 tile as pointers, and a pointer tile reinterpreted at another element format. All three are pure type changes -- the address is the same number in every one of them -- which is why each carries both its operand type and its result type and nothing else.

TokenOnlyTile is knife T19's: the alias fence and the two gdc_* signals, which take a token, define the next and have no operand or attribute of their own.

AssertTile, AssumeTile and PrintTile are knife T6's three. AssertTile defines nothing and takes the rank-1 or rank-0 i1 tile its cond_ty names. AssumeTile defines a value of the SAME type as its operand, which is what AllTypesMatch<["value", "result"]> in the dialect means, so one ty covers both. PrintTile defines a token and takes one: it is in the memory ordering chain, and its args share one type because the typed surface only builds them that way.

ForLoop is the loop: results are the values it defines after itself, typed tys (one per carried value); iv (a scalar i32) and carried (typed tys again) are the block arguments its body sees; lower, upper, step and inits are operands from outside; unsigned is the loop's unsignedCmp attribute, which decides whether the induction variable is compared against the bound as a signed or an unsigned 32-bit integer. body ends in Cont(values, tys), the values handed to the next iteration.

WhileLoop is the other loop (knife T5): ForLoop without an induction variable or bounds, because it runs until a break. Its body ends in an IfElse with no results whose then-branch is a single BreakVals and whose else-branch is an empty YieldVals, followed by the Cont. BreakVals carries the loop's FINAL values and Cont the next iteration's; both lists are as long as tys.

type Kernel = { name: String, params: List[KParam], globals: List[Global], body: List[Instr], hints: List[(String, List[(String, Int)])], producer: Option[String], debug: Option[DebugInfo] }

A lowered kernel: its entry name, the dtype name of each parameter in order (a ByPtr parameter is a scalar tile<ptr<dtype>>, a ByValue one a rank-0 tile<dtype>), the module's globals in declaration order, and its body, Ret last.

The globals ride through the lowering untouched: they are records of the module and not instructions of the entry, so nothing here numbers them or names them by value. render_kernel prints one line each above the entry and encode_kernel writes them as the Global section.

type Owner = { op: Int, inner: List[List[Owner]] }

Which operation of the program an instruction of Kernel.body came from: op is the operation's number with TileProg.ops read depth first, a region before its body (prog's op_number, MakeToken(0) being 0), and inner is the same for each body of a region instruction, in the order the text prints them (an if has two). A pointer ladder is the operation that FIRST needed it, because that is where it is emitted.

fn tensor_view_of(t: Ty) -> Ty

The tensor_view one of the three views looks into, as its own type. The writer interns it as a separate entry of the Type section, because the view's own record names it by index.