How the workload runs: one layer through Gemmini

Gemmini computes a neural-network layer as a tiled matrix multiply plus a bias. The floorplan is built around this sequence.

Back to the challenge

The operation

The layer is split into 16 by 16 tiles, and the 16 by 16 array computes one tile product at a time.

Cij = Dij + ∑k Aik · Bkj
  • Aik: a tile of activations (8-bit integers)
  • Bkj: a tile of weights (8-bit integers)
  • Dij: the bias tile
  • Cij: the output tile, summed at 32 bits
  • k: the tile index along the shared dimension
A (activations) I rows by K × B (weights) K by J columns = C (outputs) I by J Each square is one 16 by 16 tile. The highlighted tile of C is the sum of the highlighted row of A tiles times the highlighted column of B tiles: four tile products, one per step along K, each one pass of steps 2 to 4 on the array. B tile: preloaded (step 2) A tile: streamed (step 3) C tile: accumulated (step 4)

The sequence

Animation of one matrix multiply moving through Gemmini
One output tile.The array is drawn 8 by 8; the design is 16 by 16.
  1. Move in. Activations and weights are copied from main memory into the scratchpad banks, 128 bits per beat. The bias is copied into the accumulator.
  2. Preload. A weight tile shifts down from the top edge and stays in the processing elements. Each element has two weight registers, so the next tile preloads while the current one computes.
  3. Compute. Activations enter the left edge, each row one cycle behind the row above. Partial sums start at zero at the top and move down, each element adding its weight times its activation.
  4. Accumulate. Output rows leave the bottom edge, one per cycle, and are added into the accumulator on top of the bias. Steps 2 to 4 repeat for every k.
  5. Scale. AccumulatorScale applies the activation function (ReLU, rectified linear unit, or none), multiplies by a scale factor, and clips to 8 bits.
  6. Move out. The 8-bit results go to main memory and become the next layer's activations.

Bus widths

PathWidth
Main memory, in and out128 b
Scratchpad banks to the array logic512 b
Array left edge: activations128 b
Array top edge: weights and partial sums640 b
Array bottom edge: output rows320 b
Array logic to the accumulator1,024 b

For the floorplan

  • Data enters the array on the left and top edges and leaves on the bottom edge.
  • No macro connects directly to another macro; the flat logic between them carries all traffic, and the busiest point is the Scratchpad logic.