GPU
Also known as: gpu cuda opencl metal vulkan webgpu wgsl kernel shader compute parallel graphics card
● Advanced6 minBefore this: Lists: map, filter, sum, Data Types: Float and Float32Spec 15
Mark a function gpu fn and map a list with it. No buffers, no indices, no launches: the compiler and the runtime do the GPU work, and the answers are the CPU's.
gpu fn name(x: Float32, …) -> Float32 {
…
}
const results = items.map(x => name(x, …))gpu fn glow(x: Float32, gain: Float32) -> Float32 {
let v = x
for _ in 0..100 {
v = v * 0.99 + gain
}
v
}
fn main() {
const pixels: [Float32] = [0.0, 0.5, 1.0]
const gain: Float32 = 0.25
const lit = pixels.map(p => glow(p, gain))
print(lit)
}The program ran and printed:
kavach run gpu.kav
[15.849198, 16.032211, 16.215227]- Each item is a parameter: there is no index to get wrong, and nothing to copy by hand.
- This list has three items, so it ran on the CPU: sending it to the GPU would cost more than the work. A long list goes to the GPU, with the same code.
- It runs on NVIDIA, AMD, Intel and Apple GPUs, through Vulkan, DirectX 12 and Metal.
How a GPU map works
Step through both, or press Play: one CPU core takes the items one by one; the GPU pays for a trip over and back, then does every item at once.
One CPU core: one item after another
Item 1 of 8: the core works on it, then moves on. Each item waits its turn.
The GPU: copy over, every item at once, copy back
The list is in the CPU's memory, as every list starts.
A picture of the steps, not of the times: the charts below have the measured times.
What runs where today
| Operation | Runs on | Note |
|---|---|---|
list.map(f) | the GPU, when the list is long and f does enough work | with f a gpu fn, or a lambda of gpu fn calls |
calling a gpu fn directly | the CPU | a gpu fn is also an ordinary function |
sum, min, max of Int32 or UInt32 | the GPU, when the list is already in GPU memory | only the result comes back; a list on the CPU stays there |
filter of a list already in GPU memory | the GPU, when its items are one of the four number types and its lambda fits a gpu fn | the kept items stay there, in their order; not yet for records and tuples |
other filters; sum, min, max of floats | the CPU, on every core | see Loops on All Cores |
sort | the CPU, on every core | see Fast Sorting |
The GPU wins only when each item needs real work: copying a list to the GPU costs more than doubling its numbers. So short lists and light work stay on the CPU, which the runtime decides for you.
One page for each form
What Kavach does for you
The GPU only when it wins
A
mapgoes to the GPU only when the list is long and the work per item big enough; otherwise the CPU does it.
Go deeper the why, for readers who want it
The compiler translates each gpu fn into WGSL, the shading language of WebGPU. The runtime hands the GPU the list's own memory (a [Float32] keeps 4 bytes per item, as a GPU does), runs one GPU thread per item, and reads the results straight into the new list. It compiles each kernel once, and starts opening the GPU as the program starts, so that time hides behind the program's first work.
Lists a map read or made stay in GPU memory for a while, so mapping one again copies nothing over. You never see it: changing such a list copies it first, as changing any list held twice does. That is why the secondmap below is so much faster than the first.
glow, first map — 4 million Float32 items, 1,000 steps each, sent to the GPU · Windows 11 · i7-13700HX · RTX 4050 Laptop · g++ -O3 -march=native -fopenmp for 24 threads
glow, second map — the same list, already in GPU memory · Windows 11 · i7-13700HX · RTX 4050 Laptop · g++ -O3 -march=native -fopenmp for 24 threads
git clone https://github.com/Mithlesh04/kavach && cd kavach
sh scripts/run-benchmarks.sh "" benchmarks/gpuCheck yourself
map a list of 3 numbers with a gpu fn. Where does it run?