GPU: Can I use it on the GPU?

A pipeline of maps

gpu fn glow(x: Float32) -> Float32 {
    x * 0.5 + 0.1
}

gpu fn dim(x: Float32) -> Float32 {
    x * 0.9
}

fn main() {
    const xs: [Float32] = [1.0, 2.0, 3.0]
    const soft = xs.map(p => glow(p)).map(p => dim(p))
    print(soft)
    const lit = xs.map(p => glow(p))
    print(lit.map(p => dim(p)))
    print(soft.filter(v => v > 1.0).sum(v => v))
}

[0.54, 0.99, 1.4399999][0.54, 0.99, 1.4399999]1.4399999

• Chained in one expression, xs.map(…).map(…) runs as one pass on the GPU: each item goes through both functions, and the list between them is never made, copied back or stored.
• In two statements, the list in the middle, lit, is made, and stays only in GPU memory: only the GPU reads it, so it is never copied back.
• Either way the numbers are the same. On a long list the filter at the end runs on the GPU too, and the sum of floats on the CPU.

A loop of maps

A map's result stays on the GPU until the CPU reads it. So a loop of 100 maps copies the list back once, at the first read after the loop. A sum, min or max of whole numbers runs on the GPU too, and only its result comes back, even from xs.map(f).sum(g) in one expression:

gpu fn step(x: Int32) -> Int32 {
    (x * 7 + 3) % 1_009
}

fn main() {
    let xs: [Int32] = []
    for i in 0..1_000 {
        xs.push(Int32(i))
    }
    for _ in 0..100 {
        xs = xs.map(x => step(x))
    }
    print(xs[7])
    print(xs.sum(x => x))
    print(xs.max() ?? 0)
}

6935036081008

A loop that adds up such sums doesn't wait for each one: see Sums in a loop.

You don't see any of this: every read sees the list's items. A panic in a map keeps its message and line, but may show later: see Same results as the CPU.

A GPU adds the items in another order, so it checks every partial sum. If any running sum, added in order as the CPU would, leaves the type, the CPU adds the list again in order, so the panic names the same step:

fn main() {
    const xs: [Int32] = [2_147_483_000, 600, 47, 1]
    print(xs.sum(x => x))
}

panic: integer overflow: 2147483647 + 1 doesn't fit in Int32

For speed on the GPU, keep the CPU out of the middle: chain maps in one expression, and read a GPU list only when you need its items. The less a list travels between GPU and CPU, the faster.

A file bigger than a chunk

A for chunk in fs.chunks(path, size) loop with pieces of 1 MB or more reads ahead: in a native program, the next piece is read on a helper thread while the loop works on this one. There is nothing to write. A 2 GB file of Int32s, 32 chunks of 64 MB, on Windows. Each row ran 5 times, reading ahead off and on in turns, in ms:

• The file was already in the system's cache, so the disk was as fast as it gets. A file read from the disk itself isn't measured.
• The gain on the 200 steps is larger than the disk's share of the time explains, and that isn't understood yet.

A loop that adds up what the GPU works out doesn't wait for each sum either: see Sums in a loop. Left of the plan for data streaming through the GPU: as_list as a view when the chunk is still held, and network chunks.

Helpers and loops

use std.math

gpu fn square(x: Float32) -> Float32 {
    x * x
}

gpu fn hypotenuse(a: Float32, b: Float32) -> Float32 {
    math.sqrt(square(a) + square(b))
}

fn main() {
    print(hypotenuse(3.0, 4.0))
}

5.0

gpu fn steps(start: Int32) -> Int32 {
    let n = start
    let count = 0
    while n != 1 {
        if n % 2 == 0 {
            n = n / 2
        } else {
            n = 3 * n + 1
        }
        count += 1
    }
    count
}

fn main() {
    const starts: [Int32] = [6, 7, 27]
    print(starts.map(s => steps(s)))
}

[8, 16, 111]

Refused when compiling

gpu fn first(p: Ptr<Float32>) -> Float32 {
    1.0
}

error: `p` is a `Ptr<Float32>`, and a `gpu fn` works with `Float32`, `Int32`, `UInt32` and `Bool`, and records and tuples of 1 to 8 of them (spec 15.1)

gpu fn bump(inout x: Float32) {
    x = x + 1.0
}

error: a `gpu fn` can't take `inout` parameters: give the new value back instead, as `gpu fn bump(x: Float32) -> Float32` (spec 15.1)

use std.fs

gpu fn load(x: Float32) -> Float32 {
    const text = fs.read_text("gains.txt") catch _ => ""
    x
}

error: a `gpu fn` can't use `fs.read_text`: it reaches outside the program (files, the network, the terminal, time, AI), and on the GPU a `gpu fn` runs alone, item by item, with only its own values (spec 15.1)

gpu fn factorial(n: Int32) -> Int32 {
    if n <= 1 {
        return 1
    }
    n * factorial(n - 1)
}

error: `factorial` calls itself, directly or through other `gpu fn`s: a GPU has no call stack (spec 15.1)

gpu fn half(x: Int32) -> Int32 {
    if x < 0 {
        fail Error("negative")
    }
    x / 2
}

error: a `gpu fn` can't use `try`, `catch` or `fail`: it can't fail, since on the GPU there is no error to give back. Check the value first with `if`, and give a result for every item (spec 15.1)