ask<T>: When the AI fails
● Intermediate3 min
ask can fail: no key, a network or provider error, an answer that never fits the type, or a limit. Handle it like any error.
| Error | When |
|---|---|
NoKey(variable) | the environment variable with the key, ANTHROPIC_API_KEY, isn't set |
Failed(status, message) | the provider refused, or the network failed (status 0) |
InvalidOutput(answer) | the answer still didn't fit the type after asking again |
Denied(reason) | a limit on Ai refused the call |
ask fails with an AiError, so match picks the cases you can handle. Here an answer that never fits becomes a safe fallback, and anything else is passed on with fail e:
type Verdict {
/// true if the review is positive
positive: Bool
}
agent Reader {
can Ai, Console
on review(text: String) ! AiError {
const v = ask<Verdict>("Is this review positive? {text}") catch e => match e {
AiError.InvalidOutput(_) => Verdict(positive: false),
_ => fail e,
}
print("positive: {v.positive}")
}
}
fn main() {
const bot = Reader.start()
bot.review("Arrived late, but works well") catch e => print("no answer: {e}")
}The program ran and printed:
kavach run ask-fail.kav
positive: true[dry run] ask("Is this review positive? Arrived late, but works well") -> Verdict(positive: true)
Put a limit Ai on an agent to cap its model calls, count <= 1_000 per day, or the tokens they take, tokens <= 200_000 per day: past it, ask fails with Denied instead of running up a bill. See limit.
A busy provider (status 429, 529 or 5xx) or a dropped connection isn't a failure yet: the call is sent again after a wait that doubles each time (1, 2, 4 and 8 seconds, then 8 again), as the providers' own libraries do, and only then fails with Failed. Each wait is a random time between half and all of that, so calls turned away together don't all come back together.
Many asks at once, one per task, can meet a provider's limit on calls at the same time. The program's asks share a gate. After a call is turned away as busy, no more calls are sent at once than the provider was taking then, and each call that succeeds lets a little more through, up to 256 at once. A call waiting at the gate isn't a retry. So 300 asks started together all get their answer, as fast as the provider allows.