ModelAdapter

interface ModelAdapter(source)

The framework's interface to a model backend.

Answers one request; retry and output validation live in the framework, with the retry schedule tuned through retryDecision.

Implementing this needs Kotlin — respond is a suspend function.

Properties

Link copied to clipboard

Whether the model can be used right now, and if not, why.

Link copied to clipboard

Emits whenever availability changes.

Functions

Link copied to clipboard
abstract suspend fun respond(request: ModelRequest): JsonValue

Answers a single request.

Link copied to clipboard
open fun retryDecision(usage: Usage<*>, error: Throwable, attempt: Int): RetryDecision

Decides what happens after a failed attempt.