Visuals have meanings.

Sequence state

Actual attention keys and values retained per layer for a token prefix and its positions. This declaration represents KV state, not arbitrary recurrent state; weights and cached final answers are separate.

KVState: KVState.token_ids, KVState.keys, KVState.values, KVState.precision

State artifact

An identified snapshot or reusable portion of sequence state, tied to the computation and input history that produced it. Logical identity is distinct from physical placement.

KVState: KVState.checkpoint_sha256, KVState.tokenizer_sha256, KVState.token_ids, KVState.keys, KVState.values

Inference system

Composition of the implemented execution components and system-wide limits; not an algorithm or a generic lab subject.

InferenceSystem: KVResidencySystemConfig.scheduling, KVResidencySystemConfig.kv_cache, KVResidencySystemConfig.transfer, KVResidencySystemConfig.compute_service

Request

One invocation carrying input and per-request generation settings, with identity and declared arrival. An agent turn may issue multiple invocations; cancellation is not represented by the current declaration.

InferenceRequest: InferenceRequest.text, InferenceRequest.generation, RequestArrival.request_id, RequestArrival.arrival_us

Workload

An ordered request collection supplied to execution.

Workload: Workload.requests

Response

Observed completion and publication counts; no network receipt is implied.

InferenceResponse: RunOutcome.published_tokens, RunOutcome.stop_reason, RunOutcome.complete

Event

A cache-lab occurrence linked to its causes in declared virtual microseconds, not measured wall time.

ExperimentEvent: CacheEvent.event_id, CacheEvent.causes, CacheEvent.kind, CacheEvent.time_us

Hardware backend

The actual compute and memory backend recorded by execution; it does not establish a target-accelerator performance model.

ExecutionBackend: ExecutionBackend.compute_device, ExecutionBackend.device_state_storage, ExecutionBackend.host_state_storage, ExecutionBackend.library, ExecutionBackend.precision

Tokenizer

Encoding produces token IDs and byte-span evidence under an identified vocabulary and codec.

Tokenizer: Encoding.ids, Encoding.spans, Encoding.input_sha256, Encoding.manifest_sha256, Encoding.codec_version

Detokenizer

The recorded output IDs and their diagnostic display under the identified tokenizer. Selected IDs remain visible in full. Streaming text exposes only complete UTF-8; a valid pending byte triple or UTF-8 suffix is not an error until generation ends. Malformed output remains diagnostic; rendering never repairs or filters token choices.

Tokenizer: Token.id, Token.piece, Token.kind

Embedding

Look up token vectors and combine the declared input representations.

Embedding: EmbeddingOperation.name

Positional encoding

Represent sequence position according to the model architecture; the current implementation uses learned absolute vectors.

PositionEncoding: PositionEncodingOperation.name

Normalization

Normalize channels using the declared statistics, epsilon, scale, and bias.

Normalization: NormalizationOperation.name

Attention

Query and mix sequence information. The declared GPT operations implement causal softmax attention.

Attention: AttentionOperation.name

Residual addition

Add a branch output to its preserved residual input.

ResidualAddition: ResidualAdditionOperation.name

Feed forward

Apply per-position channel projections and the declared activation.

FeedForward: FeedForwardOperation.name

Output projection

Project final hidden vectors to vocabulary scores.

OutputProjection: OutputProjectionOperation.name

Work item

An executed KV model computation, identified by work ID and linked to input IDs, tensors, and before/after KV states. Transfers are separate events, not instances of this computation record.

ModelExecution: CacheComputation.work_id, CacheComputation.input_ids, CacheComputation.nodes, CacheComputation.state_before, CacheComputation.state_after

Token selection

Inspect the executed choice from target logits. The current algorithm is greedy; draft verification is not implemented.

Decoding: TokenSelection.raw_logits, TokenSelection.selected_id, TokenSelection.probabilities

Queue

Pending requests wait for admission slots and KV reservations. The active queue rotates requests while service remains serialized.

RequestScheduling: RequestSchedulingConfig.max_active_requests

Residency

KV residency is managed within logical device and host budgets. A transfer must complete before a remote payload is usable.

KVCache: KVCacheConfig.device_kv_bytes, KVCacheConfig.host_kv_bytes, KVCacheConfig.retention_policy

Link

Hypothetical bandwidth and startup delay for serialized KV transfers. This is not a fabric topology or contention model.

StateTransfer: StateTransferConfig.link_bytes_per_us, StateTransferConfig.link_latency_us

Resource

Hypothetical compute and memory-service rates, distinct from physical backend measurements and cache capacity.

ComputeService: ComputeServiceConfig.compute_flops_per_us, ComputeServiceConfig.memory_bytes_per_us

Client

Customer submissions, output consumption and cancellation.

CustomerInteraction: CustomerInteraction.actions

ModelArchitecture

Dimensions and numerical conventions that define the supported GPT computation.

Fixed within an identified model artifact.

Owns: Dimensions and numerical conventions that define the supported GPT computation.; Declared explanatory stage templates for supported runtimes; these are not captured observations or timing.

Excludes: request text; generation limits; inspection position

KVState

Immutable actual keys and values for a prefix, bound to checkpoint, tokenizer, positions, and precision.

Immutable snapshot; continuation creates a new snapshot. Residency is separate.

Owns: Immutable actual keys and values for a prefix, bound to checkpoint, tokenizer, positions, and precision.

Excludes: resource placement; cached answers; service timing

ModelArtifact

An immutable inference artifact binding numerical weights to architecture, tokenizer identity, and preparation provenance.

Immutable for an artifact identity. Preparation seed is provenance, never a request control.

Owns: The full inference payload and its lightweight identity reference.; Preparation provenance attached to the artifact.

Excludes: request text; generation limits; inspection position

ModelWeights

Numerical tensors consumed by the model: embeddings, projection weights, biases, and normalization scales.

Immutable within its identified artifact or recorded execution.

Owns: Numerical tensor values indexed by model operation; block tensors have an explicit layer axis.

Excludes: Architecture dimensions and execution conventions; Request and generation configuration; Per-request KV state

InferenceSystem

The configured execution boundary that accepts inference requests and produces their outputs.

Fixed for an identified system configuration.

Owns: Composition of separately owned execution, scheduling, cache, transfer and compute-service configurations.; System-wide supported request and output-token ceilings.; The shared containment hierarchy of ontology-owned logical components and mechanisms, with cross-component relationships.; Declared dispatch boundary; the baseline uses one direct in-process execution endpoint.

Excludes: Constituent mechanisms and their observations.; Caller-requested generation policy and response publication.; Model weights and measured results.; Containment does not claim separate processes, execution order or support for configuring every component.

GenerationPolicy

Per-request output-token budget and supported selection and termination rules.

Editable per request before submission; immutable in that request’s run binding.

Owns: Per-request output-token budget and selection and termination settings.; Recorded identity and explanation of the policy used by an execution.

Excludes: model initialization; system capacity; Backend execution strategies such as speculative drafting and verification; observed outcomes

InferenceRequest

One invocation with input and per-request generation settings, identified at its declared arrival.

Editable before submission; immutable in a run binding.

Owns: Request input and per-request generation settings.; Request identity and declared arrival time; arrival does not imply admission or execution.

Excludes: model initialization; system capacity; observed outcomes

Workload

An ordered collection of request arrivals; array order breaks equal-time ties.

Immutable in an identified execution record.

Owns: An ordered collection of request arrivals; array order breaks equal-time ties.

Excludes: Actual admission or execution order chosen by the scheduler; Generated outputs and measured completion times

InferenceResponse

The output published for one request and its observed completion outcome.

Immutable execution evidence, accumulated as output is published and the request finishes.

Owns: Selected token IDs published to the modeled output collector and their diagnostic display contribution.; Request completion, selection and publication counts, unused budget, and termination reason.

Excludes: Unpublished selections, including TURN_END, and any future speculative candidates; Internal tensors and KV state; Network receipt or acknowledgment; current publication is in-process publication.

RequestEndpoint

Accept requests and publish responses at the inference service boundary.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Request and response contracts, streaming and cancellation semantics.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Routing decisions, model computation, and customer-side receipt.; The boundary declaration alone provides no executable experiment or numerical evidence.

RequestRouter

Select an execution destination for an admitted service request.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Destination selection and affinity to model availability, capacity and resident state.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Scheduling work within a selected engine; no configurable router is implemented.; The boundary declaration alone provides no executable experiment or numerical evidence.

InferenceEngine

Coordinate request scheduling, model execution and decoding against a selected model artifact.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of request scheduler, tokenizer, model executor and sequence decoding.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Public API transport, physical device ownership, and claiming a separate process or vendor product.; The boundary declaration alone provides no executable experiment or numerical evidence.

StateManager

Coordinate the lifetime, residency and movement of inference state.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of sequence-state representation, KV cache management and state transfer.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Model equations, kernel implementation and fabric topology; no generic state service is implemented.; The boundary declaration alone provides no executable experiment or numerical evidence.

ResourcePool

The compute, memory and interconnect resources against which inference execution is constrained.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: Composition of the resource graph and declared compute-service assumptions.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Placement decisions, measured hardware calibration and claims that graph topology is executed.; The boundary declaration alone provides no executable experiment or numerical evidence.

ComputeKernel

An executable implementation of a bounded numerical operation on a compute device.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary for algorithm, precision, fusion and memory-access choices in an operation implementation.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: CUDA as a platform, model architecture, scheduler policy, and an implemented kernel selector.; The boundary declaration alone provides no executable experiment or numerical evidence.

DeviceRuntime

The execution services used to launch device work and manage its memory and synchronization.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary between model execution and device launch, memory and synchronization facilities.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: Compute-kernel algorithms, physical hardware capacity, and claiming a CUDA runtime in this browser.; The boundary declaration alone provides no executable experiment or numerical evidence.

CollectiveCommunication

Coordinate a communication operation across a declared group of execution participants.

A durable logical boundary in the declared system hierarchy; implementations and experiments may change.

Owns: The boundary for collective operation, algorithm and participant choices, such as all-reduce.; Its typed boundary in the shared configuration and explanation hierarchy.

Excludes: NCCL as a universal implementation, point-to-point state transfer, and any implemented collective simulation.; The boundary declaration alone provides no executable experiment or numerical evidence.

ExperimentSpec

One declared intervention on an ontology-owned mechanism or artifact in the inference journey.

Editable before execution; the consumed specification is frozen in the execution record.

Owns: One mechanism or artifact control and its requested settings.; Primary and related ontology subjects, resolved against admitted concepts and preserved in execution.

Excludes: Observed outcomes and execution identity; Causal events and numerical measurements

ExperimentExecution

One identified execution of the consumed controls against customer input, retaining its events and observations for inspection and replay.

Evidence accumulates during execution; the completed record is immutable.

Owns: Execution records joining the consumed specification, events, observations, and request outcomes.; Numerical traces and shared execution provenance identifying source bytes, Python/Git version, and the actual backend; provenance is distinct from requested system settings.; Resolved model, system and customer-input bindings actually consumed; these are frozen evidence, not an independent editable specification.

Excludes: Editable experiment drafts; Inspection cursor or playback position; Claims that a local run predicts production performance

ExperimentEvent

An occurrence within an execution, with explicit order, causal links, and the applicable clock interpretation.

Recorded when an occurrence happens; earlier events are not rewritten.

Owns: Causal cache event envelopes with request identity and virtual timestamps.; The current cache event envelope records cross-mechanism scheduling, cache and service transitions; it is not their semantic owner.

Excludes: A global sequential step shared by concurrent requests; Inferring causality from adjacency or equal timestamps; Treating virtual time or execution order as measured wall time; Response publication and completion payloads, owned by InferenceResponse

ExperimentObservation

Recorded tensors or local measurements attached to executed work; mechanism-specific decisions and computations have separate owners.

Captured during execution and retained unchanged with its evidence.

Owns: Recorded intermediate tensors and their dependencies.; Local wall-time measurements with their measurement boundaries.

Excludes: Requested settings and hypothetical service rates; Equating selected tokens with published response tokens; Treating a host measurement as target-accelerator performance

ExecutionSpec

A composition of experiment controls used together to execute customer input. The list specifies one setup, not a sweep.

Editable before execution; retained unchanged with the execution evidence.

Owns: Composition of ontology-bound controls, with one assignment per control.; Execution setup independently of customer input and observed results.

Excludes: Customer text and request arrivals, owned by Workload and InferenceRequest; Observed outcomes; Silently overriding conflicting controls

ExperimentDesign

A named investigation, or lab, that tests one question with ExecutionSpec settings, states which controls it still lacks, and carries its abstraction contract.

Authored with the repository; applying it replaces the editable ExecutionSpec before execution.

Owns: The question, ontology subjects, settings and missing controls of one experiment.; The abstraction contract and evidence references that bound its conclusions.

Excludes: Execution evidence and observed outcomes, owned by ExperimentExecution; Free-form pages or computations outside the declared execution operations

InspectionState

One selected position in an identified accepted token sequence, with one shared diagram/detail surface and architecture or observation selection. Selection and journey replay never execute inference.

Changes on inspection; reset when input is applied or output is reset. Execution identity is null until a computation exists.

Owns: Sequence position shared by token highlighting, derived recorded computation and the next submitted raw prefix.; Shared diagram/detail focus and journey selection, distinguishing declared architecture from captured observations.; Executed observation selection; inspecting an operand never changes the generation prefix.

Excludes: input text; model or generation configuration

ApiError

An explicit transport admission or execution failure message.

Produced for a rejected or failed service operation; a completed execution record need not exist.

Owns: An explicit transport admission or execution failure message.

Excludes: Successful inference output; A structured recovery or retry policy; Evidence that an experiment completed

ExecutionBackend

The runtime, kernel and communication implementations used to execute model work; recorded backend evidence identifies actual compute, storage, library and precision.

Fixed in the recorded run; a different backend requires a new run binding.

Owns: Actual compute device, state-storage implementation, library, and precision observed for execution.

Excludes: hypothetical service rates; logical capacity budgets; worker placement policy

ResourceGraph

Hypothetical execution locations with local memory and directed point-to-point transfer links.

Immutable scenario input; changing it defines a different scenario.

Owns: Each node identifies one exclusive compute/local-memory budget; each directed link identifies one exclusive capacity.; Rates use FLOPs or bytes per virtual microsecond; capacity uses bytes. Values are assumptions, not calibrated profiles.; A reverse link is independent capacity. Shared duplex pools or multiple aliases for the same physical resource are outside this initial representation.

Excludes: No routing, placement, resource scheduling, contention algorithm or topology execution is implemented.; No model/device compatibility, target-model scaling, weight/workspace allocation or vendor performance claim.

Tokenizer

The identified vocabulary and codec that frame conversation input and map source bytes to inspectable token IDs.

Vocabulary and codec identity are fixed for an encoding; turns are inputs and spans/results are recorded evidence.

Owns: Vocabulary entries, role-tagged framing inputs, and the resulting token sequence.; Source-byte/token spans and identities required to explain and reproduce encoding.; Identity of the token-ID input supplied to model computation; these IDs are not embedding vectors.

Excludes: Embedding vectors and model computation; Generation policy and serving decisions; Diagnostic output display is not a guarantee that arbitrary generated IDs form valid text.

Embedding

Look up token vectors and combine the declared input representations.

A model operation identity; observations are captured per execution.

Owns: Look up token vectors and combine the declared input representations.

Excludes: Positional vectors are supplied by PositionEncoding; later attention mixing is separate.

PositionEncoding

Represent sequence position according to the model architecture; the current implementation uses learned absolute vectors.

A model operation identity; observations are captured per execution.

Owns: Represent sequence position according to the model architecture; the current implementation uses learned absolute vectors.

Excludes: Does not establish extrapolation or compatibility when a mechanism is replaced.

Normalization

Normalize channels using the declared statistics, epsilon, scale, and bias.

A model operation identity; observations are captured per execution.

Owns: Normalize channels using the declared statistics, epsilon, scale, and bias.

Excludes: Does not own residual addition or feed-forward projections.

Attention

Query and mix sequence information, using accessible key/value history or a recurrent summary. The GPT baseline implements causal softmax attention.

A model operation identity; observations are captured per execution.

Owns: Query-based sequence mixing and the state-update rule of recurrent attention.

Excludes: KVState owns numerical K/V payloads; KVCache owns retention and reuse management.; Position mechanisms are owned by PositionEncoding.

ResidualAddition

Add a branch output to its preserved residual input.

A model operation identity; observations are captured per execution.

Owns: Add a branch output to its preserved residual input.

Excludes: Does not own either branch computation or normalization.

FeedForward

Apply per-position channel projections and the declared activation.

A model operation identity; observations are captured per execution.

Owns: Apply per-position channel projections and the declared activation.

Excludes: Does not mix token positions or own residual addition.

OutputProjection

Project final hidden vectors to vocabulary scores.

A model operation identity; observations are captured per execution.

Owns: Project final hidden vectors to vocabulary scores.

Excludes: Does not select, deliver, or decode output tokens.

ModelExecution

Execute model forward computations over input positions and available sequence state, producing logits and intermediate values.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Numerical forward work, its precision and completion evidence.; Declared full-prefix or cached execution implementation and numerical backend selection.

Excludes: Caller selection and stopping requirements.; Draft acceptance, output publication, request scheduling, and resource-service timing.

Decoding

Use model predictions to advance a generated sequence while honoring the requested generation policy.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Prediction-step evidence and the selected token; the implemented algorithm is autoregressive greedy selection.; The algorithmic boundary for future draft proposal, target verification, acceptance and rollback.

Excludes: Requested generation policy, owned by GenerationPolicy.; Forward tensor equations, response transport and inter-request scheduling.; Declaring this boundary does not implement speculative decoding or distribution-preserving sampling.

RequestScheduling

Decide which arrived requests may become active and which eligible work runs next under resource constraints.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Active-request admission slots; the current KV engine admits FIFO and rotates active requests between serialized service steps.

Excludes: Arrival declarations are workload input, not a scheduling decision.; KV allocation and eviction policy, tensor equations, and token selection.; No batching, priority, preemption or fair-sharing guarantee is implemented.

KVCache

Manage allocation, residency, retention and valid reuse of KV-state payloads under declared capacities.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Logical device and host KV budgets and exact whole-prefix LRU retention.; Cache validity, lookup, allocation and eviction decisions in the current KV lab.

Excludes: The numerical keys and values belong to KVState.; Attention equations, transfer service costs and request scheduling.; No partial-prefix paging or model-quality claim.

StateTransfer

Move an identified sequence-state payload across an execution or memory boundary before dependent work uses it.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Declared payload-transfer bandwidth and startup delay; current modeled duration is startup plus bytes divided by rate.; The handoff boundary for future lossless prefill-to-decode KV movement.

Excludes: Endpoint placement, routing and topology selection.; State interpretation or attention math; cache retention and eviction choices.; No vendor conversion, network contention or physical-performance calibration is represented.

ComputeService

Translate declared operation and memory-traffic counts into conditional service time under explicit resource rates.

Configuration is fixed for a run; observations identify the work that actually executed.

Owns: Hypothetical compute and memory rates; the KV lab charges the maximum of operation time and memory-traffic time.

Excludes: Numerical model results and actual device identity.; Measured physical performance, transfer service, and request scheduling.; No calibrated accelerator or multi-resource overlap model.

CustomerInteraction

Declared client submissions, cumulative output consumption and cancellation on one scenario clock.

Immutable scenario input; changing it defines a different scenario.

Owns: Ordered customer actions; equal-time array order breaks ties. Each submission creates one invocation.; Consumption describes client demand; cancellation expresses intent, not acknowledgment or immediate server termination.

Excludes: No browser rendering, API protocol, network delivery, retry grouping or agent tool orchestration is implemented.; These inputs are validated scenarios, not accepted by existing inference engines.