Glossary
Latent · 潜变量 / 潜表征
Section titled “Latent · 潜变量 / 潜表征”An internal representation. It can be a random variable in a probabilistic model or a deterministic feature array. State which meaning is used.
Token · 词元
Section titled “Token · 词元”One unit in a sequence. It may be a discrete ID or a continuous vector. A token is not necessarily one image patch.
Prior / posterior · 先验 / 后验
Section titled “Prior / posterior · 先验 / 后验”A prior models a latent before observing an input. A posterior conditions on the observed input. A learned encoder may approximate the true posterior.
Codebook · 码本
Section titled “Codebook · 码本”A finite table of vectors. Quantization chooses an entry, stores its index, and passes its vector to a decoder.
Stop-gradient · 停止梯度
Section titled “Stop-gradient · 停止梯度”An operation that preserves a value in the forward pass but blocks derivatives through that path.
Autoregressive · 自回归
Section titled “Autoregressive · 自回归”A model of conditional factors along an order. It does not require discrete values or a particular Transformer architecture.
Teacher forcing · 教师强制
Section titled “Teacher forcing · 教师强制”Use observed earlier targets as context during training. Free-running generation instead uses earlier model predictions.
Reconstruction / generation · 重建 / 生成
Section titled “Reconstruction / generation · 重建 / 生成”Reconstruction starts from an observed input and asks what survives encoding and decoding. Generation samples new content from a learned distribution, possibly with a condition.
CFG · 无分类器引导
Section titled “CFG · 无分类器引导”Combine conditional and unconditional model predictions during sampling. Guidance strength changes output quality, diversity, and possibly computation.
rFID / gFID
Section titled “rFID / gFID”FID applied to reconstructed or generated image sets. The evaluator and sample protocol matter; neither directly proves paired pixel accuracy or visual understanding.
Linear probe · 线性探针
Section titled “Linear probe · 线性探针”A linear classifier trained on frozen features. An attentive pooling head is a different, nonlinear protocol.
APE / RoPE
Section titled “APE / RoPE”APE supplies absolute position features. RoPE rotates query/key channel pairs according to coordinates, producing relative phase differences in attention.
Projection · 投影
Section titled “Projection · 投影”A learned matrix changes vector width. It does not necessarily reduce information or sequence length.
Softmax
Section titled “Softmax”Convert real scores to positive values that sum to one. Each output is exp(score) divided by the sum of exponentials.
Cross-entropy · 交叉熵
Section titled “Cross-entropy · 交叉熵”For one observed class k, the loss is −log p(k). Assigning more probability to the correct class lowers the loss.
KL divergence · KL 散度
Section titled “KL divergence · KL 散度”An expected log-density ratio between distributions. It is nonnegative and is generally asymmetric. Specify which distribution is first.
ELBO · 证据下界
Section titled “ELBO · 证据下界”A lower bound on log marginal likelihood. A VAE maximizes expected reconstruction log-likelihood minus KL from its approximate posterior to its prior.
LayerNorm · 层归一化
Section titled “LayerNorm · 层归一化”Normalize a feature vector using its mean and variance, then optionally apply learned scale and shift. This differs from dataset-level latent standardization.
Residual connection · 残差连接
Section titled “Residual connection · 残差连接”Add a transformed branch to its input: y=x+F(x). A gate can control the branch contribution. This is unrelated to residual quantization.
A paired perceptual distance based on learned visual features. Its network and preprocessing are dependencies, not raw pixel arithmetic.
GAN · 对抗训练
Section titled “GAN · 对抗训练”Train a discriminator to distinguish real from generated samples, while training a generator to produce samples that fool it. Loss variants differ.
NFE · 网络调用次数
Section titled “NFE · 网络调用次数”Number of function evaluations used by a sampler. Specify which network is evaluated; one small MLP call and one full Transformer call have different costs.
ODE / Euler · 常微分方程 / Euler
Section titled “ODE / Euler · 常微分方程 / Euler”An ODE specifies the time derivative of a state. Euler takes a step using the current derivative times a time interval. Finite steps generally introduce numerical error.
LFQ · 无查表量化
Section titled “LFQ · 无查表量化”Quantize feature coordinates into signs/bits instead of searching an explicitly stored vector table. Training still needs a gradient estimator and distribution-related objectives.
Straight-through · 直通估计
Section titled “Straight-through · 直通估计”Use a hard or discrete forward value but substitute a differentiable backward rule. It is a chosen estimator, not the exact derivative of the hard operation.
LoRA · 低秩适配
Section titled “LoRA · 低秩适配”Represent a weight update as the product of two narrow matrices, ΔW=BA. Rank controls the size of this update family.
Gram matrix · Gram 矩阵
Section titled “Gram matrix · Gram 矩阵”A matrix of inner products between feature channels or vectors. Matching it constrains correlations; it does not enforce exact pixel correspondence.
KV cache · KV 缓存
Section titled “KV cache · KV 缓存”Save earlier keys and values so causal decoding need not recompute unchanged history. Bidirectional context updates can invalidate this assumption.
Bits per pixel · 每像素 bit 数
Section titled “Bits per pixel · 每像素 bit 数”Divide a stated code payload by the number of image pixels. A nominal fixed-length count omits entropy coding, headers, model weights, and container overhead.
Temperature · 温度
Section titled “Temperature · 温度”A scale that changes the sharpness of a probability distribution or contrastive score. Its meaning depends on the exact formula and sampling implementation.
Guidance / rejection · 引导 / 拒绝采样
Section titled “Guidance / rejection · 引导 / 拒绝采样”Guidance changes sampling predictions. Rejection selects a subset of already generated candidates. Both can improve a score while changing diversity and cost.