Fluid: compare value type and generation order
An interactive field guide
Fluid separates two design choices: discrete versus continuous tokens, and raster-causal versus random-bidirectional generation.
Before this article: MAR · VQGAN
What was already possible
Section titled “What was already possible”Earlier systems often changed tokenizer, output head, and generation order together. This makes causal attribution difficult. Fluid organizes a large text-to-image study around a two-by-two design, then scales the strongest configuration under its training protocol.
What changes, and why
Section titled “What changes, and why”The discrete branch uses VQGAN codes and categorical cross-entropy. The continuous branch uses SD-VAE latents grouped into 16D tokens and a six-layer diffusion MLP. A frozen T5-XXL text encoder plus a trainable six-layer aligner supplies cross-attention conditioning. Random-order continuous modeling performs best in the reported study.
Read from left to right. Select a module to connect its role, tensor shape, and calculation. Snippets are teaching pseudocode. Sources provide the paper method or audited implementation.
Two token interfaces
The same position count hides a different payload.
tokens = vqgan.encode(image) if discrete else group_2x2(vae.encode(image))Text conditioning
Cross-attention supplies text context to the image model.
text_features = aligner(frozen_t5(prompt))
h = transformer(tokens,text_features,order_mask)Output distribution
Choose the head appropriate to the value space.
loss = cross_entropy(head(h),ids) if discrete else diffusion_loss(h,continuous_target)Work through the mechanism
Section titled “Work through the mechanism”Switch either axis in the design explorer. The representation setting changes the target space and output head. The order setting changes what can be observed during prediction. This is a structural comparison; it does not draw an invented quality surface between the four experiments.
The continuous branch takes a 32 × 32 × 4 grid and groups 2 × 2 positions into 256 tokens of width 16. This preserves 4096 scalar values. The discrete branch uses 256 IDs from an 8192-entry vocabulary. Equal sequence length therefore does not establish equal information rate.
Change a parameter, inspect the computed values, then compare this small example with the model described above.
Analytical teaching model · controls change the example, not a trained research model.
These axes change architecture and factorization. No interpolated benchmark scores are implied.
| Quantity | Value |
|---|---|
| Value space | 16-dimensional continuous vectors |
| Output head | Conditional diffusion MLP |
| Context rule | Known positions can interact bidirectionally |
Training is not sampling
Section titled “Training is not sampling”The continuous head trains on 1000 diffusion timesteps and samples with 100 steps. The main study scales to 10.5B parameters. COCO FID and GenEval use different evaluation sets, and guidance/temperature are tuned per metric. Do not assume their best reported numbers came from one fixed sampler.
What this does not establish
Section titled “What this does not establish”This is not a pure equal-rate discrete-versus-continuous ablation because tokenizers and reconstruction objectives differ. The suggested connection between learned position embeddings and raster artifacts is a hypothesis, not a controlled 2D-RoPE result. Its large-batch TPU throughput also does not measure single-image GPU latency.
Check your understanding
Section titled “Check your understanding”Which two axes must remain separate when interpreting Fluid?
The token value space and the generation order/attention rule. A change to one does not logically require a change to the other.
Sources and next steps
Section titled “Sources and next steps”Diagrams and snippets are teaching reconstructions. Paper reports, implementation observations, and analytical examples are distinguished. No research model runs in this page.