DiTo: learn the tokenizer through denoising
An interactive field guide
A compact code can condition an iterative decoder. DiTo trains that code with a diffusion objective.
Before this article: Diffusion and flow matching · Variational autoencoders
What was already possible
Section titled “What was already possible”VQGAN combines reconstruction, perceptual, and adversarial terms. Latent diffusion often trains its image tokenizer separately from its generative backbone. DiTo asks whether a diffusion reconstruction objective can itself train a useful tokenizer, without that loss mixture.
What changes, and why
Section titled “What changes, and why”The encoder produces a continuous spatial latent. A convolutional U-Net receives the noisy image, the upsampled latent, and time. Encoder and decoder train jointly with diffusion L2 / flow matching. The studied version removes KL regularization and uses LayerNorm. Despite its name, DiTo is not a DiT-only architecture.
Read from left to right. Select a module to connect its role, tensor shape, and calculation. Snippets are teaching pseudocode. Sources provide the paper method or audited implementation.
Continuous encoder
A convolutional encoder produces normalized features without the VAE KL term.
z = normalize(encoder(x))Synchronized corruption
This is an optional training branch, applied with probability 0.1.
if use_sync:
z = add_noise(z, tau)
t = uniform(tau,1)Conditional U-Net
Pixel reconstruction requires an iterative solver.
v = unet(concat(xt,upsample(z)),t)
loss = squared_error(v, target_velocity)Work through the mechanism
Section titled “Work through the mechanism”The default bottleneck has spatial downsampling 8 and 4 channels. For 256² input this gives 32 × 32 × 4 values. These values describe the input image; the decoder also receives fresh pixel noise. Reconstruction therefore combines information stored in the code with a learned conditional image distribution.
Noise synchronization sometimes corrupts the latent at time τ, then samples pixel noise time t from [τ,1]. The demo marks combinations with t below τ as invalid for this branch. Higher t means more noise under the shown convention. This is a training constraint, not a requirement that the two noise samples be numerically identical.
Change a parameter, inspect the computed values, then compare this small example with the model described above.
Analytical teaching model · controls change the example, not a trained research model.
The straight path has known velocity. A learned model must estimate velocity; this demo does not measure its error.
| Quantity | Value |
|---|---|
| Constraint t ≥ τ | satisfied |
| τ | 0.300 |
| t | 0.600 |
| x_t = (1 − t) × 2 + t × (−1) | 0.200 |
| v = ε − x | -3 |
Training is not sampling
Section titled “Training is not sampling”The paper applies noise synchronization with probability 0.1. Its stated default training uses 300k iterations and batch 64. Reported reconstruction uses a 50-step Euler ODE decoder. These calls must be included when comparing with a one-pass image decoder. A downstream latent generator remains a separate component.
What this does not establish
Section titled “What this does not establish”This article follows the versioned paper; no official code path was audited for DiTo. Its reconstruction and generation experiments do not establish text alignment or visual question answering. A simpler objective does not by itself imply cheaper inference.
Check your understanding
Section titled “Check your understanding”If τ=0.7 and t=0.3, is this a valid noise-synchronization example?
No. This training branch samples t at or above τ. The demo deliberately exposes invalid choices so the constraint is visible.
Sources and next steps
Section titled “Sources and next steps”Diagrams and snippets are teaching reconstructions. Paper reports, implementation observations, and analytical examples are distinguished. No research model runs in this page.