This paper was accepted at the Workshop on Latent & Implicit Thinking – Going Beyond CoT Reasoning 2026 at ICLR.
Autoregressive language models trained with next-token prediction generate text by sampling one discrete token at a time. Although very scalable, this objective forces the model to commit at every step, preventing it from exploring or reflecting upon multiple plausible continuations. Furthermore, the compute allocation across tokens is uniform; every token is formed based on a single forward-pass, potentially limiting the model’s expressiveness in cases where difficult tokens… Read More
Thinking into the Future: Latent Lookahead Training for Transformers
This paper was accepted at the Workshop on Latent & Implicit Thinking – Going Beyond CoT Reasoning 2026 at ICLR.
Autoregressive language models trained with next-token prediction generate text by sampling one discrete token at a time. Although very scalable, this objective forces the model to commit at every step, preventing it from exploring or reflecting upon multiple plausible continuations. Furthermore, the compute allocation across tokens is uniform; every token is formed based on a single forward-pass, potentially limiting the model’s expressiveness in cases where difficult tokens…
Related Posts
How Agentic AI Empowers Architecture Governance
One of the principles in our upcoming book Architecture as Code is the ability for…
Start learning all things AI on the new Google Skills
Google Skills is a new home for building skills in AI, and learning about other…
Inferring Optical Tissue Properties from Photoplethysmography using Hybrid Amortized Inference
Smart wearables enable continuous tracking of established biomarkers such as heart rate, heart rate variability,…