We propose that reinforcement learning (RL) from partial expert demonstrations is not merely a training heuristic, but a promising framework for solving complex sequence generation tasks. Supervised fine-tuning (SFT) relies on dense ground-truth labels, which become increasingly costly as sequence length grows. RL, on the other hand, struggles with sparse rewards and a combinatorially large output space. We address this by introducing adaptive backtracking (AdaBack), a per-sample curriculum learning algorithm that reveals only a partial prefix of the target output during training. The… Read More
RL for Reasoning by Adaptively Revealing Rationales
We propose that reinforcement learning (RL) from partial expert demonstrations is not merely a training heuristic, but a promising framework for solving complex sequence generation tasks. Supervised fine-tuning (SFT) relies on dense ground-truth labels, which become increasingly costly as sequence length grows. RL, on the other hand, struggles with sparse rewards and a combinatorially large output space. We address this by introducing adaptive backtracking (AdaBack), a per-sample curriculum learning algorithm that reveals only a partial prefix of the target output during training. The…
Related Posts
Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
Deep learning models excel in stationary data but struggle in non-stationary environments due to a…
The Java Developer’s Dilemma: Part 3
This is the final part of a three-part series by Markus Eisele. Part 1 can…
7 Feature Engineering Tricks for Text Data
An increasing number of AI and machine learning-based systems feed on text data — language…