Despite their sophisticated general-purpose capabilities, Large Language Models (LLMs) often fail to align with diverse individual preferences because standard post-training methods, like Reinforcement Learning with Human Feedback (RLHF), optimize for a single, global objective. While Group Relative Policy Optimization (GRPO) is a widely adopted on-policy reinforcement learning framework, its group-based normalization implicitly assumes that all samples are exchangeable, inheriting this limitation in personalized settings. This assumption conflates distinct user reward distributions and… Read More
Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
Despite their sophisticated general-purpose capabilities, Large Language Models (LLMs) often fail to align with diverse individual preferences because standard post-training methods, like Reinforcement Learning with Human Feedback (RLHF), optimize for a single, global objective. While Group Relative Policy Optimization (GRPO) is a widely adopted on-policy reinforcement learning framework, its group-based normalization implicitly assumes that all samples are exchangeable, inheriting this limitation in personalized settings. This assumption conflates distinct user reward distributions and…
Related Posts
Toward Machine Interpreting: Lessons from Human Interpreting Studies
Current speech translation systems, while having achieved impressive accuracies, are rather static in their behavior…
Generative and Contrastive Graph Representation Learning
Self-supervised learning (SSL) on graphs generates node and graph representations (i.e., embeddings) that can be…
Flow State to Free Fall: An AI Coding Cautionary Tale
When I was eight years old, I watched a mountaineering documentary while waiting for the…