[UCLA RL-LLM] Chapter 1.4: Deep policy gradient methods (PPO, GRPO)
Ernest Ryu · 73:29
A2C/A3C are computationally convenient but sample-inefficient: each transition yields at most one stochastic gradient and is then discarded. TRPO and PPO extract more from the same trajectories by optimizing a first-o...