Publications of Torsten Hoefler

Jiwoong Sohn, Tomasz Sternal, Kenneth Styppa, Torsten Hoefler, Michael Moor:

Process Reward Agents for Steering Knowledge-Intensive Reasoning

(ICML. presented in Seoul, South Korea, Jul. 2026, Just Accepted )

Publisher Reference

Abstract

Reasoning in knowledge-intensive domains remains challenging as intermediate steps are often not locally verifiable: unlike math or code, evaluating step correctness may require synthesizing clues across large external knowledge sources. As a result, subtle errors can propagate through reasoning traces, potentially never to be detected. Prior work has proposed process reward models (PRMs), including retrieval-augmented variants, but these methods operate post hoc, scoring completed trajectories, which prevents their integration into dynamic inference procedures. Here, we introduce Process Reward Agents (PRA), an inference-time method for providing domain-grounded, online, step-wise rewards to a frozen policy. In contrast to prior retrieval-augmented PRMs, PRA enables search-based decoding to rank and prune candidate trajectories at every generation step. Experiments on multiple medical reasoning benchmarks demonstrate that PRA consistently outperforms strong baselines, achieving 81.9% accuracy on MedQA with Qwen3-4B, a new state of the art at the 4B scale. Importantly, PRA generalizes to unseen frozen policy models ranging from 0.5B to 8B parameters, improving their accuracy by up to 25.7% without any policy model updates. More broadly, PRA suggests a paradigm in which frozen reasoners are decoupled from domain-specific reward modules, allowing the deployment of new backbones in complex domains without retraining. All code and data are publicly available at https://process-reward-agents.github.io/.

Documents

Publisher URL: https://arxiv.org/abs/2604.09482download article:

BibTeX

@article{tsternalPRA2026,
  author={Jiwoong Sohn and Tomasz Sternal and Kenneth Styppa and Torsten Hoefler and Michael Moor},
  title={{Process Reward Agents for Steering Knowledge-Intensive Reasoning}},
  journal={ICML},
  institution={ETH Zurich},
  year={2026},
  month={Jul.},
  location={Seoul, South Korea},
  note={Just Accepted},
  source={http://www.unixer.de/~htor/publications/},
}


serving: 216.73.216.232:2631	© Torsten Hoefler