Papers & projects
Research
I work on reinforcement learning, the reasoning of large language models, and trustworthy AI. Papers are listed newest first.
Papers
-
2026
Groundability, Not Scale Alone: When Weak Reviewers Can Audit Strong Coding Agents
Can a weaker model reliably decide whether a strong coding agent's patch actually solves its issue? Longer traces, more context and better-organized evidence do not make the judgment reliable. What does is groundability: whether an independent check, such as a runnable test or sound static analysis, can answer the disputed question. Given groundable evidence, weak reviewers catch the omissions that confident summaries hide, and reviewer size stops predicting quality. The open problem is producing such checks when official tests are not available.
-
2026
When Context Changes: Understanding Update Failures in LLMs
LLMs can answer with an old value of a variable even when the updated one is still in context, a failure we call stale binding. We introduce the Controlled In-Context Memory (CICM) benchmark to isolate it and trace it to attention drift: when attention scores are similar, several old values together outweigh the current one. Redirecting attention toward the current value at inference time, with no further training, corrects most of these errors.
-
2026
LLMs Should Express Uncertainty Explicitly
LLMs often give confident but wrong answers. We show that post-training can make a model's self-assessment explicit in its own response, and ask where that signal belongs: a verbalized confidence score once the answer is formed, or an uncertainty marker emitted mid-reasoning whenever the current step looks unreliable. Both sharply reduce overconfident errors while improving answer quality, through different mechanisms: the first sharpens a confidence structure already present in the pretrained model, the second teaches the model to flag high-risk reasoning steps.
-
2025
Don't Trade Off Safety: Diffusion Regularization for Constrained Offline RL
Constrained offline RL must learn safe, high-reward policies from a fixed dataset, with no unsafe exploration. DRCORL uses a diffusion model to capture the behavior policy in the data, distills it into a simple policy for fast inference, and applies gradient manipulation to balance reward against constraint satisfaction. The result reliably meets cost limits without giving up return or inference speed, and does so with the same hyperparameters across tasks.
-
2025
Meta Thinker: Thinking What AI Thinks
Reasoning strategies such as Chain-of-Thought and Tree-of-Thought help, but which one works depends on the problem. We ask whether an LLM can pick its own thinking style. After analyzing five reasoning paradigms across mathematical, logical and commonsense tasks, we propose a meta-thinking prompt algorithm that lets the model select, or synthesize, a reasoning style from the input itself, improving both accuracy and token efficiency.
-
2025
StyleBench: Evaluating Thinking Styles in Large Language Models
How well a reasoning style works depends on the model and the task, yet this interplay was poorly understood. StyleBench systematically evaluates five styles of thought across diverse tasks and open-source models of many sizes. The takeaway: no single style is universally best. Search-based styles pay off on open-ended problems but need large models, concise styles bring large efficiency gains on well-defined tasks, and small models often ignore the format and guess. Robust reasoning emerges with scale.
-
2023
Reinforcement Learning for SBM Graphon Games with Re-Sampling
Mean-field approximations assume homogeneous, fully connected populations, which rarely holds in practice. We study multi-population mean field games whose connections follow a stochastic block model, and handle the realistic case where that block structure is unknown by re-sampling agents from a graphon. The resulting Graphon Game with Re-Sampling model provably converges to the multi-population dynamics, and comes with a sample-based reinforcement learning algorithm that has finite-sample guarantees.
Under review
-
Optimal Portfolio Allocation in Incomplete Markets
Decomposes the optimal portfolio policy under CRRA utility and characterizes an investor-specific price of risk for incomplete markets. A backward Monte Carlo algorithm estimates that price of risk and computes the policy, with proven convergence rates and an extension to high dimensions.